Detection and Mitigation of Cyber Attacks on Time Synchronization Protocols for the Smart Grid

(1)

Detection and Mitigation of Cyber Attacks on Time

Synchronization Protocols for the Smart Grid

Bassam Moussa

A Thesis In

The Concordia Institute For

Information Systems Engineering

Presented in Partial Fulfillment of the Requirements For the Degree of

Doctor of Philosophy (Information and Systems Engineering) Concordia University

Montr´eal, Qu´ebec, Canada

September 2018 c

(2)

CONCORDIA UNIVERSITY SCHOOL OF GRADUATE STUDIES

This is to certify that the thesis prepared By: Bassam Moussa

Entitled: Detection and Mitigation of Cyber Attacks on Time Synchronization Protocols for the Smart Grid

and submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Information and Systems Engineering)

complies with the regulations of the University and meets the accepted standards with respect to originality and quality.

Signed by the final examining committee:

Chair Dr. Robin Drew

External Examiner Dr. Frederic Cuppens

External to Program Dr. Mustafa K. Mehmet Ali

Examiner Dr. Lingyu Wang Examiner Dr. Amr Youssef Examiner Dr. Marthe Kassouf Thesis Co-Supervisor Dr. Chadi Assi Thesis Co-Supervisor Dr. Mourad Debbabi Approved by Dr. Anjali Awasthi,Graduate Program Director

(3)

ABSTRACT

Detection and Mitigation of Cyber Attacks on Time Synchronization Protocols for the Smart Grid

Bassam Moussa, Ph.D. Concordia University, 2018

The current electric grid is considered as one of the greatest engineering achie-vements of the twentieth century. It has been successful in delivering power to con-sumers for decades. Nevertheless, the electric grid has recently experienced several blackouts that raised several concerns related to its availability and reliability. The aspiration to provide reliable and efficient energy, and contribute to environment pro-tection through the increasing utilization of renewable energies are driving the need to deploy the grid of the future, the smart grid. It is expected that this grid will be self-healing from power disturbance events, operating resiliently against physical and cyber attack, operating efficiently, and enabling new products and services. All these call for a grid with more Information and Communication Technologies (ICT). As such, power grids are increasingly absorbing ICT technologies to provide efficient, secure and reliable two-way communication to better manage, operate, maintain and control electric grid components.

On the other hand, the successful deployment of the smart grid is predicated on the ability to secure its operations. Such a requirement is of paramount importance especially in the presence of recent cyber security incidents. Furthermore, those inci-dents are subject to an augment with the increasing integration of ICT technologies and the vulnerabilities they introduce to the grid. The exploitation of these vulne-rabilities might lead to attacks that can, for instance, mask the system observability and initiate cascading failures resulting in undesirable and severe consequences.

In this thesis, we explore the security aspects of a key enabling technology in the smart grid, accurate time synchronization. Time synchronization is an immense requi-rement across the domains of the grid, from generation to transmission, distribution, and consumer premises. We focus on the substation, a basic block of the smart grid

(4)

system, along with its recommended time synchronization mechanism - the Precision Time Protocol (PTP) - in order to address threats associated with PTP, and propose practical and efficient detection, prevention, mitigation techniques and methodologies that will harden and enhance the security and usability of PTP in a substation. In this respect, we start this thesis with a security assessment of PTP that identifies PTP security concerns, and then address those concerns in the subsequent chapters. We tackle the following main threats associated with PTP: 1) PTP vulnerability to fake timestamp injection through a compromised component 2) PTP vulnerability to the delay attack and 3) The lack of a mechanism that secures the PTP network. Next, and as a direct consequence of the importance of time synchronization in the smart grid, we consider the wide area system to demonstrate the vulnerability of relative data alignment in Phasor Data Concentrators to time synchronization attacks. These problems will be extensively studied throughout this thesis, followed by discussions that highlight open research directions worth further investigations.

(5)

Acknowledgments

I would like to express my gratitude and acknowledgment to those who supported me to start and finish my PhD studies.

I am grateful for my supervisors, Prof. Chadi Assi and Prof. Mourad Debbabi, for their selfless support and guidance through this endeavor. I would have never made it without your consistent motivation, exemplary supervision, great guidance, and unconditional support. I specially appreciate your vision in seeing how pieces of this research developed and connected to each other. I am thankful for giving me the full experience any PhD student could aspire to have, including attending conferences, being involved with industry, and participating in developing grants. I would like to express my sincere gratitude to Dr. Marthe Kassouf for all the time she dedicated to meetings, thoughtful discussions and valuable feedback. Her contribution and support was essential to improve the quality of my thesis.

I am also thankful to the members of my supervisory committee: Prof. Mus-tafa Mehmet Ali, Prof. Amr Youssef, and Prof. Lingyu Wang, for their valuable constructive feedback and insightful suggestions. Also, it is a pleasure to truly ackno-wledge Prof. Fr´ed´eric Cuppens for accepting to serve as a delegate in my Ph.D. thesis examining committee.

Furthermore, I am thankful to all of my colleagues in the research lab at Concordia University including those who have left to their next adventure, and those who are still around. Thank you for the valuable discussions, help, support, and warm caring conversations. It was a pleasure to share all those moments with you.

I am blessed with precious friends, here in Montreal and back at home, to whom I am indebted for their unconditional love, support and valuable advice. Thank you for your encouragement ever since pursuing a PhD was just an idea.

I would like to express my deep gratitude to my family. To my parents, you are a true blessing in my life, you gave more than anyone can ever give, unconditionally

(6)

and endlessly. I hope that this achievement of mine meets your expectations, makes you proud, and pays back but little of what you have always given. My brothers, sisters, nephews, and nieces, I love you all and dedicate this thesis to you. You are the reason I see the world a better place everyday.

Finally, to my beautiful wife Assile, you believed in me from day one and supported me throughout this long journey. You always inspire me to be a better person. For all the good times and the tough ones, I love you and will always do.

(7)

(8)

2.3.1 Overview . . . 21 2.3.2 PTP Security Extension . . . 25 2.4 PTP Security Assessment . . . 25 2.4.1 PTP in power grid . . . 26 2.4.2 Design . . . 27 2.4.3 Implementation . . . 31 2.4.4 PTP MACs . . . 33 2.5 Standardization Efforts . . . 34 2.5.1 IEEE C37.238 . . . 34 2.5.2 IEC 61850 . . . 36 2.5.3 PTP Gap Analysis . . . 38

Chapter 3 Securing The Precision Time Protocol Against Fake Times-tamps 43 3.1 Introduction . . . 43 3.1.1 Novel contributions . . . 45 3.2 System Model . . . 46 3.2.1 Problem Definition . . . 47 3.3 Threat Model . . . 48 3.4 Approach . . . 49 3.4.1 Feedback Introduction . . . 49 3.4.2 Feedback Contents . . . 50 3.4.3 Periodicity of Feedback . . . 52 3.4.4 SNMP Introduced Objects . . . 52 3.4.5 Detection Logic . . . 53 3.5 Evaluation . . . 54 3.5.1 Network Overhead . . . 54

3.5.2 Flase positives and negatives . . . 56

3.6 Experimental Results . . . 57

(10)

Chapter 4 Detection and Mitigation of PTP Delay Attack in IEC 61850 Substation 60 4.1 Introduction . . . 60 4.1.1 Novel Contributions . . . 63 4.2 Related Work . . . 64 4.3 System Model . . . 65 4.4 Threat Model . . . 66

4.4.1 Delay Attack Tree . . . 67

4.4.2 Delay Attack Analysis . . . 69

4.5 PTP Model . . . 71 4.5.1 PTP Master Model . . . 72 4.5.2 PTP Slave Model . . . 72 4.6 Detection Approach . . . 73 4.6.1 Detection Model . . . 74 4.7 Mitigation Approach . . . 76

4.7.1 Slave clock adjustment . . . 76

4.7.2 Maintaining slave clock synchronization . . . 79

4.8 Experimental Results . . . 80

4.8.1 PCTL Properties . . . 83

4.8.2 Experimental Results . . . 85

4.9 Conclusion . . . 89

Chapter 5 An Extension to the Precision Time Protocol to Enable the Detection of Cyber Attacks 90 5.1 Introduction . . . 90 5.1.1 Novel Contributions . . . 93 5.2 System Model . . . 94 5.2.1 PTP Attack Surface . . . 95 5.3 Threat Model . . . 98 5.4 Detection Model . . . 98 5.4.1 Basic Blocks . . . 99 5.4.2 Approach Realization . . . 102 5.4.3 Approach Overhead . . . 106 5.5 Attack Detection . . . 107

(11)

5.5.1 Attack on GMC . . . 107

5.5.2 Delay Attack on Communication Network . . . 108

5.5.3 Attack on Transparent Clock . . . 109

5.5.4 Attack on slave clock . . . 111

5.6 Approach Evaluation . . . 112

5.6.1 Approach Modeling and Verification . . . 112

5.6.2 Simulation Results . . . 115

Chapter 6 Exploiting The Vulnerability of Relative Data Alignment in Phasor Data Concentrators to Time Synchronization Attacks 121 6.1 Introduction . . . 122 6.1.1 Novel Contributions . . . 124 6.2 System Model . . . 125 6.3 Problem Definition . . . 127 6.4 Threat Model . . . 131 6.5 Problem Formulation . . . 132 6.5.1 Nomenclature . . . 132 6.5.2 LP Formulation . . . 134 6.6 Countermeasures . . . 138 6.7 Experimental Results . . . 139 6.7.1 Numerical Evaluation . . . 140

6.7.2 HIL Simulation Case Study . . . 144

Chapter 7 Discussion and Future Directions 148 7.1 Discussion . . . 148

7.2 Future Work . . . 151

7.2.1 PTP Usage in Wide Area . . . 152

7.2.2 Characterization of Attacks Impact on Power Systems . . . . 152

7.2.3 Time Synchronization Data Analysis . . . 153

Bibliography 154

(12)

Appendix B PTP Extension Validation 177

B.0.1 Verification Environment . . . 177 B.0.2 PTP Model . . . 178 B.0.3 Verified Properties . . . 183

(13)

List of Figures

2.1 Timing requirements for substation applications [111]. . . 10

2.2 Traveling wave fault locating . . . 14

2.3 End-to-end synchronization message exchange [4]. . . 23

2.4 Slave synchronization using peer-delay mechanism and TCs [106]. . . 24

2.5 Attacks on PTP time synchronization service . . . 27

3.1 Schema of IEC 61850 substation with NSM entities. . . 47

3.2 Feedback collection based onSync messages . . . 52

3.3 Detection time based on request period. . . 57

3.4 Detection time based on variable request size and period. . . 58

3.5 Traffic overhead based on request period. . . 59

4.1 An IEC 61850 smart grid substation architecture . . . 62

4.2 Delay attack tree . . . 68

4.3 Model of the master clock in PTP . . . 72

4.4 Model of the PTP slave clock . . . 73

4.5 Model of the guard clock . . . 75

4.6 Mitigation Model at the PTP slave clock. . . 80

4.7 PTP model execution . . . 82

4.8 Delay attack effect . . . 82

4.9 Delay attack detection . . . 82

4.10 Guard model execution . . . 82

4.11 Offset calculation at guard machine . . . 86

4.12 Offset calculation at slave machine . . . 87

4.13 Time difference at the two machines . . . 88

5.1 PTP attack surface for an IEC 61850 substation . . . 95

5.2 The state diagram of a slave clock . . . 100

(14)

5.4 Network clustering in the presence of extended PTP. . . 105

5.5 A sketch of the experimental network setup [106]. . . 116

5.6 Time needed to detect an attack. . . 116

5.7 Synchronization error at attack detection. . . 117

5.8 Synchronization error due to cyber attacks . . . 118

5.9 Network overhead due to the deployed extension. . . 119

6.1 Relative data alignment as defined by C37.244-2013 [7]. . . 126

6.2 Data frames dropped using relative data alignment. . . 130

6.3 Measurement delivery restrictions in presence of attack. . . 136

6.4 Post attack system observability. . . 145

6.5 Measurements delivered to control center. . . 146

B.1 The system clock component. . . 179

B.2 The slave component. . . 179

B.3 The master component. . . 181

B.4 The master to slave network component. . . 181

B.5 NTR model - master component. . . 182

(15)

List of Tables

2.1 Summary of time distribution mechanisms and their fulfillment of smart

grid applications requirements . . . 21

2.2 IEEE C37.238 main profile specifications . . . 35

2.3 Synchronization classes of IEC61850-5[57] . . . 37

2.4 Summary of PTP security analysis from the literature . . . 41

2.5 Summary of PTP security analysis from the literature (cont.) . . . . 42

5.1 Report message fields . . . 101

5.2 NTR Synchronization Thresholds . . . 105

6.1 PMU to PDC Connectivity . . . 139

6.2 Optimal PMU number and placement for IEEE test systems . . . 141

6.3 Attack Vector and impact for 14 bus system . . . 142

(16)

Abbreviations

PTP Precision Time Protocol

NTP Network Time Protocol

PMU Phasor Measurement Unit

PDC Phasor Data Concentrator

IEC International Electrotechnical Commission

IED Intelligent Electronic Device

GMC Grand Master Clock

PPS Pulse Per Second

IRIG Inter-Range Instrumentation Group

UTC Coordinated Universal Time

GNSS Global Navigation Satellite Systems

GPS Global Positioning System

SNMP Simple Network Management Protocol

MIB Management Information Base

NSM Network and System Management

BMCA Best Master Clock Algorithm

TC Transparent Clock

NTR Network Time Reference

CTL Computational Tree Logic

(17)

ICV Integrity Check Value

WAMS Wide Area Monitoring System

(18)

Chapter 1 Introduction

1.1 Overview and Motivation

The current power grid, a system engineered in the early twentieth century, is con-sidered outdated when contrasted to other systems tangled to our daily life and the technological advancements they witnessed. Its success in delivering power to consumers has been acceptable so far. However, its adequacy for future systems is questionable when considering the several blackouts experienced over the past years. This gave rise to the need for a self healing grid, a grid providing quality power for 21st century, clean and renewable energy, and active participation from customers. Those characteristics inspired the motive to integrate the advances in the information and communication technologies to define the grid of the future - the smart grid.

Today, the robust operation and the availability of the power grid is a critical requirement. The grid is exposed for threats both on its cyber and physical sides. Cyber-attacks are a consistent threat that is intensified with advances in the deploy-ment of the smart grid. The increased dependency on the communication network and the integration of both systems present a potential attack surface for cyber-attacks. Further, the physical components of the grid are subject to attacks targeting

(19)

their functionality, such as the reported attack on the high voltage transmission line in the United States in 2013 [113] and Canada in 2014 [24]. More recently, in the last week of 2015, a cyber attack targeted the Ukrainian power utilities and resulted in a blackout leaving hundreds of thousands of people without electricity for several hours[71]. Those blackouts demonstrate that our critical infrastructure is susceptible to faults and attacks that threaten its availability and functionality. The presence of such threats call for an innovative analysis of the functionality of the grid that results in a robust design of a smarter grid: a grid capable of restraining the effects of attacks and survive the loss of any of its components.

Moreover, control operations, effective monitoring and management of the grid require the presence of accurate synchronization of the grid events. Modern compo-nents are introduced to timestamp the observations they make about the grid status and the data they collect about its conditions. Indeed, having precise time available across the entire grid enables utilities to better monitor and control power systems with faster response times to effectively manage disturbances and ultimately prevent system-wide blackouts [10]. The need for accurate timing in power systems and alig-nment of data to a unified time source was stressed by the North American blackout in August 2003 [36]. Furthermore, with the adoption of the North American Elec-tric Reliability Cooperation (NERC) Standard PRC018-1 in 2006, it is now a legal obligation that all recorded data must have an accuracy of 2 ms or better in relation to universal coordinated time scale (UTC) [23]. Thus, timing is a major issue in the design of such systems which typically use a time-slotted control protocol to perform sensing, computation, networking, and actuation on a periodic schedule [28].

On the other hand, the advance in deployment of smart grids depends on the ability to secure their operations. Such a requirement is of paramount importance for critical infrastructure including the power sector. Those concerns are escalated

(20)

with the increase in dependence on information and communication technologies, global positioning satellite systems, and communication networks among others. The integration of those technologies into the power grid improves its availability and reliability. However along with their advantages, they carry a lot of security threats that need to be addressed, and they expand the attack surface used to target the grid’s cyber side. As such, those additional security concerns should be addressed with proper prevention, detection, and mitigation mechanisms to ensure cyber and physical security of the future smart grid.

Hence, the deployment and wide spread of the smart grid is dependent on se-curing its components. Indeed, the transition phase from traditional power systems to the grid of the future carries a lot of challenges. The secure functionality of the grid, and an analysis of threats targeting its operations are on top of the list of chal-lenges. To incite this transitional phase, we aim at assessing the security of one of the essential building blocks of the grid functionality, namely the time synchroniza-tion mechanism used to distribute accurate timing signals to the grid’s components (substations, phasor measurements units, etc.). Availability of accurate timing is an enabler of the grid’s monitoring, protection, and control applications on a wide scale which is currently referred to as WAMPAC systems (wide-area monitoring, pro-tection, and control). Security concerns associated to the synchronization mechanism in use are brought forward by this mechanism to smart grid components relying on its services, and thus imposing a major threat on the availability and functionality of those components. IEEE 1588, more commonly known as the Precision Time Proto-col (PTP) [4], is one of the recommended time synchronization mechanisms for use in the smart grid. PTP, in its current version, is vulnerable to a multitude of threats that affect its usability. Thus, our security assessment of time synchronization me-chanisms is centered around PTP and its associated cyber attacks. We believe that

(21)

securing PTP is one fundamental step towards a secure and cyber-attack resilient smart grid.

On the other hand, time synchronization is a candidate for use as an attack sur-face to target the functionality of other grid components. Indeed, due to the critical nature of the smart grid, there is a need to align and correlate events dispersed across its domain. Such a correlation is made possible through accurately timestam-ped and synchronized measurements sampling the grid dynamics in real time, thus intensifying the dependency of monitoring, protection and control systems on time synchronization. In this aspect, a thorough analysis of the specifications governing the functionality of different intelligent electronic devices, and their use of accurate time synchronization is needed to identify gaps and vulnerabilities that may be ex-ploited by attackers to target the smart grid. The impact of such vulnerabilities escalate in the smart grid due to its dynamic nature, the interdependency between its power and communication components, and its cyber-physical nature. We tackle the presence of such vulnerabilities in phasor data concentrators, a key component of the wide area monitoring system (WAMS), to expose the impact of exploiting the system dependency on time synchronization, and leveraging existing vulnerabilities in mechanisms providing those services to target essential monitoring, protection, and control applications. This highlights the vulnerability of our critical infrastructure to cyber attacks, and emphasizes the need to consider security as the main require-ment in each of their enabling technologies. We take a first step in that direction by addressing the security of time synchronization, and proposing solutions to prevent, detect and mitigate cyber attacks targeting those mechanisms and all systems built on top of the services they provide.

(22)

1.2 Thesis Contributions

This thesis aims to supplement the existing and ongoing research efforts towards a more secure and attack resilient smart grid. A first step in that direction is addres-sing threats that target time synchronization in the smart grid mainly by improving the security posture of PTP being the main candidate for time distribution at the substation level. The first contribution of this thesis manifests itself in a brief, yet comprehensive overview of the state-of-the-art security assessment of the Precision Time Protocol. Precisely, Chapter 2 of this thesis presents an in-depth review of PTP along with an introduction and classification of other available time synchronization mechanisms, and the time-dependent power system applications with an emphasis on the accuracy requirements of each of those applications.

Next, Chapters 3 through 6 will mainly address three problems pertaining to PTP and smart grid security. These problems are briefly presented next, and detailed in their dedicated chapters.

1.2.1 PTP Security Vulnerabilities

PTP is well-known for its capability to synchronize clocks of different qualities to a common time source while providing accuracy of the order of microseconds. Howe-ver, PTP was not standardized with security in mind and is found vulnerable to a multitude of attacks targeting its services. For this purpose, in Chapter 3, we identify and address a security vulnerability in the authentication scheme followed by PTP security extension. In Chapter 4, we leverage the IEC 61850 substation synchroni-zation requirements to devise a detection and mitigation schemes for the well-known PTP delay attack. The proposed mechanisms are formally modeled, validated, and evaluated on a real implementation of PTP. In addition to that, we introduce an extension to PTP that allows the collection of synchronization status from connected

(23)

clocks for security purposes. The introduced extension is defined, formulated, and ve-rified in Chapter 5. Through the proposed extension, we believe that a PTP network becomes more security aware and more resilient to cyber attacks which eases up the use of PTP for time synchronization in the smart grid.

1.2.2 Vulnerability of WAMS to Time Synchronization

At-tacks

Time synchronization is of immense importance for situational awareness in the smart grid, and for wide area monitoring and control. Through accurately timestamped sampling and collection of power parameters, the control center can devise necessary actions to maintain the grid stability and availability. This sampling and collection is enabled through the deployment of phasor measurement units (PMUs) and phasor data concentrators (PDCs) at selected locations in the grid. However, this dependency on accurate timing can be leveraged to exploit vulnerabilities in the specifications and functionality of those devices, and eventually impact reliant power system applica-tions. We examine this problem in Chapter 6, where we identify and capitalize on a vulnerability in one of the methods used for phasor alignment at the PDC. We approach this problem using a linear program, and we consider system observability as the targeted power application. Through the presented model, we can identify an attack that is enough to prevent full observability of the power system, and thus open a window for an attack that leverages this weakness to initiate a cascading failure in the smart grid.

(24)

1.3 Thesis Organization

The rest of this thesis is organized as follows. Chapter 2 presents a brief overview of applications of precise timing in the smart grid along with the candidate time synchronization mechanisms for providing this precise timing, followed by a survey of the existing literature that addresses security concerns associated with PTP. Chapter 2 also presents a gap analysis for PTP that highlights open research problems that need to be addressed to secure PTP. Chapter 3 of this thesis introduces a shortcoming of the authentication scheme associated with PTP through its security extension, and identifies a potential amendment for this issue based on existing network and system management solutions. Chapter 4 addresses one of the well-known attacks against packet exchange based time synchronization protocols, the delay attack. We consider the use of PTP in a substation as recommended by IEC 61850 [57], the substation automation standard, to propose a detection and mitigation mechanism for the delay attack. We use formal model checking to evaluate relevant security properties of the proposed solution, and we demonstrate its usefulness on an actual implementation of the protocol. In Chapter 5, we build on top of the theory established in Chapter 4 to propose an extension for PTP that allows to collect messages from the network and analyze the collected information to assess the security posture of a PTP network. We once again use formal model checking and verification to validate the soundness of the proposed extension, and we demonstrate its usefulness using numerical simulation. Chapter 6 considers the wide area monitoring system as a scope of interest, and exploits the vulnerability of a data aggregation scheme followed by PDC to a time synchronization based attack. Through a linear program, we identify a PMU as an attack target along with an attack vector to be injected in its timing. As an outcome of this attack, the PDC receiving measurements from this PMU will drop phasors received from other benign PMUs. The outcome of this attack is formulated in terms

(25)

of system observability, and the approach is demonstrated using hardware-in-the-loop simulation. Finally, Chapter 7 concludes the thesis and highlights potential research problems for future consideration.

(26)

Chapter 2 Preliminaries and Literature

Review

In this chapter, we will overview time synchronization in the smart grid. Power system applications dependent on precise time will be presented. A review of mechanisms used for time synchronization will follow along with a security assessment of PTP. Standardization efforts related to our scope of interest will be shortly presented along with security related cuts. We conclude this chapter by carrying out a security gap analysis for PTP that highlights open research problems to be addressed.

2.1 Applications of Precise Time in Smart Grid

Critical applications in the smart grid require the presence of synchronized time across the infrastructure. These applications demand a common notion of time. Their measurements and monitored events need to be correctly aligned to enable proper actions and decisions. These actions define the self-healing characteristics of the smart grid. Indeed, providing real-time situational awareness to grid operators will decrease the impact of the outages by isolating the problem areas and avoiding

(27)

(28)

system to supply real-time voltage and current synchrophasors. These readings are synchronized to absolute time, and used to analyze the state of the power system and maintain its stability. Synchrophasors increasingly contribute to the reliable and economical operation of power systems as real-time control and protection schemes become broadly used [49].

With a fixed temporal reference frame, synchrophasor measurements may be used to determine useful information about operation of the grid [15]. Compared to tra-ditional SCADA measurements, synchrophasor measurements have higher sampling frequency, are able to provide direct measurement of power system states, and al-low for more accurate monitoring of power systems and faster remedial actions [126]. Voltage stability monitoring, and stabilization of large disturbances rely on phasor measurements of voltage and current supplied by synchrophasors as pointed out in [79], [107] and [98]. PMU measurements play a fundamental role in power system state estimation as demonstrated by [51], [30], and [66]. Interested readers can refer to [34] and [13] for a survey on the usage of synchrophasor measurements in power system stabilizers among other applications.

A key requirement by synchrophasors is the precise time synchronization of PMUs that are sampling the readings across the power system. IEEE C37.118 standard [3] specifies that accuracy limits for the measurements shall not exceed a 1% total vector error (TVE). This translates into a maximal time error of ±31.8 or ±26.5 microseconds for 50 or 60 Hertz systems, respectively.

2.1.2 Disturbance/Fault Recording

Due to the complexity of the power grid, a disturbance taking place in one part of the grid affects operation elsewhere. When these interactions result in major events such as cascading failures and large blackouts, recording devices installed at various

(29)

points in the grid generate large numbers of reports and data files [15].

To make sense of the collected files, there is a need to align the data recorded by several intelligent electronic devices (IEDs) at various locations to a common frame of reference. This data is used in post-event analysis. It allows to identify what happened where, and what happened when. It serves in finding the root cause of the disturbance, assessing the severity and duration of the fault, and taking any neces-sary remedial actions. The interpretation and alignment of fault records are eased by accurately time stamping the events during recording.

Recorded data from recording devices can be synchronized to assess the impact of a disturbance such as loss of generation, line trips, and loss of load. They have been installed in several power systems in North America [29]. A disturbance identifica-tion scheme to analyze the disturbance events recorded by those devices [29] and [127], accurately identifying the location of a fault upon its occurrence based on the integration of information available from disturbance recording devices [131], and ra-pid stability assessment of wide-are post-disturbance records [63] are available in the literature.

Nowadays, the trend is to equip all recorders with proper time synchronization. A one millisecond error is often regarded as sufficient for such applications [108].

2.1.3 Differential Protection

Numerical differential protection [132] works by evaluating Kirchhoff circuit law with current values obtained in numerical form from the different terminals involved. De-pending on the principle, the current values can be delivered as phasors or as instan-taneous values [132].

In a fault free system, the total of the currents is zero. The so-called differential current occurs in a faulty system when the currents are not balanced and their sum

(30)

is distinct from zero. This is considered as a criteria for tripping [108]. However, differential relays support safety margins to account for time error among other pos-sible errors. In protection systems, relays at the terminals of a differential protection line are synchronized. Current differential protection which utilizes wide-area current data will be effective for wide-area backup protection although such protection needs system-wide timing synchronism for the simultaneous current sampling at all remote terminals and data exchanges among them [102]. A fair time synchronization error is within the limits of 100µs.

2.1.4 Sampled Values

The IEC 61850 process bus involves the exchange of high-speed, real-time instanta-neous voltage and current measurements using an Ethernet network [10]. Voltage and current sampled values are delivered to protection and control IEDs along with control commands sent to switchgear. These values are produced at high rates (ty-pically 4 to 16 kHz). Merging Units continuously send sampled values of currents and voltages acquired from primary equipment. These digitized sampled values have to be received in synchronism by the relays so that the protection algorithm functi-ons properly. Data shifted at the receiving IEDs by just 30 microseconds will result in half of degree phase angle error [77]. A technique to assess the overall network performance of sampled value process buses in IEC 61850 is presented by [59].

As Sampled Values (SV) are distributed to independent devices throughout the substation, time synchronization becomes critical for all applications that require data from multiple locations (e.g., bus differential protection) [101]. The demanded time synchronization precision is less than 1 microsecond.

(31)

(32)

Locating faults using the traveling wave principle received much interest from the power community. Early approaches on fault localization using digital relay data in the literature is available in [91] and [67]. With the introduction and wide use of PMUs in the power transmission system, the literature presented approaches relying on the measurements supplied by PMUs for fault localization as indicated by [61], [42], and more recently in [76]. A recent manuscript on using joint PMU and SCADA data for fault localization on a multiterminal transmission line is presented in [86].

After highlighting the critical time dependent applications in the smart grid, we will present the mechanisms used to provide the timing signal for use in power systems. Some of these mechanisms have been deployed in power systems for decades. However, as our overview shows, they are no longer suitable to meet the accuracy requirements in the modern power systems or their deployment and maintenance requires a separate infrastructure which is not favored by power utilities.

2.2 Time Distribution Mechanisms

The time distribution mechanisms we will discuss are the ones currently in use, and candidate for use in the future smart grid. We will outline the basic characteristics of these mechanisms, and the accuracy level they provide. This will filter the ones that do not satisfy the smart grid applications’ accuracy requirements.

2.2.1 Pulse Per Second

A pulse per second (PPS) is an electrical signal of less than one second width, and a sharply rising or falling edge that accurately repeats once per second. PPS is consi-dered a simple and accurate time distribution mechanism. A time server distributes

(33)

pulses synchronized to the second rollover over a dedicated network to connected devices. This signal is limited by the care taken in the quality of the connection of the source to the device being synchronized [15]. PPS does not have the notion of absolute time, or clock changes (e.g. time of day). PPS is suitable for devices requiring synchronization within the second [108]. The supported accuracy is of the order of micro-seconds. However, its use in power applications is taken over by IRIG time codes. PPS is still commonly used in standards laboratories, to compare time and frequency sources at the highest level of accuracy [15].

2.2.2 IRIG-B

The IRIG time codes were originally developed by the Inter-Range Instrumentation Group (IRIG), part of the Range Commanders Council (RCC) of the US Army. The standard was first published in 1960 and has been revised several times by the Telecommunications and Timing Group (TTG) of the RCC [16].

The IRIG standard defines a family of serial time codes with different pulse rates. These codes use a continuous stream of binary data to transmit information on date and time. Each of these time code formats are distinguished by the signal charac-teristics (modulated, unmodulated), signal transmission techniques, data rate, and by the information carried in the transmitted data. Among the family of IRIG time codes, IRIG-B is the most known and used time format.

IRIG-B has a pulse rate of 100 pulses per second, through which it produces 100 bits of data. Out of these bits, 74 bits contain time, date, time changes, and time quality information of the time signal. IRIG-B code may be used in either logic-level (unmodulated) format, or as an amplitude-modulated signal with a 1 kHz carrier. IRIG-B presents time as a set of logical ones, zeroes, and position identifier bits. Connected IEDs to the IRIG-B service synchronize their clocks based on the data

(34)

collected from this signal. IRIG-B has three functional groups of bits: Binary Coded Decimal (BCD), Control Functions (CF), and Straight Binary Seconds (SBS). IRIG-B supports time-of-year and year information in a IRIG-BCD format, and an optional seconds-of-day in its SBS.

IRIG-B was extended in 2004 to use reserved bits of the CF part of the time code. Additional feautures such as calender year, leap seconds, daylight saving time, local time offset, time quality, parity and position identifiers are assigned to previously reserved bits in IRIG-B time code CF portion.

IRIG-B can be transmitted using various techniques either when its time code signal is moduled or unmodulated. Typical techniques for transmission of unmodu-lated IRIG-B include RS-485 differential signal over shielded twisted-pair cable, and RS-232 over shielded cable for short distances among others. For the transmission of modulated IRIG-B, coaxial cable terminated in 50 ohms or shielded twisted-pair cable can be used.

Most substation IEDs that accept the unmodulated IRIG time code use an optically-isolated input. This breaks ground loops, making possible direct connection throug-hout a control room witthroug-hout excessive concern for grounding and potential differences. Such optocouplers only require a few milliamperes of input current, making it possible to connect many loads to a single IRIG-B driver [14].

IRIG-B supports accuracy of the order of microseconds and is currently used by electric utilities to provide time synchronization to critical power system devices such as protection relays, PMUs, and digital fault recorders (DFRs) [94]. However, IRIG-B code signaling is unidirectional, with minimal error checking capability (single parity bit) [112]. Undetected by receiving devices, the processing of bad IRIG-B time frames result in faulty time synchronization.

(35)

2.2.3 Network Time Protocol

The Network Time Protocol (NTP), defined in the RFC 5905 - Network Time Pro-tocol Version 4: ProPro-tocol and Algorithms Specification [78], is a widely used time transfer protocol over data networks. Through a message exchange, a NTP daemon synchronizes the local device clock with that of one or more external reference time sources. Information included in the NTP message allows the daemon to determine the server time with respect to local time and adjust the local clock accordingly. In addition, the message includes information to calculate the expected timekeeping accuracy and reliability, as well as select the best server. This daemon plays the role of a client in collecting time references from servers. As a server, it can make its own time available as reference for other clients. Moreover, the daemon can be a peer in a comparison of different system times with other daemons before agreeing on a “true” system time to synchronize to.

These features can be used to set up a hierarchical time synchronization structure. Each of these hierarchical levels is labeled as a stratum. A smaller stratum number means a higher level in the hierarchy structure. The daemon with the most accurate time has the smallest stratum number and is located on top of the hierarchy.

Each NTP daemon can be configured to use several independent reference time sour-ces. The daemon polls these sources periodically to classify them as good or bad sources. This aids the daemon in choosing a new system peer once its current peer becomes unavailable.

To achieve synchronization, NTP includes methods to estimate the round trip delay between the server and the client. NTP also ignores estimates that vary sig-nificantly from the typical delay values. NTP uses time clock drift estimation to compensate time deviation and provide time stability in the absence of the time

(36)

source [94]. Accuracy level achieved by NTP depends on the performance of the de-vices’ operating systems, and the nature of the connection between the client and the server. Best accuracy levels are achieved when the logical connection between client and server is kept as short as possible.

The achievable accuracy through NTP time synchronization is of the order of milliseconds. Thus, NTP does not guarantee the accuracy level required by merging units (MU) in a substation. However, most IEDs satisfied by NTP services use its simplified version, Simple NTP (SNTP). SNTP uses the same messages as NTP, and achieves the same accuracy level. However, it does not consider some algorithms that maintain clock stability over long periods of time.

NTP’s standard level of performance is adequate to resolve the one-second am-biguity of a 1-PPS signal, so NTP and 1-PPS together make an acceptable method of accurate time synchronization in a substation [15]. However, this means that IEDs in a substation will receive time information over two connections. This is not feasible when compared to other available mechanisms and protocols especially PTP.

2.2.4 Precision Time Protocol

Precision Time Protocol (PTP) [4] is introduced in the IEEE 1588 standard as a candidate to fulfill the timing requirements of forthcoming systems. PTP allows he-terogeneous systems that include clocks of various resolution, precision and stability to synchronize to a single time reference with a sub-microsecond accuracy [4]. Moreo-ver, PTP power profile allows the usage of PTP for power system protection, control, and automation applications. PTP is recommended for time synchronization at the substation level by IEC 61850[57]. A more detailed overview of PTP will follow later.

(37)

2.2.5 Global Navigation Satellite Systems

Global Navigation Satellite Systems (GNSS) provide timing and location informa-tion for receivers over the globe. GNSS mainly consists of GPS[75], GLONASS[37], Galileo[68], and Beidou[1]. GNSS satellites are constantly transmitting signals which are collected and processed by receivers. Although GPS is the most widely used system, the services provided by these systems are similar. The timing accuracy achieved is below 1 microsecond and is most suitable for use by power systems pro-tection and control applications. Thus, time distribution mechanisms use this timing signal to synchronize devices in a substation since it is infeasible to equip all devices with a GNSS receiver.

The accuracy levels supported by the presented mechanisms vary and thus their suitability for use in various substation applications. In Table 2.1, we present a comparison of the different capabilities and drawbacks of these mechanisms. The table also highlights the ability of these mechanisms to fulfill the timing requirements of various substation applications. As can be concluded from the table, PTP and GNSS are the most suitable mechanisms for time supply. They are both capable of meeting the accuracy requirements while not needing any dedicated network. Thus, power utilities rely on GPS-synchronized clocks to synchronize devices in substations, control centers, and distribution feeder circuits [10]. However, since the use of GNSS signals requires dedicated receivers, GNSS supplies the accurate timing to a designated IED in the network. This IED will play the role of a PTP master and distribute timing information to other IEDs connected to the already available Ethernet network.

(38)

Table 2.1: Summary of time distribution mechanisms and their fulfillment of smart grid applications requirements

Mechanism Typical Accu-racy Level Synchrophasor Measure-ments Fault Recor-ding Differential Pro-tection Sampled Values SER Re-ports Fault Locali-zation 1-PPS 1 µs X X X X X X IRIG-B 100µs × X X × X × NTP/SNTP 1-10 ms × X _× _× X _× PTP 1 µs X X X X X X GNSS 1 µs X X X X X X

2.3 Precision Time Protocol

2.3.1 Overview

Precision Time Protocol (PTP) [4] is a time-transfer protocol defined in the IEEE Standard 1588, a standard for a precision clock synchronization protocol for networ-ked measurement and control systems [4]. It was developed to improve precision over current Ethernet protocols achieving a microsecond synchronization accuracy [4]. Moreover, PTP is capable of using the communication infrastructure available without the need to setup a PTP dedicated one.

PTP follows a packet-based message exchange approach to maintain time syn-chronization in the network. A designated time master sends periodic time-stamped messages to communicate accurate timing to the connected devices. Through the best master clock (BMC) algorithm, PTP establishes a master-slave hierarchy in the network. The established setup includes a single grand master clock (GMC), and a set of slave clocks synchronizing their time to that of the master. In addition to that, one can distinguish other types of clocks in the system with PTP defined functiona-lity. Such clocks include boundary clocks and transparent clocks. Boundary clocks

(39)

are used to maintain the timescales used in a domain. On the other hand, transpa-rent clocks (TC) measure the residence time of a PTP event message at a TC, and supplies this information to recipients of the message in transit.

To achieve clock synchronization at slave devices, PTP provides two mechanisms: end-to-end synchronization, and peer-to-peer synchronization. Both mechanisms rely on synchronization messages sent by the GMC, yet they follow different approaches to measure the master to slave path delay. The end-to-end synchronization mechanism is depicted in Figure 2.3. As Figure 2.3 shows, four timestamps are collected by a slave. t1 is the master time when the Sync message is sent, t2 is the time the slave receives the Sync message, t3 is the slave time when the slave sends a Delay Req message, and t4 is the time the master receives the Delay Req message. Those timestamps define the trip time from the master to slave, tms, and slave to master, tsm. The

slave calculates the round trip path delay using the collected time stamps as equation (2.1) specifies. The calculated path delay is used in clock offset computation using equation (2.2). The slave uses this offset to update its clock as per equation (2.3), whereT imeslave and T imenewslave are the time at slave before and after synchronization

respectively. At the end of this process, the slave clock is synchronized with that of the master.

P ath Delay = (tms+tsm)

2 =

(t4−t3) + (t2−t1)

2 (2.1)

Clock Of f set= (t2−t1)−P ath Delay (2.2)

T imenew

slave =T imeslave−Clock Of f set (2.3)

(40)

(41)

(42)

If that is not possible, clocks follow the two step synchronization where follow up messages are used to convey the accurate timestamp of the previously sent messages. Using either of the two approaches, the needed timestamps are collected at slave clocks, and used for accurate synchronization.

2.3.2 PTP Security Extension

A security extension, Annex K, was added to PTP to provide group source authentica-tion, message integrity and replay attack protection for PTP messages. The extension specifies two mechanisms to achieve the security goals specified. The integrity pro-tection mechanism verifies the source, integrity, and freshness of the received messages by using message authentication codes and counters. The challenge-response mecha-nism allows for the affirmation of new authenticated sources and the management of trusted relations.

It is worth noting that the implementation of this security extension is optional. Clocks requesting secure PTP message exchange indicate that by setting a flag bit in the message header to indicate the presence of the security authentication fields in the transported message.

2.4 PTP Security Assessment

PTP is vulnerable to a wide range of attacks targeting the services provided by the protocol. Systems relying on PTP time synchronization services suffer the impact of these attacks. We will next study those attacks, along with the countermeasures applied to detect and defend against those attacks as presented in the literature.

PTP secure functionality was the interest of much work in the literature before and after the introduction of Annex K extension. In[116], the authors provided a

(43)

description of the security extension to PTP along with various attack points to target the PTP network and an attack targeting the master election algorithm. In [81], the authors discussed the delay attack in time synchronization through a game theoretic approach and suggests using multiple paths between master and slave clocks for time synchronization to mitigate the risks of that attack and its applicability to PTP. In [97], the authors studied the so-called selective packet delay attack on PTP to identify the fields of the message that need to be compromised to carry the attack. They study the presence of fake masters in the network and their effect on clock synchronization at slaves as well. Figure 2.5 presents a look at the attacks targeting time synchronization under PTP. Others study vulnerabilities present in the design of PTP along with the attacks targeting its specifications. The literature is summarized in Tables 2.4 and 2.5 and will be divided over three parts; approaches targeting PTP use in power grid, others discussing issues in the design of PTP, or suggesting usage of already available secure mechanisms in implementing PTP functionality and enhancing its security through the use of Transport Layer Security (TLS) [41] and Internet Protocol Security (IPsec)[17] along with approaches discussing the suitable algorithms for PTP message authentication codes (MACs).

2.4.1 PTP in power grid

The use of PTP in power grid systems is analyzed in [115] where the authors provide an approach to enable determining clock drift at electric devices when a connection inside the substation is broken resulting in desynchronization between the master clock and its slaves, a phenomenon known as islanding. They start their approach by summarizing the various threats targeting time synchronization. They sum up their discussion by dividing these threats into two categories, either the slaves are aware of their desynchronized clocks after the attack or they are unaware. They discuss the

(44)

(45)

the applicable attacks, and suggest countermeasures to guard against them.

The discovered attacks include denial of service (DoS), byzantine master, interruption of control loop, removal of packets from control loop, packet manipulation, packet insertion, replay attack, and selective packet delay attack[119, 50].

The security properties of PTP before the introduction of the optional Annex K extension are investigated by Gaderer et al. in [50], while Tsang et al. in [119] provide a compilation of attacks targeting PTP. The output presented is a set of attacks applicable to a PTP network where the optional security extension, Annex K, is not deployed. To carry on these attacks, the authors assume that the adversary has access to the network, can monitor, collect and analyze exchanged messages. The attacker targets the master, slave clocks, and the control loop. The effect of his attacks range from introducing incorrect offset to the slave clocks, to complete control of the time synchronization mechanism and the prevention of this mechanism through DoS.

The countermeasures presented to defend against these attacks are addressed in the annex K extension of PTP especially the ones related to authenticating the nodes, protecting message integrity and preventing replay attacks. Other countermeasures [50] include using cryptographic techniques along with QoS monitoring to protect against these attacks. It is worth noting that, Annex K security extension mitigate most of the causes of these attacks. However, other attacks such as selective packet delay is still a threat targeting time synchronization under PTP.

In computing the master-to-slave path delay, PTP assumes that the communi-cation network is symmetric. This assumption can be targeted through what is known as delay attack, or selective packet delay attack. This attack is pointed out in [119, 50, 120, 125], but is formulated and studied in [120, 125]. Indeed, Ullmann et al. [120] indicate the vulnerability of PTP to delay introduced in the communication

(46)

channel. This delay affects the accuracy of path delay calculation based on the arrival time of synchronization and delay request messages. A similar approach is followed by Yang et al. [125] where the attack model presented defines a man-in-the-middle capable of introducing a random time resembling quantity in path delay calculation by manipulating the master-to-slave and slave-to-master message exchange. The at-tack analysis shows that delaying the synchronization message sent by the master affects all the slaves in the network while delaying request messages affects only the slave sending the message. The analysis quantifies the error in the offset calculation in terms of the introduced delay. Such an attack succeeds in jeopardizing the time synchronization mechanism and is hard to detect by the involved parties.

To defend against such attacks, the authors in [120] suggest implementing specific network security mechanisms to protect the network, and monitoring the usual pro-pagation delays across the network. They also point out that security mechanisms ensuring the authentication of the communicating nodes and the integrity of this communication do not counter the described attacks. On the other hand, Yang et al. [125] propose a detection mechanism based on hypothesis testing. Their hypothesis monitors the ratio of the master clock to the slave local clock. They assume that the system is under attack when this ratio exceeds a threshold specified by 1 mi-cro second. However, deploying such a mechanism means that the slave and master know each other clock values which is unrealistic in a PTP network; or the existence of a monitoring entity capable of observing the clock values at various nodes in the network.

PTP specifications are targeted by Treytl et al. in [117] to reveal vulnerabilities in the specifications. The identified weaknesses assume the presence of a man-in-the-middle capable of capturing and modifying the exchanged synchronization messages. The first flaw allows the attacker to modify the source and destination in the time

(47)

synchronization messages. This is eased by the fact that those addresses are not included in the calculation of the integrity check value (ICV) at the receiver side. The attacker can create forged security associations and change the clock value at the slave side. The authors suggest using a source port identity field in the PTP header to retrieve a unique source address which is included in the calculation of the ICV. The second flaw is related to the presence of transparent clocks in the IEEE 802.1 network, and the fact that those clocks modify the MAC address present in the exchanged messages. Transparent clocks terminate the incoming link and create a new frame with the PTP payload, the source MAC address of the outgoing bridge and a modified correction field. Security associations relying on unmodified source protocol address will discard such frames. The authors suggest creating double entries for the core functions of the standard containing the master MAC address along with that of the last transparent clock or use the source port identity field in security associations. However, the authors indicate that to target PTP with the above indicated flaws is implementation dependent, and would require a brute force breakdown of the random value of the lifetime field available in the requests. Their practical applications is related to the feasibility of that brute force attack.

Recently, Moreira et al. [85] performed a comparison of the proposed approaches to secure PTP in its future version based on the discussion in the standardization committee, and proposed a hop-by-hop group authentication and integrity solution using MACsec and IEEE 802.1X standards. On the other hand, Narula et al. [89] established a fundamental theory for secure clock synchronization. The authors found that PTP is not secure based on the necessary and sufficient conditions of this theory, and they presented a specialization of specific conditions for PTP secure clock syn-chronization. Those conditions include the availability of an authenticated encryption scheme between the communicating parties, negligible difference between forward and

(48)

backward path delays, and a previous knowledge of this path delay. Although those conditions provide a secure version of PTP, their applicability is subject to much con-cerns especially when it comes to a previous knowledge of the estimated path delays in a local area network. An analysis of PTP security is provided by Itkin et al. in [60] where the authors exploit PTP vulnerability to a multitude of threat models and subsequent attacks. The presented attacks can be carried by in-band and out of band weak and skillful attackers targeting the communicated messages and various entities of the PTP network. The described attacks are addressed with suitable security me-chanisms, and a revised PTP security extension is proposed. However, even with a revised extension, PTP will remain vulnerable to attacks due to its nature and design considerations namely the delay attack performed by an in-band attacker.

The study of PTP showed some vulnerabilities in its design as the previously discussed works show [120, 125, 117]. However, the most critical among these vul-nerabilities is the delay attack. Such an attack is hard to differentiate from network congestion and delays, difficult to detect, and succeeds in targeting PTP slaves. The countermeasures presented neither detect nor prevent this attack [120], or base their detection on non-realistic assumptions [125]. Moreover, although the issues raised by Treytl in [117] threaten PTP functionality, the feasibility of using such issues to affect time synchronization under PTP can be questioned. However, these efforts present a good starting point for assessment of PTP security based on its design.

2.4.3 Implementation

To secure PTP services, the use of IPsec and MACsec is investigated in [118] and [80]. While Mizrahi et al. [80] presented a threat analysis in the presence of IPsec and MACsec, Treytl et al. analyzed the impact of the use of IPsec tunnels on PTP clock synchronization and the accuracy levels it provides in [118]. Indeed, the author

(49)

in [80] described the common IPsec and MACsec deployment scenarios and present a subsequent threat analysis. In this analysis, internal and external attackers are con-sidered. These attackers are capable of intercepting and manipulating the exchanged messages, capturing and injecting messages into the network. The attacks presented target the integrity and authentication functions. Packet injection and manipulation, spoofing, replay attacks, rogue master, packet interception and removal, packet delay manipulation, layer 2 and/or 3, DoS, and time source spoofing are the attacks dis-cussed. The applicability of these attacks to networks protected by IPsec, MACsec, or Annex K specifications is presented. These attacks result in slave nodes aligning with a false time value or inability to synchronize their clocks. The author concluded that a hybrid approach deploying a combination of these mechanisms can securely support PTP operations.

On the other hand, IPsec effect on PTP time synchronization is compared by Treytl et al. [118] to that of the native security measures in IEEE 1588[4]. IPsec usage is illustrated in the protocol stack used to synchronize PTP clocks. A unit for message protection using cryptographic operations is introduced at the network layer in the transmit path along with another for the verification of incoming messages. Additionally, two security state machines responsible for the security management are directly integrated in the IP stack. The analysis shows that the delay presented by IPsec results from the use of encryption algorithms, packet size, and security sche-mes. The performed measurements show that the jitter introduced is noticeable over the receive path compared to little jitter on the send path. The results contrast the use of IPsec to that of unprotected IP. The authors analyzed the use of a hardware timestamping unit to eliminate the pre-mentioned jitter. They concluded that the use of a MAC based timestamper for IPsec with an adjusted clock frequency of the SHA unit can be an effective solution. This modification is a limiting factor in embedded

(50)

systems and might affect IEEE 1588 clock synchronization over IPsec.

2.4.4 PTP MACs

Algorithms used for generating PTP message authentication codes (MACs) are cove-red in [93, 83, 84]. As a conclusion, these papers suggest an alternative MAC protocol implementation that can satisfy the need of PTP authentication security and provide better performance than the ones specified in PTP Annex K extension.

Indeed, in [93], the authors point out that the use of HMAC-SHA256 MAC spe-cified in Annex K is suboptimal in terms of delay resulting from MAC calculations. Based on testing of other MAC protocols implementation, they suggest the use of Chained MAC (CMAC) and claim that it allows on-the-fly calculation of the MAC. Another major modification they suggest to the Annex K is dropping the three-way handshake and replace it by a one-way authentication. As a node joins the net-work, it shall send periodic authenticated supervision frames to introduce itself to other nodes. These nodes can validate the authenticity of these frames by checking the ICV value. This approach makes use of the pre-shared keys in the network and avoids the additional overhead caused by message exchange in the three-way hands-hake. In addition, the authors find that the used sequence numbers are too short for effective protection against replay attacks and suggest using absolute time instead. Through their thorough analysis of Annex K, they suggest other modifications to its specifications. These alterations require removal of some parts (challenge-response exchange, security association update exchange, etc.), modification of the ICV test to start from the destination protocol address rather than the PTP header, a col-lective replay protection mechanism that uses a 48-bit register, and changes in the security association structure, secure message transmitting, transparent clock rules,

(51)

shared key distribution and authentication TLV. The suggested modification prove that there is enough room for modifying the Annex K and enhancing its efficiency. However, Moriera et al. in [83] and [84], demonstrated the feasibility of using SHA-3 (KECCAK) as the MAC function in PTP message security. Their study is based on a comparison between AES-128 and SHA-3 where the hardware implementation of SHA-3 provides the same security level and latency but with lower area consumption. The literature review exposed the threats associated with the use of PTP for time synchronization. Although there are much countermeasures proposed, some of these threats (as the case with PTP delay attack) rise as a main concern for any system relying on PTP to synchronize its devices. In an upcoming section, we highlight the gaps threatening PTP secure functionality and we address some of them in the upcoming chapters.

2.5 Standardization Efforts

This section covers the developed standards that guide the time synchronization in the smart grid. First, we will cover the PTP power profile which aims at specifying PTP options to be implemented in clocks for use in power industry. It also specifies the default values for a set of PTP attributes to suit power industry requirements. Second, IEC standardization efforts related to time synchronization in power substations will be covered. This mainly includes IEC 61850-5[55] and IEC 61850-90-4[56].

2.5.1 IEEE C37.238

The second release of PTP in 2008 included the definition of new devices (e.g., trans-parent clocks) along with a set of attribute options (e.g., transport over the IEEE 802.3 Ethernet or UDP) and optional features (e.g., unicast messaging). However,

(52)

aiming at ease of setup with minimal administration, the concept of PTP profile was introduced to identify a set of required features and assign default values for attribu-tes based on the needs of industry. Thus, the profile specifies a subset of the protocol features to be implemented based on specific industry requirements.

A PTP power profile customized for power system applications is introduced in the IEEE C37.238 [5]. This profile defines PTP features and attributes for use in power system protection, control, automation, and data communication applications utilizing an Ethernet communications architecture. The profile specifies a well-defined subset of PTP mechanisms and settings aimed at enabling device interoperability be-tween different vendors, robust response to network failures, and deterministic control of delivered time quality [5].

Among the profile specifications, the IEEE 802.3 Ethernet is specified as the pre-ferred physical layer for PTP related communication and parameters configuration. The profile also specifies the use of peer-to-peer delay mechanism in measuring the propagation delay over the communication link. It recommends using one-step ope-ration in communicating time information in the network. However, it also allows using two-step operation for less expensive silicon solutions. An overview of main PTP power profile specifications are available in Table 2.2.

Table 2.2: IEEE C37.238 main profile specifications

Profile Option Value

BMCA Default BMCA

Transport IEEE 802.3/Ethernet

Delay mechanism Peer delay only

Management SNMP MIB (mandatory for grandmaster-capable devices

only)

The profile defines strict requirements to ensure the time accuracy and quality required by substation applications. It demands that inaccuracy introduced by a

(53)

transparent clock must not exceed 50 nanoseconds. This allows achieving an accuracy of one microsecond by a slave clock connected to the GMC over 16 network hops. The profile also specifies that there should be at least two or three devices in the network capable of being GMC in case the later fails. Finally, SNMP Management Information Base (MIB) use is specified for configuration and status messages.

2.5.2 IEC 61850

The IEC 61850[57] standard is developed to make substation automation interopera-ble and cost-efficient. It was designed to operate over modern networking technologies. The standard ensures interoperability in power systems among many other features. In Part 5 [55], IEC 61850 covers communication requirements for functions and device models. And Part 90-4, technical report, network engineering guidelines for Ethernet networks are presented. Among the contents of Part 5 and Part 90-4, issues related to time synchronization in substation and power systems in general are presented.

Through these two parts, the IEC 61850 defines time models and time synchroni-zation requirements at the substation level. It targets the synchronisynchroni-zation of precise clocks at various levels of substation automation, and aims at specifying required accu-racy levels for various events(e.g., time-stamped measurements, sequence-of-events). The standard specifies the need for only one time base in substation, and a unified time tagging format for all devices in the power system. It specifies the use of absolute time synchronization for synchrophasors while relative time synchronization is used for protection functions.

The IEC 61850-5 defines different synchronization classes related to the application using the time signal. These classes are indicated in table 2.3. According to table 2.3, accuracy requirements vary from±1µsec for protection functions to±1 msec for event logging.

(54)

Table 2.3: Synchronization classes of IEC61850-5[57]

Class Accuracy Usage

T1 ±1 ms Event logging

T2 ±100 µs Zero crossing for the distributed synchrocheck. Time tags to support point on wave switching

T3 ±25 µs Class P1 protection functions

T4 ±4µs Class P2 protection functions (e.g. busbar protection

function). Time tagging of samples

T5 ±1µs Class P3 protection functions and high precision time tagging of samples

Time synchronization specifications presented by IEC 61850 can be summarized as follows:

• A dedicated time server present in the substation receives time signal from an external source outside the substation (GNSS, long-wave radio, etc).

• In case of absolute time usage, two time servers of different types must be avai-lable.

• For PTP use in substation:

– Only layer 2 communication and peer-to-peer delay can achieve the required accuracy.

– Alternate master option is recommended for implementation.

– Hold over time for slave is 5 seconds in case of master failure.

– Reference clock should be located on the station bus, and used for synchroniza-tion of devices on the process bus as well.

• The time signal for time synchronization shall be easily derived from a global time reference system like GPS.