Server Application Layer Design and Implementation

In this sub-section, the architectural design of a standard application layer module built on top of IP-TCP/UDP, where ACSI servers and virtual representations of real devices can be configured, is proposed and described in detail. The core ACSI services using the client/server communication model do not require any complex mechanisms. It is

Chapter 5: Communication Processor Design

sufficient that the requesting node knows about the IP address of the destination node. However, this is not the case for the publish/subscribe communication model requiring information exchange service models such as GOOSE. The success of these services relies on the ordered use of mechanisms for the tasks of registering, binding, filtering and making subscriptions. The key behind the design of the real-time publish/subscribe communication model can be briefly summarized as follows. Non-real time activities such as getting publication or subscription rights happen outside the real time loop ideally at the start-up. Conversely, the generation, transfer and reception of messages are real-time activities happening in the real-time loop requiring very fast response times. There are two types of nodes. Subscriber nodes can only subscribe to messages whereas “publisher-subscriber” nodes can publish messages as well as subscribe to messages published by other nodes. The following sub-sections describe how the tasks of registering, binding, filtering and making subscriptions are accomplished at the server application layer module. The architectural components, which play part in the carrying out of these tasks and others including the sending, reception and execution of IEC 61850 associated messages, are also described.

5.5.1.1 Registering

Ideally at the start-up, each publishing node must be associated to a multicast group each having a unique label, which is referred to as the IP Multicast Group Address (MGA). The Multicast Membership Service (MltcMS) is responsible for storing IP MGAs. Entries within the MltcMS contain publishers’ IP host names and MGAs. Various nodes on the network communicate with the MltcMS to create/delete entries and to add/delete their publication/subscription rights.

Chapter 5: Communication Processor Design

A node interested in publishing messages gets its publication right by creating an entry in the MltcMS’s IP MGAs table. On the other hand, those nodes interested in receiving messages from a particular publisher have to make use of the publisher’s IP MGA in order to join themselves to the multicast group associated with that publisher. This is referred to as registering. Any node interested in subscribing to messages gets its subscription right by retrieving the IP MGA for the multicast group it seeks to join from the MltcMS. Once the subscriber obtains the IP MGA, it can complete its registration and join a multicast group by following the subsequent sequence:

1. The registry manager process running within the subscriber joins a multicast group, Group 1, by sending a join request to its IP module using a remote interrupt, 2. The IP module forwards an IGMP membership report message to the neighbouring

multicast router, and

3. The multicast router sets up a distribution tree for Group 1 adding the interface details of the joining subscriber so that it can receive packets sent to Group 1.

5.5.1.2 Subscription, binding and filtering

Registering has to be followed with a subscription request sent to the relevant publisher. It should be kept in mind that registering basically serves the purpose of setting up a distribution tree for each multicast group within the multicast router. In applications where publishers need not to be aware of their subscribers, the process of registering is by itself adequate. However, as indicated previously, each publisher in a substation network is required to hold detailed information related to its subscribers. Thus, the mechanism of sending subscription requests and receiving confirmation messages in return has been designed and implemented in this project in order to solve this problem.

Chapter 5: Communication Processor Design

Subscription requests are sent immediately after the subscriber registers for any multicast group. One subscription request has to be sent to each one of the publishers.

A subscription request is no more than a packet containing a number of fields. By setting the fields of a subscription request, a subscriber can inform any publisher over the network about its own local details such as the node name and IP address in addition to information specifying whether it wants to subscribe for GOOSE and/or MSVCB messages produced by that node. Hence, a combined approach of publisher and subject based subscription mechanism has been adopted, which has a number of advantages when used in conjunction with the registering process.

Strictly speaking, the task of binding in publishers is unnecessary since each publisher uses a Multicast Group Address to publish its messages. However, each publisher still produces a list of its own subscribers to be tagged onto the outgoing multicast message. This serves two purposes. First of all, such a list can be cross-examined by the router against its own registry list reducing the possibility of unwanted messages from being sent to nodes showing no interest in them. Secondly, when a multicast message reaches its destination, the subscriber can access to the tagged information and store the names of the other subscribers that subscribe to the same publisher as required in substation applications. Similarly, the task of filtering can also be fully avoided since after all the measures taken, the chances of an unwanted message reaching at any node is fairly low. However, with the intention of being precautious, a fairly simple filtering process has been implemented that checks the source of messages comparing it to the entries in the node’s list of publishers. In cases where a match can not be found, the packet will simply be destroyed.

Chapter 5: Communication Processor Design Data Publisher Subscriber IEC-MOM Real-Time OS Delivery Manager Registry Manager ACSI Configuration Manager

My Publishers /Subscribers list Node Node Node Node Network MltcMS Operations Membership Execution Manager

5.5.1.3 Architectural components

Figure 5.12 depicts the detailed architectural overview of the server application layer module. Such a model is appropriate and efficient for client/server requests as well as periodic and synchronous updates between sources and sinks. It exclusively supports the publish/subscribe model as it makes use of the previously discussed mechanisms.

Figure 5.12 Architectural components of the ACSI server application layer module

The following sequence of events occurs when a source (publisher) pushes multicast packets out of its output interface destined for a particular multicast group:

Local Manager

Chapter 5: Communication Processor Design

1. The application running within the source multicasts a packet using the Multicast Group Address (MGA) that is also referred to as the multicast membership address, 2. When the packet reaches its rendezvous point (IP address of the router responsible

for distributing multicast traffic to the specified multicast group), the router’s IP process forwards the multicast packet to the “ip_pim_sm” process, and

3. The “ip_pim_sm” process is one of the child processes of the IP module, which makes multiple copies of the multicast data and sends one copy to each one of the subscribers listed in the multicast route table.

All the architectural components shown in Figure 5.12 assist in the successful operation of the application layer communication model in one way or another. An individual discussion for each one of the architectural components is provided below:

Local Manager: Local manager executing in every publisher and subscriber is

responsible for the creation/deletion of publication or subscription rights. One of the tasks of the local manager is to periodically update its “multicast membership addresses” list based on the periodic info received from the MltcMS regarding the status of the publication/subscription rights.

Registry Manager: Subscribers interested in joining any multicast session group do

this with the help of the registry manager, which simply contacts the IP module expressing the subscriber’s interest in joining a multicast session group. Registry manager also handles the process of sending subscription requests. In cases where publishers subscribe to other publishers’ multicast session groups, it also performs the task of updating the “publishers” component of the MyPublishersSubscribers list.

Chapter 5: Communication Processor Design

Execution Manager: Execution manager is responsible for determining the type of the

packet arriving from the IEC-MOM middleware module. After it determines the type of the packet, it executes the relevant packet execution/destroy mechanism. Examples of the packet types expected at an ACSI server application layer include reports, GOOSE messages, ACSI service requests, GOOSE or MSVCB subscription requests, MSV messages and etc. Individual packet execution/destroy mechanisms have been designed and implemented for various packet types. The packet execution/destroy mechanism are mainly used for accessing the relevant fields of the received packet obtaining the information stored within its fields. Once this is concluded, they execute a response based on the information acquired. For example, in the case of service requests received from ACSI clients, the information stored within the packet include the name of the service to be performed and input parameters to be passed to the service. The ACSI service request execution/destroy mechanism accesses this information executing the relevant service with the input parameters received in the request. The names and possible input parameters of ACSI services were previously covered in Chapters 3 and 4. After the packet is executed, the packet execution/destroy mechanism updates a number of statistics such as the application-to-application delay statistic of the packet before destroying it.

Delivery Manager: The delivery manager thread is executed shortly after the execution

of a service request received from an ACSI client. Each service request needs to return a reply back to the calling client. This task is accomplished by the delivery manager, which creates and sets the fields of a reply packet headed to the requesting client. All the output parameters returned by the executed service are inserted into the same packet along with a timestamp designating the current time and date.

Chapter 5: Communication Processor Design

ACSI Configuration Manager: The ACSI configuration manager basically acts as a

device parameter configuration tool. It is mainly used for configuring an ACSI server at the server application layer module. Substation configuration describes which of the optional information is used in a specific device, what the instance names of all LNs are. IEC 61850-6 has specified a description language for the configuration of electrical substation IEDs. This language is called the Substation Configuration Description Language (SCL), which is based on the XML schema language. However, in this study, the SCL has not been utilised. Instead, the LNs, LDs, data and data attributes as well as the services used and provided by an IED are configured utilising the C++ programs described in Chapters 3 and 4.

5.5.1.4 Server Application Layer Implementation

Software based implementation of the ACSI server application layer module was once again carried out using the OPNET Modeler software. Figure 5.13 shows the process model of the processor module where the general behavioural model of an ACSI server was implemented.

Chapter 5: Communication Processor Design

Processor modules are the primary building blocks of node models. They can be connected to other modules via a number of packet streams. They can act as traffic generators and/or sinks. As illustrated, the STD of the server application layer module consists of 5 states and transitions between these. The actions performed in each state are listed below:

• Init State: The ACSI configuration manager and the local manager are the two

components executing in the “Init” state. The ACSI configuration manager configures an ACSI server whilst the local manager updates the “multicast membership addresses” list of the node with the information it receives from the MltcMS. The state variables used by this process model are also initialised in this state. The process proceeds to the “P_Subs” state once all these tasks are accomplished.

• P_Subs State: Registry manager is the sole component executing in the

“P_Subs” state. It carries out the registering and sends subscription requests to the publisher nodes.

• Inactive State: The process stays in the “Inactive” state until a stream interrupt

is received triggered by the arrival of a new packet. Only then, the process model proceeds to the “Decide” state.

• Decide State: Execution manager is the main component running in this state. It

determines the type of the received packet and calls the relevant packet destroy/execution mechanism. If the packet is a service request message, the process proceeds to the “Generate” state. Otherwise, for all other message types, it defaults back to the “Inactive” state where it waits the arrival of the next packet.

Chapter 5: Communication Processor Design

• Generate State: The “Generate” state is where a reply (confirmation) packet is

created for the executed service, its fields set and forwarded to the ACSI client where the service request initially originated from.