Process Distribution

Figure 5.8.: Subcontracting as Means for Process Distribution among Peers and Super-Peers.

The capability of distributed process execution was identified as one important requirement for CPS workflow engines in Section 2.6 (Requirement R4 ). CPS and IoT environments consist of a large number of devices and entities, which are often organized in hierarchical structures. The current trend of Fog Computing [SW14] tries to leverage these properties of IoT systems to provide scalability, security and fast preprocessing [BMZA12] by also using processing resources of devices along the network (cf. Section 2.3.5). Network gateways provide computing resources closer to the edge that can also be used for process execution and distribution of tasks. For that reason, process management systems have to be scalable in terms of manageable process resources and the support of decentralized SoS architectures. The PROtEUS WfMS supports these requirements as described in detail in [SNS14b, SHA17].

5.5. Process Distribution

Figure 5.8 shows an extended workflow based on the Emergency scenario pro- cess to illustrate the advantages of distributed process execution in the context of a mobile health care/AAL scenario [PRS+13, PRBA15]. After the detection of an emergency situation by a wearable blood glucose monitor, a service robot is in- structed with a subprocess to retrieve an insulin injection from a particular shelf within the smart home. This subprocess consists of three process steps to be per- formed by the robot: driving to the shelf, retrieving the injection, and driving to the human in need. These three special process steps are executed locally by the robot as an edge device (cf. Section 2.3.5) without any other process resources involved and without the need of communicating with other process engines, which is why the robot is able to enact this subprocess autonomously. The overall process terminates upon the successful execution of this subprocess.

5.5.1. Distributed Systems Architecture

Figure 5.9.: Hierarchical Overlay Network Structure of Peers and Super-Peers. As suggested in Figures 5.9 and 5.8, we rely on a hybrid overlay network structure, which consists of “regular” PROtEUS WfMS peers and higher order D-PROtEUS super-peers [SNS14b]. This structure reduces bottlenecks and single points of failures compared to classical client server architectures and it is also more efficient regard- ing communication and organisational overhead compared to pure peer-to-peer ap- proaches [YGM01] or fully decentralized systems (e. g., Multi-agent Systems [Fer99]). Peers and Super-Peers form a multi-level hierarchy providing a scalable solution with respect to the number of process engines and process instances and leveraging lo- cality in the sense of Fog Computing as certain workflow tasks can be moved closer to the computing edges onto specialized devices, which also increases autonomy of these devices (cf. Section 2.3.5).

Peers

Within the hybrid peer–super-peer infrastructure, peers are arbitrary devices that are able to run the PROtEUS system and therefore execute process instances as

described so far. Each peer is connected (statically) to a dedicated super-peer, which manages it and from which the peer receives process fragments and execution commands. Upon the initial connection to the corresponding super-peer, a peer sends its profile including its name, type, IP address, and capabilities to the super- peer. In CPS, peers may comprise various classes of devices ranging from regular desktop computers to resource-constraint embedded and mobile devices.

Super Peers

Besides also running the PROtEUS system for executing process instances, super- peers have an active Distribution Manager component, which is responsible for man- aging the network infrastructure and the distributed process execution. This config- uration is called D-PROtEUS. The role of a super-peer is assigned manually by the system architects based on a device’s available computing resources. Super-peers should be more reliable with respect to availability and have a steadier network connection as well as more computing resources as they have to perform additional management tasks. Due to the super-peers also running the WfMS, they can execute processes and act as “regular” peers to higher level super-peers. In order to maintain a global state of the process executions, super-peers exchange global synchronization information about the execution of process instances among each other. A multi- level hierarchy consisting of peers, super-peers and super-peers, which also act as peers allows for delegating subprocesses and tasks along the paths of the tree-like network hierarchy. Suitable resources for the distributed process execution can be found in the peer–super-peer network in various ways, e. g., by using the Kademlia algorithm based on distributed hash tables [MM02].

5.5.2. Distribution Manager

The Distribution Manager is the component running on a super-peer, which is re- sponsible for communicating with Remote (PROtEUS) Engine Clients via the Web- Socket server (cf. Figure 5.1). Its main task is to extract, distribute and merge subprocesses and process fragments to and from peers in the sense of subcontracting for processes [vdA00] and process instance migration [ZHKL10]. Figure 5.8 shows the subcontracting and migration mechanisms with respect to the “Retrieve Insulin Injection” subprocess instance. The super-peer instantiates the particular process and enacts the process instance using the D-PROtEUS system (0–1) (cf. Figure 5.10). It evaluates the corresponding Resource and Distributed attributes of process steps and subprocesses (cf. Section 4.2.5). If a subprocess contains the active distributed flag and a resource identifier belonging to one of its associated peers, the Distribu- tion Manager is invoked with the corresponding process step as input parameter (2). The Distribution Manager creates a snapshot of the subprocess instance as proposed in [MCS16] and then calls the WebSocket server to serialize relevant data as a pro- cess fragment–the process step model, the current state of the process instance and ingoing data instances–and to transfer it to the peer (here: Service Robot running the PROtEUS system) in question (3) (cf. instance migration [ZHKL10]). Here the WebSocket server deserializes the received data (4) and invokes the Process Man- ager to instantiate and execute the subprocess based on the model information, data instances and state of the process (5). The peer’s process manager instantiates the

5.5. Process Distribution

Figure 5.10.: Message Flow during Distributed Process Execution.

subprocess, the PROtEUS WfMS executes the instance (6–7), and the result of the execution (i. e., new states of the process instance and outgoing data instances) is transferred back via the WebSocket server to the super-peer’s Distribution Man- ager (7–10). The Distribution Manager then merges these results into the main process instance (11) and the execution continues on the super-peer (12).

With subcontracting, the hierarchical structure of the network can be leveraged when distributing tasks and processes. Specific process activities and subprocesses can be moved closer to the compute edge for autonomous and fast local preprocess- ing or to the Cloud for resource-intensive computations (cf. Section 2.3.5). Offline execution is enabled through subcontracting as devices do not necessarily have to rely on a permanent connection to their designated super-peer when executing sub- processes. This increases the resilience of the process execution, especially on mobile devices (e. g., service robots), which are likely to disconnect from and reconnect to the network more frequently. Subcontracting also increases data security as only data relevant for the execution of the specific subprocess is transferred to the peer, while global process data is only known to the responsible super-peers. The selec- tion of suitable peers is currently done in a static way based on the URI attribute contained in the process step model, which points to the specific peer. More sophisti- cated mechanisms of dynamically assigning resources automatically to subprocesses exist (e. g., as described in [SYY07, MVSA16]), but are out of scope of this work. As the PROtEUS system is also available as a Docker container9, multiple instances of the peer (PROtEUS ) or super-peer (D-PROtEUS ) configurations can be started easily and be automatically distributed to various nodes based on different criteria (e. g., by using the Kubernetes10 container orchestrator). The subcontracting for

9_{https://www.docker.com/}

10

processes as applied in our approach provides a mechanism for the top-down distri- bution and delegation of tasks in the network hierarchy from higher order peers to lower order peers. This is in contrast to the “classical” way of programming for Fog computing, where processing is done on the nodes closer to the edge first and then data and tasks are moved up the hierarchy (cf. Section 2.3.5).

Figure 5.11.: Mobile Dashboard Application for Managing PROtEUS Resources.