Comparison of Real-time Network Traffic Estimator Models in Gain Scheduler
Middleware by Unmanned Ground Vehicle Network-Based Controller
Zheng Li, Rangsarit Vanijjirattikhan, Mo-Yuen Chow, Yannis Viniotis
Advanced Diagnosis, Automation, and Control (ADAC) LaboratoryDepartment of Electrical and Computer Engineering North Carolina State University
Raleigh, NC 27695, USA
zli7@ncsu.edu, rvanijj@ncsu.edu, chow@ncsu.edu, candice@ncsu.edu
Abstract – We propose a Gain Scheduler Middleware (GSM) module, for use in IP network-based control applications. The network traffic estimator (NTE) module of GSM must provide good estimation of the real-time network delay in order for the overall IP network-based control to provide satisfactory results. In this paper, we demonstrate the advantages of the NTE module vs. non-delay estimation control. Five delay estimation models are introduced and compared by using three typical network delay datasets. A network-based control of an unmanned ground vehicle (UGV) path-tracking problem is used to illustrate the performance of the NTE model. Simulation results show that the generalized exponential model outperforms the other models on the three criteria, including the UGV tracking error, tracking time and the computation time of the estimation model.
I. INTRODUCTION
Control technologies have extensive applications in areas such as motor drives, robotics and manufacturing plants. However, most control applications nowadays are often constrained geographically. Network-based Control (NBC) technology has been investigated and developed to combine communication networks with control systems to form the Network Control System (NCS) (e.g., control network in automobiles, the tele-operation and coordination of distributed Unmanned Ground Vehicles (UGV), robot arm manipulators) so that different parts of the control system can communicate with each other remotely through a network. Fig. 1 shows a typical NCS in which the controller controls the plant through a communication network. The control signals and feedback measurements from the plant are all transmitted via a network. The control applications using NCS can reduce investment and maintenance cost for wiring complexity (e.g., in factory automation), enable tele-operation, and enable new control concepts and applications such as Intelligent Space [1].
The emerging NBC technology has gained much attention during the last decade. Different types of networks have been used in the NCSs. Control Area Network (CAN) such as DeviceNet has been developed and used in the automotive industry since 1983 to handle time-critical industrial applications. Other similar types of networks, such as ControlNet and Profibus, are token-passing bus control networks which can transmit the control signals and feedback between the controller and plant via the cycling tokens. However, these kinds of networks have restrictions on long-distance and world-wide applications. For example, the maximum DeviceNet control line is less than 100m, and
the maximum number of nodes in the ControlNet is 99 [2]. Asynchronous Transfer Mode (ATM) has been used in NCS applications because of its world-wide distribution. Using the circuit-switching philosophy, ATM reserves bandwidth to guarantee network performance. However, due to its dedicated bandwidth, circuit switching has high cost, so that it is mostly used in limited applications, such as tele-surgery. Compared with these networks, packet-switching networks, such as IP, are more promising for their world-wide and low-cost applications. Sensor Feedback Controller Network + -Control Signal Controlled Plant Z.O.H. T Control Signal Reference Signal Output
Fig. 1. Network Based Control system.
One major challenge in all the network-based control systems is the presence of network delay, which might degrade the overall system performance and even destabilize the closed-loop control system. Alleviating the network delay effect is especially important for the long distance applications, e.g., time-critical tele-operations. Several techniques, such as buffering [3], optimal stochastic control [4] and sampling time scheduling [5], have been proposed to reduce the effects of network delay. But most of these techniques are only appropriate for linear systems or some specific forms of nonlinear systems.
Without redesign, replacement or reinstallment of the controller to fit in the IP network and controlled systems, Tipsuwan and Chow have proposed Gain Scheduler Middleware (GSM) as a distinguished solution that can alleviate the IP delay effects and optimize system performance [6]. Fig. 2 shows GSM’s schematic diagram.
Controller
Gain Scheduler Middleware Network traffic estimator Feedback preprocessor Gain scheduler Communication
Network Remote system Probing
Control
signal Feedbacksignal
() C⋅ g , ˆ ˆ ( )τCt y ( ) Rt u ( ) Ct u ( ) Rt y ( ) Ct y ()⋅ ψ ˆ q () ξ⋅ () β⋅ uβ,C( )t ( , , , ) R=R R R Rt x f x p u ( , , ) R= R R R R y h x p u ˆ q q ( )⋅ N
Fig. 2. Schematic diagram of Gain Scheduler Middleware (GSM). In Fig. 2, GSM handles all network connections between
the central/main controller and the remote system controlled over the network. It enables the network operations such as sending and receiving packets, bandwidth and resource allocation, network traffic monitoring, etc. GSM has three components: Network Traffic Estimator (NTE), Feedback Pre-processor (FP) and Gain Scheduler (GS). NTE probes the network traffic “state” q (e.g., round-trip delay and packet loss rate), estimates the current network traffic qˆ and feeds this information to FP and GS. FP pre-processes the data measurements, based on the controlled system and network traffic condition, before forwarding them to the controller to generate new control signals. When the main controller sends out the control signals, GS modifies them by using a gain scheduling algorithm to provide an optimal performance based on the current network traffic conditions. GSM makes the network transparent to the control applications. Among the three components of GSM, NTE is particularly important, because its estimates form the basis for the decisions of FP and GS.
The main traffic metrics of the IP network include network delay (reflecting the traffic load) and loss rate (reflecting the network reliability). In the NCS, data loss can be remedied by sending a duplicate packet if no acknowledgement packet is received for a certain time. Delay in IP network is measured by the Round-trip-time (RTT), recorded from the time when the server (controller) sends a packet out to the time when the controller receives the acknowledgement. RTT depends on network equipment, throughput and the congestion condition on the network during packet transmission. The network connecting the controller and the remote system typically consists of network equipment such as computers, routers, switches, cable and/or wireless connections. In a typical IP network, bandwidth on the links sending and receiving packets is not reserved. Therefore, it is difficult to predict the network delay based solely on the IP addresses of the controller and the remote system.
In order to solve this issue, the NTE module in GSM frequently probes and records the network RTT delay values for the packets between the controller and the remote plant. Then the network conditions are continuously estimated by the traffic estimator algorithm. Network traffic research was a popular topic in the networking area in the 1990s, producing a wide variety of models. For example, proposed models include Markov processes [4], autoregressive processes (ARMA), long-range dependent and multi-fractal processes for matching and explaining the network traffic. These models have been used to both analyze networks and generate delay data for network simulation. To the best of our knowledge, these models have not been fully used in the NBC area, such as in real-time network delay prediction for control signal transmission purposes.
The paper is organized as follows. In section II, we briefly describe three classes of network delay. In section III, we explicitly define five estimator algorithms to be used in the NCS over IP networks. In section IV, we implement these algorithms in GSM’s NTE module and test their
performance in an unmanned ground vehicle (UGV) path tracking application; the vehicle is controlled remotely via an IP network. Two metrics, the deviation of the UGV from the path and the travelling time, are used to evaluate the traffic models. Finally, in section V, we summarize the results of this study.
II. NETWORK DELAY MODELS
In this section we describe three classes of network delay models previously used in communication networks and their features.
A.Constant delay model
Constant delay is the simplest network delay model. This model uses a constant a as the estimation of the (random) network delay, τ . The model is applicable to networks whose traffic load does not vary significantly. For example, in Control Area Network (CAN), there is basically at most one packet sent in the network channel at any given time, because of the token bus network topology used [2]. Delays in CAN can be fairly accurately modelled as having a (low magnitude) constant value.
In an IP network, constant delay can be achieved by using compensative timed buffers [3]. After compensation, each transferred packet will have the same delay time. From the NCS controller point of view, the communication network has a constant delay. The constant delay model for IP network-based NCS has the following features:
1)It is easy to calculate; and the sampling time is fixed. 2)It provides a reasonable approximation to networks
with low delay bursts and loss rates. Examples of such networks include LANs and short-distance networks. 3)The model requires implementation of the time buffer. Since the constant delay model needs to have extra network buffer implementation, in this paper, we will not discuss and compare it with the other delay models.
B.Statistical delay model
In the IP network of the NCS, transmission delays have been modelled with an underlying probabilistic distribution defined as an independent, identically distributed (i.i.d.) process. Network delay can not be described with a single time-invariant mathematical model over a long period of time; however, it has been verified that, when smaller time periods are considered (e.g., 3 min), the model provides sufficient accuracy [7].
In previous research, different statistical distribution models have been used to provide a better estimation of the real-time network delay in IP networks. In [8], the authors have modelled the most recent network delays over a moving window by a generalized exponential distribution to adaptively predict the next delay. In [9], the authors predict the delay based on the mean and median of the previous delay values. The statistical delay model we used in NCS has the following features:
1)It is simple to calculate, and suitable for real-time delay prediction.
2)The model parameters can be dynamically adapted, according to different network types and traffic conditions; this can significantly reduce the delay prediction errors.
C.Markov chain model
The Markov chain model was introduced into the NBC system for delay and stability analysis in [4]. This model focuses on the dynamic changes of S, the network delay state. Each of these states represents a possible range of network delay values. The dynamic state change is recorded as the state transition matrix P and is used in the prediction of the new Markov state. Markov chain model can capture the dependency between successive time delays which is not an i.i.d. process. This model has the following features:
1)It has the potential to provide a better prediction, since it can capture network traffic dynamics better.
2)It has a higher computation complexity; the number of states, s, is limited, since the computation time grows as O(s6).
III. ALGORITHMS FOR NETWORK DELAY
ESTIMATION
In this section we define the criteria to evaluate the network delay estimation models and formulate five estimation algorithms based on the basic models introduced in section II.
Implemented in the NTE module of GSM, the real-time delay estimation algorithm predicts the current network delay, which will be used by the PE and GS modules in the GSM. Consequently, the NTE estimate affects the overall performance of the controlled plant. Three main criteria are used to evaluate the goodness of NTE traffic estimation:
a) Robustness of the control system,
b) Accuracy and efficiency of the control system,
c) Real-time delay estimation cost, e.g., computation time.
Controller Controlled Plant ( )t y ( τsc) k kT− y ( τca) k kT− u Z.O.H Network 1 c wait k k τ +τ sc k τ ca k τ (kT) u (kT) y (kT) r 2 wait k τ
Fig. 3. NBC schematic and RTT.
Fig. 3 shows the schematic difference of a typical NBC system in which a plant is controlled by a main controller via a network. The controller sends the control signal u(kT) to the controlled plant based on the reference signal r(kT). The output y(kT) of the plant is fed back to the controller via the network. In this NBC system, five typical types of delays exist. Among them, if we omit the controller and plant’s processing time τkc and waiting time τkwait1 and τkwait2
which are in general much smaller than the delay induced by the network, the overall RTT delay can be estimated by
ca sc
k k k
τ ≅τ +τ , where τkca is the delay for the control signal
travelling from the controller to the plant, and sc k
τ is the delay for the measured feedback signal from the plant to the controller. The delayed control signal and delayed feedback are denoted as
(
ca)
k
kT−τ
u and y
(
kT−τksc)
respectively. The actual and estimated RTT delay are denoted as τk w|and τˆk w| respectively, where k=1,2,…, is the delay time measurement index and w is the window size representing the number of previous RTT delays used in order to predict the next delay. The traffic delay estimation model calculates the estimated RTT delay τˆk+1|w based on the most recent w
delays
{
τk w− +1 ...τk−1τk}
, denoted as:{
}
(
)
1| 1 1
ˆk w k w , ..., k , k
τ + =NTE τ − + τ − τ , (1)
where NTE( )i is the network traffic estimator algorithm. A larger window size means the model uses more delay measurements to perform its prediction. The error between the estimated and actual delays is denoted as:
1| ˆ 1| 1|
k w k w k w
e + =τ + −τ + . (2)
In this paper, all delays are assumed positive, τk w| >0, and the network delay characteristic is parameterized by q.
Five real-time network delay estimation algorithms derived from Section II are proposed and will be compared using the three criteria in the following sections.
A.Mean value estimation algorithm
This algorithm predicts the current RTT delay by the mean value of last w measured RTT delays. The predicted delay is denoted as τ and the characteristic network delay parameter is denoted as qμ =τk w| , where
{
}
1| | 1 1
ˆk w k w k w , ..., k , k
τ + =τ =mean τ − + τ − τ . (3)
B.Median value estimation algorithm
This model predicts the current RTT delay by the median value of the last w measured RTT delays. The predicted delay is denoted as τ and the characteristic network delay parameter is denoted as qσ =τk w| , where
{
}
1| | 1 1
ˆk w k w k w , ..., k , k
τ + =τ =medianτ − + τ − τ . (4)
C.Max value estimation algorithm
This algorithm has been used as the network delay model in [6, 9]. It chooses the larger between the mean and median values based on the last w measured RTT delays and uses it as the next estimated network delay value. The characteristic network delay parameter is denoted as qmax=max
{
τk w k w| ,τ |}
{
}
{
}
1| | |
ˆk w q qμ σ, k w k w,
τ + =max =maxτ τ . (5)
D.Generalized exponential distribution estimation
algorithm
This algorithm has been used as the real-time delay model in [8], where the characteristic network parameters are
, T
E= ⎡⎣η φ⎤⎦
exponential distribution with probability density function: [ ] 1 ( )/ , , 0, . k k k k e P τ η φ τ η φ τ τ η − − ⎧ ≥ ⎪ = ⎨ ⎪ < ⎩ (6) This function is calculated from the previous w network RTT delay measurements
{
τk w− +1, ...,τk−1,τk}
, where theexpected value of the RTT delays E⎡⎣τk w| ⎤ = +⎦ φ η and variance
2 2
σ =φ .
To estimate the delay, we first choose η based on where 1|
ˆk w
P⎡⎣τ η= + ⎤⎦ reaches its peak in
{
τk w− +1, ..., τk−1, τk}
’s distribution p.d.f., then predict{ } 1| 1 1 ˆ var ... k w k w k k φ+ = τ − + τ− τ , (7) and delay τˆk+1|w=φˆk+1|w+ηˆk+1|w. (8)
E.Markov chain estimation algorithm
Markov chain algorithm predicts the delay range by classifying the Markov states and its state-transmission matrix. The Markov states at time k are denoted as
{
}
| 1, ,
k w
s ∈ … S , whereS is the total number of states. They are classified by the w sequencing network RTT delays
{
τk w− +1, ..., τk−1, τk}
. The state transmission matrix, denoted as RS S× ={ }
r i jij , ∈{
1, ,… S}
, captures the dynamic changes from state to state; its values are denoted as(
1| | |)
ij k w k w
r =P s + =j s =i . (9) and they are calculated from the apriori probability distribution of previous dynamic state changes.
The probability distribution of τk w| is assumed given by the state sk of the Markov chain. For example, a 3-state
Markov chain with continuous observation densities can be used and the probability distribution is calculated by:
( )
(
|)
i k k k
f τ =Pτ s =i , (10) where i=1, 2,3 corresponds to low, medium and high (H) load on the network.
IV. SIMULATION RESULTS AND DISCUSSION
In this section we present the results of the NTE models used in the simulation of a network-controlled UGV tracking system. The overall system of the UGV path-tracking control over a communication network is depicted in Fig. 4. This system is composed of the UGV dynamics, the communication network N( )⋅ with delay times τCR and
RC
τ , and the central controller gC( )⋅ . The network separates the system into a remote plant site and a central controller site, between which the transmitted control signals u
( )
t and feedback signals y( )
t are received with the network induced time delay.The UGV dynamics, path information, and the position of the UGV are assumed known or measurable. This
information is used to calculate the control signals to drive the wheel of the UGV to track the path.
Central Controller U G V Dynam ics Path Tracking Controller ( ) 2,C⋅ g Comm unication Network N avigation M odule ( ) 1,C⋅ g ( )⋅ N ( ) Ct y ( ) Rt u ( ) Ct u CR τ RC τ ( ) Ct r eC( )t ( ) Rt y + - xR=f x p uR(R,R,R, )t R= R y x
Fig. 4. UGV Path-Tracking Network-Based Control System. A straight path with a 90° turn, as shown in Fig. 5, is used as the test path for our investigations. This test path is chosen because it resembles the step input function test for a classical single-input, single-output system so that we can investigate the “step response” of the UGV subject to time delay. 0. 5 m 1.0 m GOAL UGV TRACKING PATH
Fig. 5. The test path used for the time-delay effect analysis.
Fig. 6. Delay datasets and histogram (NC, TU, TH).
Three datasets of previously recorded network RTT delays are used to represent three typical communication network delays. They contain one-second measurements between the server in ADAC Lab and the servers in North Carolina local area (NC), University of Texas (TX) and Thailand (TH), respectively. Unlike the NC and TX datasets, the TH dataset contains measurements in which delay changes dramatically for a period of time. This part will be used in the simulation to test the UGV tracking performance with highly-random delays. Fig. 6 shows the 3 delay datasets and their corresponding histogram plots.
Two cost functions are used to evaluate the performance of the overall NBC system. The first cost function, J1, is the deviation of the UGV from the desired path:
1 J 0 ( ( ), ( )) f t t D x t y t dt =
∫
, (11)where D is the shortest distance between the position of the UGV
(
x t y t( ) ( ),)
and the tracking path as depicted in Fig. 7.U G V D(x(t),y(t)) TRACKING PATH UGV TRAJECTORY (x(t),y(t))
Fig. 7. The shortest distance between the UGV and the path.
0
t is the initial time, and tf is the final time when the UGV reaches the destination. In the following, we set t0 = 0.
The second cost function, J2, is the traveling time for the UGV to reach the destination:
2
J = tf− =t0 tf . (12)
During the simulation, we focus on the comparison of UGV path tracking cost values:
(1) with and without the NTE, and
(2) with the NTE, among the five different NTE models.
A.Simulation results with and without the NTE module
Table 1 summarizes the results of the network-based UGV path-tracking system simulation with and without the NTE module. It is apparent that the costs J1 and J2 with the NTE module in effect are much smaller than those without the NTE module; this means that, with the NTE module, the UGV can track the path with less error and time. We can also observe that the system performance degrades as network delays increase. The UGV trajectories for the three different network traffic conditions are shown in the bottom three graphs of Fig. 8, to indicate how the UGV behaves due to the NTE module and different datasets. Conforming with Eqn. (11) and (12), higher J1 and J2 values indicate higher overshoot and oscillation amplitude of the UGV trajectory.
Table 1. Cost values J1 and J2 (with and without NTE module).
NTE J1 J2
NC TX TH NC TX TH
with 0.0812 0.0609 0.1912 11.937 12.562 15.927
without 1.5605 2.1259 3.4830 32.157 40.272 46.909
B.Simulation results with different NTE algorithms
We have also simulated the UGV path-tracking application with different NTE we simulate the UGV path-tracking program with different NTE models, in order to choose the optimum network delay algorithm in GSM. In order to derive statistically significant results, we ran each experiment at least five times, using different time models for the delay measurements to observe whether the results are consistent. We used the median value to represent the result for each model case. Table 2 shows the values of the cost functions J1 and J2. From the cost values in three dataset
lines, we see that the deviation J1 from the path increases
when the delay time increases from NC to TH datasets. Among the five different NTE models, in the local and less-variable networks represented by the NC and TX datasets, the generalized exponential (GE) and Markov models outperform the other three. But in the large delay variation network represented by the TH dataset, the Max model gives the best results among the five models. This is because by definition, when the new measured delay data τk changes substantially from the previous delays
{
τk w− +1, ..., τk−1}
,both GE and Markov models will not be directly influenced until the new changes dominate the window ofw delays. However, the Max model is directly affected by the mean value which is sensitive to a sharp change on any of the delay values. This is also observed in the values of the J2
metric. According to the simulation, the sensitivity of the models to delay changes also depends on the window size
w. In this paper, we experimentally chose w=60, a value that gave us estimations with the least errors. The “optimal” choice of the window size is a topic for further research.
In Fig. 9, 3 groups of UGV path tracking trajectories are plotted; each one was tested with a particular dataset. In each graph, the five trajectories are plotted in five different colours. The UGV in the simulation with NC and TX delay datasets eventually tracks the path, after a transient period of adjustment. With the high frequency and large changes of the delay values in the TH dataset, these 5 NTE models can not track the path. A potential remedy is to implement a more complex delay estimator in NTE model to check this circumstance. The NBC process can be suspended until the network becomes more stable to prevent instability of the closed-loop control system.
Table 2. Cost values J1 and J2 (with different NTEs). J1 J2 NC TX TH NC TX TH Mean 0.1399 0.2069 0.3162 16.700 16.146 15.583 Median 0.1448 0.1903 0.3350 16.846 15.944 15.834 Max 0.1399 0.1912 0.3174 16.701 15.927 15.582 GE 0.1164 0.1871 0.3998 16.074 15.125 17.665 Markov 0.1094 0.1856 0.3375 15.847 15.016 15.996
For real-time applications, the computation time for the NTE model is an important factor to be considered. Table 3 shows the range of clock cycles each model uses, with 2 parameters added.
Table 3. Computation time of NTE models.
Clock cycle ranges
Algorithm Initial (w=60) Update (w=60)
Mean [102, 103] [10, 102]
Median [103, 104] [10, 102]
Max [103, 104] [10, 102]
GE [103, 104] [10, 102]
Markov (S=5) [105, 106] [103, 104]
Among the 5 algorithms, the Markov chain algorithm is far more expensive than the other ones in terms of computation time. In addition, increasing the number of states, S (in order to improve the estimation precision) will increase the computation time in the order of O(s6).
Therefore we will prefer to use GE model in the real-time NBC simulation and experiment to prevent introducing unnecessary additional computation delay into the NCS.
V. CONCLUSION
In this paper, we have evaluated the advantages of using the NTE module in the GSM framework, for IP network-based control systems. We have also compared the real-time network RTT delay estimation algorithms used in the NTE model based on the simulation of a network-based UGV path-tracking control system. From the three classes of network delay models used in previous research, five delay estimation algorithms were introduced and evaluated by two cost functions on three typical network delay datasets recorded between ADAC Lab and three remote servers.
Among the algorithms, the generalized exponential and Max algorithm outperformed the others in our evaluation process. Further calculation of computation time shows that the Markov chain model is not preferable, due to its expensive computation time requirements. The models and results in this paper can be used in the design of real-time network delay estimators for the applications of NBC systems.
VI. ACKNOWLEDGEMENTS
This work was partially supported by the National Science Foundation (IIS-0426852).
VII. REFERENCES
[1] W.-L. Leung, R. Vanijjirattikhan, Z. Li, L. Xu, T. Richards, B. Ayhan, and M.-Y. Chow, "Intelligent space with time sensitive applications," Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2005.
[2] F.-L. Lian and J. R. Moyne, "Performance evaluation of control networks: Ethernet, ControlNet, and DeviceNet," IEEE Control Systems Magazine, vol. 21, pp. 66-83, 2001.
[3] R. Luck and A. Ray, "An observer-based compensator for distributed delays," Automatica, vol. 26, pp. 903-908, 1990. [4] J. Nilsson and B. Bernhardsson, "LQG control over a Markov
communication network," Proceedings of the 36th IEEE Conference on Decision and Control, 1997.
[5] Y. H. Kim, W. H. Kwon, and H. S. Park, "Stability and a scheduling method for network-based control systems," Proceedings of the IECON 22nd International Conference on Industrial Electronics, Control, and Instrumentation, 1996. [6] Y. Tipsuwan and M.-Y. Chow, "Gain scheduling middleware
for networked mobile robot control," Proceedings of the American Control Conference, 2004.
[7] Y. Zhang and N. Duffield, "On the constancy of internet path properties," Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, pp. 197-211, 2001.
[8] Y. Tipsuwan and M.-Y. Chow, "Gain scheduler middleware: a methodology to enable existing controllers for networked control and teleoperation-part II: teleoperation," Industrial Electronics, IEEE Transactions on, vol. 51, pp.1228-1237, 2004. [9] R. Vanijjirattikhan, M.-Y. Chow, and Y. Tipsuwan, "Feedback Preprocessor for Unmanned Ground Vehicle Network-Based Controller," Proceedings of the 30th Annual Conference of the IEEE Industrial Electronics Society, 2004.
