PART II THESIS CONTRIBUTIONS
Chapter 5 Evaluation Methodology
5.2 Analysis by Mathematical Modelling
The mathematical based work is organised in two parts. We first studied aspects related to the complexity of the proposed agent location algorithm to prove its viability for the proposed agent system (Chapter 6). We were not able to model the precise algorithm for general network topologies, but we could analytically find upper bounds on the performance indicators. This allowed the estimation of the time-scales of the agent deployment process and to draw conclusions on its scalability with respect to network size and number of agents.
In the second part of our analysis we started looking into the goodness of the proposed algorithm (Chapter 7). We look for particular conditions which lead to optimality or near optimality, hoping that these are realistic enough to justify the algorithm. We, then, carry out a theoretical assessment of the agent system both at transient time and at steady state. In these cases, we manage to model the system mathematically. We model two different flavours of a centralised polling approach and carry out a comparative performance analysis between that approach and the agent solution.
The conclusions which we then draw about performance and scalability of our solution are valid under the given restraining conditions on the network topology. We discuss these conditions with respect to real network topologies.
A further step is to study the algorithm at steady state for general network topologies but this was difficult to carry out using this approach. The work was then continued switching to a simulation-based approach, as described later in this chapter (page 107).
5.2.1 Network Model
We have adopted the network model described in [Rescigno 97]. The network is modelled by a connected graph G=(V,E) with the vertices, V corresponding to nodes (processors) and the edges, E corresponding to communication links, which are modelled by the All Ports-Full Duplex communication model. This network model has been widely used because it generally reflects the hardware characteristics of networks (see references quoted in [Rescigno 97]).
Packet forwarding is modelled using an ‘on/off’ model for link transmission. Only one packet can traverse a single link at any time. A link is modelled as an atomic resource that is switched off during the transmission of a packet over that link and is switched on at the end of each packet transmission. Links are characterised by a fixed latency.
Packets arriving at a node are immediately forwarded through an outgoing interface to the relevant link only if that link is on. Packets are, otherwise, queued at the node. Infinite queues are assumed and, consequently, no packet dropping mechanisms are taken in consideration.
5.2.2 Fixed Parameters
Fixed parameters are the ones that affect the system performance and do not vary during the analysis [Jain 91]. The main parameters of the mathematical based analysis are reported in Table 5-1.
Fixed Parameter Symbol Value
Node degree (generic node) Ô generic
Node degree of the monitoring station node d(u) generic
Size of polling request packet h generic
Size of polling response packet br generic
Size of agent notification packet hnot generic
MA size ^‘^■^size generic
MA transmission time TRANSMlime generic
MA forwarding time FORWtime generic
MA serialisation time SERIALtime generic
MA de-serialisation time DESERIALiime generic
MA cloning time CLONtime generic
Table 5-1. Fixed parameters of mathematical modelling.
5.2.3 Factors
Those parameters that are varied during the evaluation are factors and their values are called levels [Jain 91]. In general, with both experimental and simulation-based techniques the list of factors, and their possible levels, is larger than the available resources will allow. It is, thus, necessary to limit the levels during the evaluation in order to study the system in its normal range of operation. For example, it will be difficult, if not impossible, to measure the impact of a particular agent migration policy on the overall monitoring traffic if we saturate the network with background traffic.
The mathematical approach usually allows a larger degree of freedom in constraining the factor levels because the system is modelled rather than physical. The main factors of the mathematical-based analysis are reported in Table 5-2.
Factor Symbol Level
Number of monitored nodes N unconstrained
Number of Mobile Agents P 0 < p < N
Levels at which MAs reside in the spanning tree L 0 < L < R (u )
Polling rate Pr unconstrained
Notification rate Nr unconstrained
Network radius R(u) R(u) > 1
Table 5-2. Factors of mathematical modelling.
5.2.4 Workloads
The workload consists of a list of service requests to the system [Jain 91]. When possible reference traffic models, or benchmarks, are used to compare the performance of a computer system. This was not possible in our case because, to the best of the author’s knowledge, no benchmarking procedures have been specified yet for management systems.
In our case the workload is the traffic injected by the monitoring system to carry out monitoring tasks. We adopted the basic polling management model and, hence, monitoring tasks inject traffic which is proportional to the polling rate, Pr. For a given task and a given polling rate, the overall traffic and response time will generally vary for different agent configurations. Thus, our simple reference workload allows measuring the system performance under a range of input values specified by the levels of the various factors of Table 5-2.
5.2.5 Data Analysis and Presentation
Data analysis is useful to compare different alternatives through experimentation or simulation. In the case of mathematical modelling there is no need for that because the produced models are analytical functions that can be directly interpreted.
Results are directly presented in graphical format. The algorithmic complexity is obtained by studying the behaviour of the models when the various factors tend to infinity. Similarly, the system behaviour under near-optimal conditions is carried out by direct interpretation of the models.
Steady-state performance indicators are plotted against the number of MAs to assess the effects of varying agent configurations. Contour plots are used to capture performance as a function of two parameters instead of a single one.