Simulation Environment

The simulation setup follows guidelines that ensure every step is thoroughly performed to ensure nothing will be overlooked. Each stage of the methodology is performed before moving on to the next stage, and if necessary stages can be revisited if modifications that are necessary are discovered in a later stage. The following twelve stages were performed during the simulation setup and experimental runs.

Stage one of the methodology involves defining the problem. This stage was performed very early in the research and is discussed in section 1.3 of Chapter 1 in the Research Methodology. The problem is developing a protocol that is superior under certain conditions to previous protocols. The goal of this research is to show, using simulations, that the protocol is efficient in terms of resources available, and effective in terms of the various security attributes that a full protocol is required to provide.

Stage two of the methodology required the planning of resources. This was a straightforward process of ensuring the simulator would run on a desktop computer in a simulated laboratory environment. Matlab is software that is both powerful and efficient in running on a desktop computer. No problems with running a custom design in Matlab were expected and this proved to be the case. Table 6.1 shows the simulation environment.

Table 6.1 Simulation environment.

Artifact Description

Computer CPU Intel Pentium 4 2.4 GHz

Operating System Windows XP Professional v 2002

Simulator Software Matlab v7.6.0.324

Simulation Type Dynamic

Pseudo Random Number Generator Twister

Stage three of the methodology involves defining the system to be studied in general terms. This was a long and complex process of designing the protocol and has been thoroughly described in previous chapters.

Stage four involves a conceptual model of the system, including the parameters for the experiments. This first involves defining the fixed parameters for the simulation. Table 6.2 shows the fixed parameters for all simulation runs.

Table 6.2: Simulation fixed parameters.

Simulation area (metres square) 2500

Simulation time (seconds) 600

Nodes present at start 1

Node growth per minute 30

Node leave per minute 10

Node mobility model RWPM

Node pause time (seconds) 0-10

Nodes malicious (percent) 6%

Malicious message threshold 3

Accusation ejection threshold 5

Accusation ejection timeout (seconds) 60

Simulation runs averaged 10

Communication distance (metres) 300 Private message exchange rate / sec 0.1

Stage five occurs next and it is here that decisions are made as to what variable parameters should be used. This was a complex task as the protocol is designed to provide a very tunable environment meaning many combinations of parameters exist. This process involved two main areas of research. Firstly, previous protocols that were relevant to the current design and secondly the type of application that the protocol is likely to be used for. By examining previous protocols where the authors have provided simulation results, the input parameters could be identified and utilised where appropriate. This gave results that could then be compared to the results of previous protocols to compare performance of the new protocol. Looking at the type of application that this new protocol may be used for, fixed and variable parameters were

identified that are appropriate. These two approaches were not necessarily mutually exclusive. Once simulations had been performed that could directly compare to other protocols in input parameters, comparisons in performance could then be made. The input parameters could then be altered to those more appropriate to the very large and dynamic network topology where comparisons are less easily made as most other protocols have not had similar input parameters or their performance has not been described.

Stage six involves input data preparation where all the inputs for the experiment are decided upon. From stage six onwards proved to be an iterative process, as preliminary experiments were required to help define what variables should be used for the main experimental runs. This meant that as described in stage five, it was necessary to run preliminary simulations to assist with the decision on the range of variables.

Stage seven involved the programming of the code in Matlab to build the simulation software.

This was followed by stage eight where verification and validation of the parameters and outputs was performed. This was part of the iterative process of revisiting stage six through to stage nine where the final simulation runs were prepared.

Stage nine involved initial experiments where results were examined and minor modifications made to the range of variables. From the trial results, server placement modifications were made so that a greater choice existed from the simple random placement. This came from the preliminary results where the possibility of greater

efficiency was identified by choosing different server placement rules. For example, during preliminary runs it was found that up to five servers required for a certificate was the maximum reasonable number as any more than five servers reduced the success rates to an unreasonably low level. This was especially true as the trust level required was also raised resulting in more failures. Very quickly the certificate refusal rate approached 100%. Therefore, the maximum number of servers required was set at five. Any more would be ineffective and so deemed not to be a realistic choice. Other variables such as maximum speed were also experimented with to find a realistic range. With the preliminary simulation results used as a guide, modification to the input variables was made to reduce the experimental runs to a manageable number. Once the ranges of the variables were set, then the simulations could be planned with precision.

Stage ten was next and involved running the final simulations. This took several months due to the very large number of experiments required. Each simulation with the same parameters was run ten times with ten different seeds for the pseudo random number generator (PRNG). This ensured that whenever a random number was generated, each simulation would not generate an identical sequence. Random numbers were used for such things as placement of nodes on the grid, which nodes were mobile or servers, how fast a mobile node would move within the ten kilometer per hour window amongst others. The results of ten simulation runs with the same parameters but with different PRNG seed values was then averaged and the average used as the final result for that simulation. In this way for example, a series of one hundred and ten different simulations would require eleven hundred simulation runs. The list of variable parameters is shown in Table 6.3.

Table 6.3: Simulation variable parameters.

Percentage servers 20-100

Servers required 1-5

Trust threshold 0.1-0.9

Node speed 0-100kmh

During the simulation, nodes in the network would request communication with other nodes in the same network. The process of contacting a server, validating the certificates of both parties, contacting the other node to request communication and receive a reply was performed at each request. The nodes could then choose to use symmetric or asymmetric encryption as desired. The parameter of 0.1 requests for communication per second was set so that nodes that were part of a network and held a valid certificate would request communication with another node once every ten seconds. The purpose of this message exchange was to give an opportunity for malicious nodes to misbehave as the simulation ran so that misbehaviour did not just occur when a certificate issuance request passed through them. Whilst this process added considerably to the realism of the growing networks, metrics were not recorded in the results as this did not add to the performance measurement of the protocol.

Stage eleven involved collating the results and comparing them with what was expected prior to the simulation runs. Results were positive and showed the protocol performed as expected.

Finally, stage twelve involved documenting the results and displaying the identified trends in meaningful ways by construction the tables and graphs of the results.