Relative Costs

In this section, the costs of modelling program modifications relative to the costs of realising them are discussed. In the previous section, it was assumed that these costs were negligible; this assumption is relaxed in this section and the next. In this section, the costs themselves are considered; in the following two sections, the effects of finite costs on risk management are considered. In all three sections, assumptions about costs based on estimates are used; the rationale for this was discussed in Section 2.5. The importance of model modification costs may be understood by considering extreme cases. If all of the modifications evaluated by modelling are eventually realised, there may be no saving and, indeed, the costs of modifying the model will increase the overall cost of tuning. Hence, a lower cost of modifying the model results in a lower overhead on successful modifications. If many modifications are modelled but are found to be unsuccessful, the cost of modelling is thoroughly justified, because the costs of modifying the program have been saved. However, the cost for modifying the model is still important, because a low cost of model modification allows the tuning tree to be searched more thoroughly.

Given a gcw&^-generated DAG program which requires tuning, the first step is re-generate the corresponding model harness from the common modules. If necessary, the computation grain sizes are re­ measured to generate the model-specific modules. This gives all that is required to compile and link a model simulation. When the indirect tuning is completed, the gendag utility is used to create a new

program harness. The tuning plan is then realised. This may involve modifications to the computations and to the hardware to reflect the successful tuning plan.

The cost of software engineering is often the major component of cost in the direct tuning of programs. These costs are not incurred in model modification because models abstract from the actual computations carried out. Modifications to the computations in a program are expensive because they demand that computations be re-structured in various ways. The activities involved include analysis of computations prior to modification, estimation of the effects on grain sizes, implementation of modifications, testing of modifications, testing of the system as a whole and debugging. The modifications to the model, by contrast, concern only measurements or estimates of the amount of computation in each grain and the structure of grains and messages in the tasks, so the cost of evaluating a modification using a model can therefore be much less. However, the level of cost varies, depending on the type of modifications considered. For each type of modification, costs are considered for programs in general and then for the particular case of DAG programs.

For event tuning modifications, such as modifications to the mapping of tasks to processors, to task priorities, the order of messages and to buffering between tasks, the realisation costs can be very low. Modifications to mappings, priorities and message orders can be implemented by changing values in the program configuration file. Modifying the buffering between tasks involves adding a simple task to a configuration file and connecting it between the tasks it supports; the modifications can be applied to a model by altering the corresponding values in the model script. In the particular case of DAG programs developed using gendag, event tuning modifications can be evaluated by changing the common modules, altering values such as task priorities and task mappings or the features of the common modules such as message orders and the buffering between tasks. The model can be re-generated after each modification to evaluate performance. The modifications made to the common DAG program specification module are automatically transferred to the program harness when it is re-generated after tuning is complete. In terms of the cost of software engineering, indirect tuning offers no saving with respect to direct tuning, since the modifications take a matter of minutes to carry out in both cases. However, savings are made when the cost of program execution is non-negligible or when the program run-time causes significant delay to the project, since models can generally be executed quickly, on a low cost platform.

Grain size reductions are expensive to implement in software engineering terms. The computation must be tuned and then tested. Programming errors are easy to introduce and hard to remedy. However, grain tuning modifications can be modelled in a matter of minutes, because this simply involves reducing the grain size value. A similar argument applies to reductions in the sizes of messages. For the particular case of DAG programs, a grain tuning modification can be modelled by modifying the grain size values in the model-specific modules.

Parallelism tuning is time-consuming to implement because it involves re-arranging computations among tasks and re-structuring the computations and messages so that each task has the data it requires. The costs of analysing the computations, modifying serial code, testing and debugging can amount to days or

weeks of effort. However, it is possible to abstract from these details when modifying the model, since parallelism tuning involves replacing a grain by a model of a sub-program, modifying the model of the program to reflect the proposed changes to messages and grain sizes. These modifications can often be modelled in a time-scale of a few minutes. In particular, parallelism tuning for DAG programs involves modifying the common modules to reflect the required task topology and setting up estimated grain sizes in the model-specific modules. The work of analysing, modifying and testing the actual computations need not be carried out until the model shows that the modifications are successful.

Modifications to the hardware in the system itself to support direct tuning involve very substantial costs. Firstly, hardware must be acquired and installed. The financial costs of acquisition, even for relatively low- cost digital signal processing systems, can be of order £10000 or more, but costs can greatly exceed this figure for different types of hardware. Secondly, software modifications are often needed to adapt the software to the new hardware. At worst, software must be ported to different processors and to different system software, the effort for which may involve engineering effort of weeks or months. With indirect tuning, a model of the hardware can be constructed by setting up processor and link simulation modules in the topology required. The costs are lowest if simulation modules for hardware and system software components have already been constructed and can be drawn from a library. Further reductions in costs can be often be achieved if graphical tools have been provided to set up the required topologies. Given suitable modelling tools, therefore, the model of the hardware may take as little as a few minutes to modify.

A summary of the cost estimates is presented in Table 3.

Type of Tuning Cost to Resolve Risk with Direct Tuning

Cost to Resolve Risk with Indirect Tuning

event minutes minutes

grain size hours minutes

parallelism days minutes

hardware days/£10ks minutes

Table 3 : Estimated Costs of Program Modification and Model Modification