DSS DEVELOPMENT AND EVALUATION

The review in this section is categorised under three main areas: (a) general; (b) user involvement; (c) components of a DSS (such as decision making, model base, database, and interface); (e) test and evaluation.

(a) General

As Loucks and da Costa (1991) stated, the following questions should be considered in developing an effective DSS:

• What database system should be developed or purchased; how can security and control of access be achieved; how can data errors be detected and corrected; how can missing data be replaced; how can data be maintained and augmented; how can feedback be established between data providers and data users; how can data be accessed when on different computers at different locations; and how can a database system be integrated within the decision support system?

• Which models and model types are most appropriate for the client; how can the model input data be managed, checked and displayed; how can model documentation, playback files, training tutorials and user’s manuals best be prepared for efficient and effective technology transfer; how can the models be used to explore and synthesize as well as analyse alternatives, their impacts, and their uncertainties; how can sensitivity analyses and model calibration be performed most effectively; and how can the model results, together with their uncertainties, be displayed for maximum understanding?

• What software language and graphics tools to use; how should different modules developed by different developers linked together; how should software testing be performed; how should model misuse be prevented; what levels of on-line help should

be provided; how much effort should be placed on the models themselves, the database, interface, and all necessary and desirable training and documentation?

• How can close cooperation and communication among the developers and the clients be established and maintained; how can the maintenance and upgrading of the decision support systems be assured for changes in needs, data, and institutional procedures? (b) User Involvement

Alter (1978) used examples from a survey of fifty-six decision support systems to illustrate each of four implementation patterns that were directly observed in the sample. Alter defined four DSS development patterns in terms of combinations of high or low degree of (1) initiation by the user and (2) participation of the user in the development process. It was found that systems initiated by high degree of users and implemented with a high degree of user involvement comprised of less than one fourth of all cases.

Barbosa and Hirko (1980) pointed out the conflict of rigid design against user’s changed perception of the problem. For example, as users develop further understanding of the problem in a DSS development, their perception of underlying structure of the problem could be changed. This has brought in a conflict that the rigid framework in which the algorithm or model resides cannot easily match the user’s changed perception of problem.

Through three DSS case studies, Mann and Watson (1984) indicated that the level of user involvement can vary significantly. They believed the DSS technology (i.e. DSS tools and generators), the management activities and the nature of interdependence work together to influence the level of the user involvement. A tentative contingency model, which included intermediate involvement and technician involvement as well as user involvement, was then developed and discussed in their research.

(c) Component of a DSS

• Modelbase

Barbosa and Hirko (1980) discussed using algorithmic aids or models for decision making to improve interface, feedback, flexibility and integration in DSS. The specific algorithmic approaches were applied to a GEO-Data Analysis and Display System. The difficulties of

using algorithms were discussed such as obtaining the user acceptance of the assumptions underlying the model, which the algorithm supported. Two general approaches, which can overcome these general difficulties in the use of algorithms, were highlighted, and one of them was the interactive algorithm approach. The interactive algorithm approach is that the DSS designer develops a procedure by which information can be mined from the user and the user may describe his/her problem parametrically. The idea is similar to that in an interactive hydrological or hydraulic model.

Huber (1981) discussed four models characterising the organization decision making environment and presented the information requirements and analytic aids for each model when designing and developing a DSS. This article makes clear that the nature of organization decision environment will more and more be seen as a key variable in the development of DSSs.

Some factors regarding models and model management were given by Sauter (1997) as follows:

ƒ For modelling: functionality, flexibility, appropriateness of included models and ease of use.

ƒ For system architecture: analysis capabilities, sensitivity analysis, what-if analysis, impact analysis, symbolic reasoning evaluation, and context-sensitive model help.

• Interface:

According to MacLean (1986), the specification in the design and development of user interface should be prepared considering the following factors: displays and controls, user- system dialog, consistency, re-entry dialog, alternative data entry methods, error routines, report capabilities, online report display, Ad Hoc query language, online report help facility, graphics display capabilities, data review and modification, bulk data maintenance tasks, external data capture, online data maintenance, etc.

Loucks and Costa (1991) believed the most important features of user interface should include: (a) user dialogues, symbolic interaction, and possibly embedded rule-based methods, (b) graphic visualization and display options, (c) integration and transparent data handling, and (d) maintenance and customisation. Sauter (1997) described user

friendliness, support of modelling and data needs, graphics, and reporting forms as user interface considerations in DSS.

• Database:

The most reported DSS applications include a database or a database subsystem. The major differences in the database part in these applications were database software, integration technologies and related functions in the database subsystem. Different database software had been used such as HEC-DSS developed by U.S. Army Corp of Engineers (Ford and Killen, 1995), Oracle by Oracle Corporation (Dunn et al., 1996), MS ACCESS by Microsoft Corporation (Shim et al., 2002), and ArcView by Environmental Systems Research Institute (ESRI) (Makropoulos et al., 2003).

• Spatial and graphic data Display and Analysis (SGDDA):

SGDDA is reviewed in detail in Section 2.5, and therefore not described here.

• Decision Support:

Brown and Jones (1998) developed a model of aided decisions that incorporated both the level of reliance on the decision aid and the agreement of decisions with the decision aid's advice. The model also included the four classes of factors that could influence the decision strategy selection: decision aid features, decision-maker characteristics, decision task characteristics and factors affecting the evaluation of decision strategies. By comparing aided and unaided decision makers (the aided decision maker received decision making support while the unaided decision maker did not), it was found that the decision aid (or decision support) had greater influence in making decisions in the harder tasks than in the easier tasks. More details on decision choice support are given in Section 2.6.

(d) Test and Evaluation

When a DSS is developed, it is necessary to test the functionality of the DSS. It was found that there is no detailed test methodology specified in published DSS applications, although some developers have discussed this issue in their evaluation sections (Mowrer, 1997). However, the test technology frequently used in computer information system domain can be

used for DSS testing. Evaluation of a DSS is more than its testing for functionality. DSSs have been evaluated in the past considering many issues. For example, Power (1997, 2002) suggested to evaluate a DSS by its (a) capabilities, (b) cost of the package, (c) ease of use, (d) ease of installation and operation, (e) performance, (f) vendor reputation and (g) reliability. Mowrer (1997) employed seven criteria categories to evaluate the computer-aided decision support system already developed for forest ecosystem management, namely the scope of each developed DSS, the spatial capacities, the computational methods, the development status, the input and output requirements, the user support availability, and the system performance. Mowrer’s evaluation work was based on the questionnaire responses from DSS developers, and these responses were presented to compare the system capabilities. Some common factors for evaluating DSS software packages such as cost of the package, vendor reputation and ease of installation were not considered in Mowrer’s evaluation work.

The five subsystems, namely, Interface, Modelbase, Database, SGDDA and Decision Support were developed in DSSFCMR in this research project with many functionalities for flood forecasting and warning. The developed DSS was properly tested. The detailed test technology used in this research for development of DSSFCMR is given in Chapter 5.