The development of an information modelling
system for regional water resource assessments
D E N I S H U G H E S
Institute for Water Research, Rhodes University, Crahatnstown 6140, South Africa
e-mail: denis@.iwr.ru.ac.za
Abstract The paper outlines the development of a software package that integrates spatial and time series information in a flexible framework that allows for a wide variety of data storage, retrieval, analysis, display and modelling options. In the context of regional water resource assessments, the advantage of such a tool is that a wide variety of information and data analysis methods can be brought together into a single package. This allows a wide range of users to make use of the same information storage system and apply consistent methods to water resource assessments, permitting a greater degree of commonality of approach within the region.
Key w o r d s databases; software; modelling; water resources
I N T R O D U C T I O N
During the late 1980s and early 1990s, the Institute for Water Research (IWR) developed a D O S based hydrological model application system ( H Y M A S ) (Hughes et al., 1994) which has proved to be a very useful and flexible tool for applying a variety of hydrological and water resource models. However, the system has some major disadvantages, which are mainly linked to the methods of information storage and the lack of a spatial reference component. When consideration was given to converting the package to a Windows environment, it was decided to adopt a different approach that would enhance the capabilities of the new system. The approach being adopted by the Centre for Ecology and Hydrology at Wallingford, U K (CEH) provided a large part of the inspiration for the design of the system that is under development at IWR. While some of the motivation for the development came from the need to replace H Y M A S , there was also a pressing need for integrating the methods used in determinations of instream flow requirements to support the new South African Water Law (Department of Water Affairs and Forestry, 1997). This paper provides a brief summary of the software design considerations and an overview of the developments that have been completed so far.
D E S I G N C O N S I D E R A T I O N S
The new software is referred to as SPATSIM which stands for SPatial And T i m e Series Information Modelling. The main design characteristic is the use of a spatial interface to control access to other information stored in database tables. The software design had to account for the following facilities:
(a) importing/exporting information from/to a variety of different raw data formats, (b) efficient access to the information,
(c) editing and displaying the information,
(d) generating secondary information and storing it,
(e) using the stored information as input to a wide variety of analysis methods and specifically hydrological or water resource models.
One of the experiences inherited from the development of H Y M A S was the need to incorporate an approach whereby new data modelling methods could be added without the need to modify the core software program. This suggests that a generic interface to external programs is required which should be as simple as possible such that others, beside the main development team, can contribute to the package without needing to modify the main program.
S O F T W A R E D E S I G N O V E R V I E W
There are four main components of the design; the spatial interface, the main database structure, the internal utilities and the generic approach to accessing external utilities and models. The software is being developed using DELPHI code due to the personal preferences of the developers but any other visual language would have been equally suitable.
The s p a t i a l interface is achieved using ESRI MapObjects© and shape files, which have been captured externally. Each shape file is identified in SPATSIM as a "feature" and while they may contain additional information, this is not used within SPATSIM and the information "attributes" associated with the spatial elements are stored in separate database tables. The main reason for this is the fact that shape files do not allow for the storage of time series data and the need to be consistent in the handling of different types of information.
Table 1 Structure of the data dictionaries controlling the interface between the spatial and other data
attribute components of SPATSIM.
Data dictionary 1 Data dictionary 2 Data dictionary 3 Data dictionary 4
Feature code Feature code Attribute code Attribute code
(unique number (link to data (link to data (link to data
identifying the feature) dictionary 1) dictionary 2) dictionaries 2 and 3)
Feature name Attribute name Table code Spatial record
(unique number (IDField value of the
identifying the table) spatial element)
Feature source Attribute code Database alias Table record
(path and name of the (unique number (name of the database (RECID field value of
shape file) identifying the attribute) where the attribute data the attribute table) are stored)
IDField Datatype (0 to 7) Table name
(shape file field that defining which data (name of the table that
identifies a point, line or type stores the attribute data)
polygon)
DescField Max. rec
(shape file field selected (maximum record
as a description of the number in the attribute
The main database structure consists of the shape file tables, four data dictionary tables and data tables to store the attribute data associated with each spatial element (polygon, point or line) of a shape file. The shape file tables are required to have a field containing a unique numeric reference to each spatial component (IDField), and a text field containing descriptive information. Dictionary 1 (Table 1) defines the features that have been loaded into an application of SPATSIM, while dictionary 2 links the features with the names, codes and data types of any associated information (attributes).The attributes can be text (up to 80 characters), single integer or real numbers, time series, graphic (pictures or short video clips), arrays (one- or two-dimensional arrays of real numbers, e.g. mean monthly values for several variables), memos (text longer than 80 characters) and linked (links to an attribute stored for a different feature). Pre-defined database table structures are provided for each data type and all have a common RECID field, which is unique for each record within a single table. For example, the time series type tables consist of the RECID field, 15 fields defining the format, data units, start and end date, etc. of the time series and a BLOB (binary large object) that stores the full time series in a single field.
Data dictionary 3 links the database tables with the attribute codes of dictionary 2 (Table 1). All information associated with the same attribute is stored in the same table. Furthermore, it is possible to store the same type of data, but associated with different attributes, in the same table. The final dictionary component (4) provides the links between the attribute codes, the spatial record (IDField in the shape file) and the attribute database table records (RECID fields). Once a feature has been displayed, an attribute and one or more spatial components of the feature selected, the data dictionaries efficiently access the correct record in the appropriate attribute table. The only unresolved issue is the efficiency of searches in dictionary 4 when a large number of features and attributes are involved. The current test database at IWR has nearly 32 000 entries in dictionary 4 and identification of data table records is instantaneous.
The internal utilities are facilities that are being incorporated into SPATSIM as part of the main program and fall into several categories. They have been identified as being generally applicable to most applications of SPATSIM:
(a) Map or feature manipulation facilities. These include facilities to add or remove features from the database, modify features (e.g. add, remove or m o v e points), annotate and render features, as well as create new features from merged or intersected data.
(b) Attribute manipulation facilities. This group offers a wide range of facilities to manage attribute information and includes creating new empty attributes, removing attributes, loading attribute data from different raw data sources, displaying and editing attribute data, as well as exporting data for use with other software. These are critical components of SPATSIM if it is to be useful as a regional water resource analysis tool, as it is important for users to be able to import data efficiently and accurately.
(c) Spatial interpolation facility. A frequently required facility for hydrological and water resource modelling is the need to interpolate from point data to generate areal averages (converting point rainfall data to basin rainfall data, for example). A procedure has been developed that allows point data to be spatially interpolated to areal data using an inverse distance weighting method plus an additional weighting using gridded mean monthly data.
(d) More specialized facilities. As a large part of any S P A T S I M database is expected to consist of time series data, several specialist facilities are being included that represent standard summary analyses of time series and which store the results in another attribute. Examples include the generation of monthly and annual duration curve data (an array attribute), monthly means, medians and standard deviations, as well as low- and high-flow frequency analysis.
E X T E R N A L U T I L I T I E S
At present there is one specialized external utility and a generic facility for running other external programs. The specialized utility is a time series display and analysis program (TSOFT) that had already been developed before the design for S P A T S I M had been completed. The T S O F T program (Hughes et al., 2000) operates with a "profile", which is a reference to sources of time series data stored in several different ways (database tables, binary files, etc.). The S P A T S I M procedure is simply to identify the required time series attribute and click on the spatial elements that one
Table 2 Example of a model requirement file for a monthly semi-distributed rainfall-runoff model (the Pitman model).
File contents Comments
Pitman model requirements
1 st parameter = 0:Required, 1 :Optional 2nd parameter = Table type (-9, Multiple variable file; 99, Other file; else Attribute datatype code)
1. EXE file 0 99 mpit.exe
2. Output requirement file 0 99 Pitman.rqo 3. Model results data
0-9 4. Basin area
0 2
5. Physiographic parameter data 1 5
6. Model parameter data 0 5
7. Mean monthly evaporation 0 5
8. Mean monthly distribution data 0 5
9. Basin average rainfall 0 3
10. Basin average potential evaporation 1 3 11. Upstream inflow 1 3 12. Transfer inflow 1 3 13. Downstream outflow 1 3
Three standard header lines followed by two lines for each requirement. The second line holds two parameter values and sometimes a file name.
The external program name to call from SPATSIM for running the model.
Defines the structure of a temporary output file created by the model if necessary.
Name of the temporary output file created by the model if necessary.
Single real value attribute requirement.
Optional array type attribute data for some basin variables. Array type attribute data storing the model parameters. Array type attribute data storing the seasonal distribution of evaporation.
Array type attribute data storing the seasonal distributions of other data.
Time series type attribute storing the rainfall input to the model.
Optional time series type attribute storing the evaporation input to the model.
Optional time series type attribute storing any upstream inflows.
Optional time series type attribute storing any artificially transferred inflows.
Optional time series type attribute storing the modelled down stream outflows and written back to SPATSIM by the model.
wishes to add to a "profile". When T S O F T is called from SPATSIM all o f t h e selected time series are available for display and further analysis.
A generic facility has been included to link a wide variety of external models and other data analysis programs to SPATSIM, without having to re-code the main program. All models will inevitably have different data requirements and therefore one of the linking facilities required is to be able to associate data sources (attributes) with model requirements. This has been achieved through the use of model requirement text files and an example for a monthly rainfall-runoff model is given in Table 2. The exact procedures followed within SPATSIM depend upon the type of model or analysis procedure to be applied.
(a) Select the feature and an attribute that defines the drainage links between spatial elements. The latter is required only for semi-distributed type models that need to know the upstream/downstream relationships between spatial elements.
(b) Select the spatial element(s) that is(are) to be modelled. In the case of semi-distributed models this involves clicking on the most downstream element first and allowing the drainage link attribute to define all those upstream.
(c) Select the model requirement file that defines the model or analysis process to be run. This displays a table of requirements (Fig. 1).
(d) For each requirement (either model inputs or outputs), select an attribute that satisfies the required data type (Fig. 1).
(e) Save the requirement information and run the model.
Features Attribute Procedure Application
[ethhH a] | _ J
Attributes
Incremental Monthly Floi * I MAR
Monthly Evap. Distribute Monthly Water Use Net Area (krrT2) Pirnan Model Parameters Pitman Model Results
WFS9C
Yield Estimation Paramer
Requirement File/Attribute
1. EXE File mpit.e::e
Pit matt, rqo
Piman Model Parameters 2. Output Requirement File
mpit.e::e Pit matt, rqo
Piman Model Parameters 3. Model Résuit* data (T/S)
mpit.e::e Pit matt, rqo
Piman Model Parameters 4. Catchment Area
mpit.e::e Pit matt, rqo
Piman Model Parameters
•5, Physiographic Parameter Data
mpit.e::e Pit matt, rqo
Piman Model Parameters 6. Model Parameter Data
mpit.e::e Pit matt, rqo
Piman Model Parameters ?. Mean Monthly Evaporation
mpit.e::e Pit matt, rqo
Piman Model Parameters
S. Mean Monthly Distribution Data
mpit.e::e Pit matt, rqo
Piman Model Parameters
9, Catchment Average Rainfall ("DS) WR90 Zone Rainfall
10. Catchment Average PE (T/S)
11. Upstream Inflow (Tf~S) =12. Transfer Inflow (Ttë)
Run Process Save Requirements Exit
Maps ) Add Data j Attributes J Add Arrays) Spatial Interim
The attribute requirements and the relevant spatial element details are stored in a single requirement database table, which is accessed by all the external processes. The code to access the contents of this table will vary from process to process, but generic rules are being developed for distribution to other potential developers. One advantage of this approach is that once an application has been established and saved, the model can be run independently without entering SPATSIM first.
The models that have already been established include the Pitman monthly rainfall-runoff model (Pitman, 1973), several models that support the quantification of the ecological reserve for South African Rivers (Hughes, 2001) and the so-called "Patching" model of Hughes & Smakhtin (1996). Plans are well advanced to add a daily rainfall-runoff model and to cooperate with the University of the Witwatersrand, Centre for Water in the Environment, on adding a river hydraulics model.
D I S C U S S I O N A N D C O N C L U S I O N S
One of the critical issues associated with developing this type of software is that the basic design should be sufficiently robust that it does not require changing significantly as new facilities are identified and added. Additions to the software can then be made quickly and efficiently and total development time is reduced. In the case of SPATSIM, there are a number of facilities that have yet to be added, while others are being improved and made easier to use. However, the software has already been widely used for research and practical purposes associated with setting instream flow requirements for the new South African Water Law, as well as being the main tool for a regional rainfall-runoff modelling study in Zambia.
During the development of SPATSIM, the South African water resources database (WR90—Midgley et ai, 1994) was used as one of the main sources of information for testing. This database consists of monthly rainfall, streamflow, evaporation, Pitman model parameters, etc. for 1946 quaternary basins covering the whole of South Africa, Swaziland and Lesotho and includes the relevant spatial information. Within IWR, SPATSIM has now become the primary tool for accessing the W R 9 0 database. On the basis of this experience, this type of approach has been recommended for use in a proposed South African Development Community (SADC)-wide water resource assessment project (Water Research Commission, 2001). This project proposes the use of a single rainfall-runoff model to develop a database of simulated naturalized streamflow for approximately 10 000 sub-basins within the region. It is inevitable that a large amount of supporting information (observed flows, rainfall, evaporation, water use, basin physical characteristics) will also be required and for the region to obtain the maximum benefit from the project, a range of data display and analysis facilities will be required.
T w o of the most important components of the proposed SADC water resource assessment project are the need to develop a common understanding of water resource issues and to build capacity in water resource analysis methods. Experience suggests that the development of capacity, as well as common understanding, can be more readily facilitated when similar tools and methodologies are employed throughout the region. Rather than having a number of different tools, all of which need to be learnt and supported, the development of experience in the use of a single flexible tool should
promote further learning. It must be noted that this does not refer to the use of a single model or analysis method, but a single framework in which a variety of models can be applied. Promoting a tool similar to SPATSIM, does not therefore amount to promoting total standardization and the institutionalization of water resource analysis methods within the region.
The transfer of analysis methods within a large region is often difficult because methods are developed to access a specific data format and new users have to convert their data. It is the intention of I W R to add a wide range of time series data import options that will ultimately be able to cater for any data type that is found in the SADC region. Experience of these different data types has already been gained through the Southern African FRIEND programme (Hughes, 1997).
In conclusion, SPATSIM offers a flexible and efficient approach to storing water-resource-related information, as well as a vehicle for carrying out a wide variety of data analysis and modelling methods. It is still in the early stages of development, but has already been found to be a useful tool for several different purposes. As it becomes more widely used it is inevitable that improvements will be identified and new facilities added. The bulk of the development work has been carried out under funding from the Water Research Commission of South Africa and therefore SPATSIM is available free of development costs and is not a commercial software package. There are, however, distribution costs related to MapObjects applications and consideration will have to be given to future development and support costs. These issues will be resolved as and when there is broader interest in the use of the software.
R E F E R E N C E S
Department ofWaler Affairs and Forestry (1997) White Paper on a National Water Policy for South Africa. Dept of Water Affairs and Forestry, Pretoria.
Hughes, D. A. (1997) Southern African "FRIEND"—the application of rainfall-runoff models in the SADC region. Report to the Water Research Commission hy the Institute for Water Research, Rhodes University. Water Research
Commission Report no. 235/1/97, Pretoria, South Africa.
Hughes, D. A. (2001) Providing hydrological information and data analysis tools for the determination of ecological inslream flow requirements for South African rivers../. Hydrol. 241, 140-151.
Hughes, D. A. & Smakhtin, V. (1996) Daily How data time series patching or extension, a spatial interpolation approach based on How duration curves. Hydrol. Sci. J. 41(6), 851-871.
Hughes, D. A., Forsyth, D. & Walkins, D. A. (2000) An integrated software package for the analysis and display of hydrological or water resources time series data. Water Research Commission Report no. 867/1/2000, Pretoria,
South Africa.
Hughes, D. A., Murdoch, K. A. & Saini, K. (1994) A hydrological model application system—a tool for integrated river basin management, in: Integrated River Basin Development (cd. by C. Kirby & W. R. White,), 397-406. John Wiley, Chichester, UK
Midgley, D. C , Pitman, W. V. & Middlelon. B. .1. (1994) Surface water resources of South Africa 1990. A user guide plus six vols of data appendices and a map. Water Research Commission Reports 298/1/94 to 298/6/94, Pretoria, South
Africa.
Pitman, W. V. (1973) A mathematical model for generating river Hows from meteorological data in South Africa. Report
no. 2/73, Hydrological Research Unit, University of the Witwatersrand, Johannesburg, South Africa.
Water Research Commission (2001 ) Assessment of surface water resources. PCN14 terms of reference. Complied for the SADC Water Sector Coordination Unit. Water Research Commission Report no. 1112/1/01, Pretoria, South Africa.
