Testing the Models

Once the existing models were run and runoff hydrographs generated for the selected high flow events, the observed and predicted storm hydrographs were compared and the error between the hydrographs quantified to allow conclusions to be drawn regarding the ability of the models to predict flood flows for the current land use in each catchment. The models were run using stochastically disaggregated daily rainfall as a model input and daily rainfall disaggregation based on station data as inputs. This was intended to allow quantification of the effect each method of rainfall disaggregation had on the accuracy of the models.

Key outputs considered in the model evaluation were the values of objective functions applied to the observed and predicted hydrographs and a visual comparison of the model output and observed hydrographs, which identified any obvious error.

While a number of objective functions have been discussed, many of them employ similar mechanisms and can be expected to give similar measures of error. Hence the NSE was the primary objective function used to evaluate model performance. PEPF and TPE were also employed as these simple objective functions focus on important parameters in flood modelling of peak flow magnitude and the timing of the flood peak. The value of the objective function that constitutes a ‘good’ or ‘acceptable’ model prediction was subjective, although it had been suggested that a NSE of 0.7 was acceptable and able to explain the error

56 in a model (Bandaragoda et al., 2004). The relative bias Rb of each model prediction was also calculated since model bias can influence the efficiency of a hydrologic model.

The objective functions focussed on how accurately the models predicted peak flows, but it was important to identify which section of the hydrograph before and after the peak to include in the evaluation. Selection of the period over which the objective functions were applied was subjective and determined on a case-by-case basis, ensuring that each evaluation period was long enough to include the rising limb of the hydrograph, the flood peak, and a reasonable section of the falling limb of the hydrograph. In most cases, this occurred over a period of between 72 and 120 hours (3 to 5 days), and appeared to be dependent on the flood magnitude and the duration of the storm.

To assist in the evaluation of the model, and in an attempt to identify rainfall input as a potential source of error in the model predictions, the observed rainfall hyetographs and the rainfall hyetographs input to the models by the VCSN were compared. Attention was given to the total rainfall across the catchment and the distribution of the rainfall. The Ahuriri River catchment had a consistent rainfall record from four gauging sites within the catchment boundaries, and two of the stations were operational at any time. Hence, it was expected that comparing the observed rainfall hyetograph with the rainfall provided by the VCSN in the Ahuriri River catchment would allow the error in rainfall input to be quantified. The rainfall for each event was taken as an average of the observed rainfall at the active precipitation gauge sites within the Ahuriri River catchment that recorded the event (Figure 3-13). The Pelorus River catchment, however, did not have a rain gauge station within its boundary; rather the rainfall data was provided from two or three stations out of a total of five stations, depending on the timing of the event, located near the downstream end of the catchment (Figure 3-13). A comparison between the VCSN rainfall estimate and the observed rainfall data for each event used in the evaluation of the model was still conducted for the Pelorus River catchment. However, the observed data may not have been an accurate reflection of the actual precipitation event that lead to the high flow event in the Pelorus River.

The models were also run over the period used by NIWA in the calibration of each model. The models were calibrated to a stochastic disaggregation of the daily rainfall estimate. In effect, by using the daily rainfall estimate disaggregated into hourly rainfall using observed station data as a model input, as well as running the model using stochastic disaggregation of daily rainfall as an input, the model simulated a different rainfall series, albeit with the same

57 net rainfall over the period. It was intended that this would assess the calibration of the model, attempt to quantify the difference between the two rainfall input methods, and provide some insight in to the accuracy with which the calibrated model parameters represented the physical processes in the catchment.

The evaluation of the models also determined which method of rainfall disaggregation to pursue for the remainder of the research project. While the models were calibrated using stochastic disaggregation of daily rainfall, station-based disaggregation may yield an improved rainfall input to the model and a greater level of accuracy in the model predictions, depending on the quality of the observed station data.