Comparative Study between RMSNN and DMSNN Models

As mentioned in Section 6.4.1, forecasting was carried out over a 12-month period at each current time step of the validation period except for the last 12 months. It was found that performance of both RMSNN and DMSNN models (i.e., model 3 and 8 in Tables 6.2 and 6.3 respectively) decreased with increasing forecasting lead time in terms of forecasting ability. It was also found that the models performed poorly beyond the forecasts of first 6 months. Therefore, only the results up to 6 month forecasts were used for comparing the performance of the two models.

The comparison of the original NADI time series computed from observed data and the forecast NADI time series from both RMSNN and DMSNN models considering 6 month forecasts are presented in Figures 6.6 and 6.7 respectively. Each figure has six components, showing how 1, 2, 3, 4, 5 and 6 time step forecasts compared against computed NADI values. In these figures, comparative performance evaluations are presented for all three data sets (i.e. training, testing and validation) to show the optimum generalization ability of the forecasting models that was achieved in the model calibration process. The R values between the observed and the forecast NADI values are also shown in these figures. It can be seen from Figures 6.6 and 6.7 that the forecast time series matches well with the original NADI time series for all three data sets when forecast lead times are smaller and the differences become larger when the lead time increases. The R values show that they are close to each other for forecasts in all three data sets up to 3 months ahead, which implies the good generalization abilities of the forecasting models up to 3 months ahead forecasts.

Moreover, the R values within the three data sets decrease when forecast lead times increase showing the generalization abilities of the forecasting models decrease with the increase in forecasting lead time.

Figure 6.6 Comparison of computed NADI time series with forecast NADI time series from RMSNN model (i.e., Model 3)

Figure 6.7 Comparison of computed NADI time series with forecast NADI time series from DMSNN model (i.e., Model 8)

The R, RMSE and MAE values of each lead time forecast obtained by both RMSNN and DMSNN models for the validation data set are presented in Table 6.5. It shows that in both RMSNN and DMSNN models, R values decrease, and RMSE and MAE values increase when forecast lead time increases, which are expected. Moreover, Table 6.5 shows that both models produce the same R, RMSE and MAE values for one 1-month lead time. However, the forecasting performance is slightly better in the RMSNN model than in the DMSNN model for 2 to 3 months ahead forecasts in terms of RMSE and MAE, although the performance of both models is the same in terms of their R values. However, the DMSNN model results are slightly better than those of the RMSNN model for forecasting lead times 4 to 6 months. In terms of the R values, the forecast NADI values were statistically significant at 0.01 level in both RMSNN and DMSNN models.

Table 6.5 R, RMSE and MAE values for validation data set

t+1 t+2 t+3 t+4 t+5 t+6 RMSNN 0.75* 0.70* 0.64* 0.56* 0.49* 0.48* R DMSNN 0.75* 0.70* 0.64* 0.58* 0.50* 0.49* RMSNN 0.61 0.67 0.76 0.93 1.05 1.08 RMSE DMSNN 0.61 0.70 0.81 0.92 1.02 1.07 RMSNN 0.49 0.53 0.60 0.73 0.81 0.87 MAE DMSNN 0.49 0.56 0.64 0.72 0.80 0.86 *Correlation is statistically significant at the 0.01 level

The forecast values of NADI were also compared by classes of different dryness/wetness (or drought classifications in Table 5.1 in Chapter 5) to evaluate the accuracy of forecasts across events of different extremity, which is also ensure cross validation of the forecasting model. This approach was also used by Morid et al.

(2007). For easy reference in this chapter, Table 5.1 is reproduced as Table 6.6, it should be noted that wetness thresholds are also included in this table as these thresholds were used in the evaluation of forecast accuracy study.

Percent of forecast accuracy in the validation data set is presented in Table 6.7 for forecasts up to 6 time steps. In this table, ‘1’ in the DC column (DC = Difference in Classes) means that there is only one difference in relation to the level of dryness/wetness between the original and the forecast NADI values (e.g., if ‘Extreme drought’ is observed in the original computed NADI series and ‘Severe drought’ is

forecasted in the forecast NADI series, then DC = 1). Table 6.7 shows that when forecasting lead time increases the percent of forecast accuracy decreases, as expected. Moreover, for all 6 month forecasts, the DC values of the RMSNN model forecast with 0 (i.e. original drought classes are exactly same as forecast drought classes) are between 52% and 67%, whereas the corresponding figures in the DMSNN model are between 53% to 67%. However, if both 0 and 1 DC values are considered together, the percent of accuracy becomes between 88 to 98% and 85 to 98% for all 6 month forecasts for the RMSNN and DMSNN model respectively. As was seen before (Table 6.5), Table 6.7 also shows that both RMSNN and DMSNN models show same forecasting performance for 1-month lead time. Moreover, the table also shows that the RMSNN model gives slightly better forecasts than the DMSNN model for 2 to 3 months ahead forecasts, while the DMSNN model forecasts are slightly better than the RMSNN model forecasts for 4 to 6 months ahead forecasts.

Table 6.6 Drought classification based on NADI thresholds* > 1.63 Extreme wet > 1.20 to ≤ 1.63 Severe wet > 0.88 to ≤ 1.20 Moderate wet > -0.84 to ≤ 0.88 Near normal > -1.64 to ≤ -0.84 Moderate drought > -2.27 to ≤ -1.64 Severe drought ≤ -2.27 Extreme drought *Reproduced from Table 5.1

Table 6.7 Percent of forecast accuracy in dry or wet classes – validation data set

DCa t+1 t+2 t+3 t+4 t+5 t+6 RMSNN 0 67 66 64 53 53 52 1 31 30 31 43 38 36 2 2 4 4 3 8 11 3 0 0 1 1 1 1 (0+1) 98 96 95 96 91 88 DMSNN 0 67 64 58 57 54 53 1 31 32 35 31 32 32 2 2 4 5 11 12 13 3 0 0 1 1 2 2 (0+1) 98 96 93 88 86 85

6.5.

Summary

Drought forecasting is important in terms of planning and operation of water systems, especially during continuing dry climatic periods. In the context of drought forecasting, researchers and professionals have used several techniques in the past for forecasting Drought Index (DI) values as the forecasts of future drought conditions. Based on the study on several existing DIs (Chapter 4), a new DI namely Nonlinear Aggregate Drought Index (NADI) was developed in this study which was presented in Chapter 5. The NADI defines the overall dryness as the drought condition within the system by considering all significant hydro-meteorological variables (i.e. rainfall, potential evapotranspiration, streamflow, storage reservoir volume and soil moisture content) that affect the droughts. The use of NADI in the drought forecasting model forecasts the overall dry condition within the catchment beyond the traditional forecast of rainfall based DIs as the forecast of drought condition. In this study, the most successfully used Artificial Neural Network (ANN) modeling technique was used to develop the drought forecasting model to forecast the NADI values.

Two types of ANNs namely: Recursive Multi-Step Neural Network (RMSNN) and Direct Multi-Step Neural Network (DMSNN) were used to forecast the NADI values for the Yarra River catchment in Victoria (Australia). The present and several months of past NADI values were tested as inputs in the RMSNN model, whereas the present and several months of past NADI values along with the present and several months of past hydro-meteorological variables that were used in NADI calculations were tested as inputs in the DMSNN model. It was found that the best forecasts were obtained from both RMSNN and DMSNN models, when the combination of the present and up to past 2 months of NADI values were used as model inputs. Moreover, the RMSNN model was required only two neurons in one hidden layer to get the best forecasts, whereas only three neurons were sufficient for the DMSNN model. The best developed RMSNN and DMSNN drought forecasting models were capable of forecasting drought conditions reasonably well up to 6 months ahead forecasts; these forecasts were statistically significant at 1% significant level. Moreover, it was found that both models show the same performance for forecasting 1-month lead time. However, the RMSNN model gives slightly better forecasts than the DMSNN model

for lead times of 2 to 3 months and the DMSNN model gives slightly better forecasts than the RMSNN model for forecast lead times of 4 to 6 months. Poor forecasts were observed beyond the forecast lead time of 6 months.

7. SUMMARY AND CONCLUSIONS,