Subsample 2 : 2001-2006 training subsample and 2007-2009 testing subsample

In this section, using the same 17 financial and nonfinancial variables, the CART bank FSR group membership model is built using training subsample2 and is tested using testing

subsample2.

Table 5.9: Classification results for CART using training subsample2 Actual Group Membership No. of Cases Predicted Group Membership

High FSR Low FSR High FSR 105 102 3 97.1% 2.9% Low FSR 130 4 126 3.1% 96.9%

Source: Developed by the researcher (based on the statistical output).

Table 5.9 shows the classification results for training subsample2 using the CART technique.

Table 5.9 reveals that the ACC rate is 97% ((102+126)/235), which is equal to the ACC rate associate with CHAID using the same subsample. Unlike the CHAID model, the predictive accuracy of the CART model for high FSRs (97.1%) is somewhat higher than the predictive accuracy for low FSRs (96.9%). In line with this, using training subsample2, the EMC

associated with the CART model (0.217) is more costly than the EMC associated with CHAID model (0.072). This is mainly a result of the fact that the Type II error rate for the CART model (3.1%) is four times greater than the type II error rate for the CHAID model (0.8%).

Table 5.10: Classification results for CART using testing subsample2 Actual Group Membership No. of Cases Predicted Group Membership

High FSR Low FSR High FSR 67 55 12 82.1% 17.9% Low FSR 49 8 41 16.3% 83.7%

177

From results revealed in Table 5.10, the ACC rate associated with the CART model using the testing subsample2 is 82.8% ((55+41)/116). This ACC rate is lower than the ACC rate

associated with the CHAID model (88.8%) for the same sample. In addition, it is significantly lower than the ACC rate associated with the CART model (92.24%) using the testing subsample1.

As shown in Figure 5.2, the difference between CART models using testing subsample1 and

testing subsample2 can be observed clearly in the graphical analysis. This significant decline

in the ACC rate is mainly a result of the lower predictive power of the CART model (82.1% for high FSRs and 83.7% for low FSRs) using testing subsample2. Accordingly, the EMC

associated with the CART model using testing subsample2 (0.931) is more expensive than the

EMC associated with the CHAID model using the same subsample (0.776). Figure 5.2: Gains charts for testing subsample1 and testing subsample2 using CART

178 5.4 Multilayer perceptron neural networks

In this section, MLP models are developed because of the categorical nature of the dependent variable. MLP bank FSR group membership models are designed using the same 17 financial and nonfinancial variables listed earlier for the entire data set, subsample1 (training and

testing) and subsample2 (training and testing). 5.4.1 Entire data set

The PASW® Modeler 14 was used in this thesis to design the MLP bank FSR group membership model using the entire data set and the 17 independent variables.

Table 5.11: Classification results for MLP neural network using entire data set

Actual Group Membership No. of Cases Predicted Group Membership

High FSR Low FSR High FSR 172 158 14 91.9% 8.1% Low FSR 179 7 172 3.9% 96.1%

Source: Developed by the researcher (based on the statistical output).

Table 5.11 presents the classification results for the MLP bank FSR group membership model for the entire data set. Table 5.11 indicates that the ACC rate is 94.02% ((158+172)/351), which is the lowest ACC rate across all other machine-leaning techniques employed in this thesis to predict banks’ FSR group memberships (i.e., CHAID and CART). Moreover, of the 172 high FSRs, 158 (91.9%) were predicted to be high FSRs. The predictive accuracy for low FSRs is exceptional at 96.1% (172/179). The EMC associated with the MLP model is more costly (0.279) than the EMCs associated with other machine-learning techniques, namely, CHAID (0.256) and CART (0.265). This is supported by the fact that the Type I error rate is significantly higher for the MLP model than for other machine-learning techniques.

179

5.4.2 Subsample1: 67% training subsample and 33% testing subsample

In line with the same method used in the entire data set MLP section, and using only training subsample1, all of the 17 independent variables were used to build the MLP bank FSR group

membership model. Testing subsample1 was used to test the predictive power of the fitted

model.

Table 5.12: Classification results for MLP neural network using training subsample1 Actual Group Membership No. of Cases Predicted Group Membership

High FSR Low FSR High FSR 114 110 4 96.4% 3.6% Low FSR 121 10 111 8.3% 91.7%

Source: Developed by the researcher (based on the statistical output).

As seen in Table 5.12, the MLP model predicts high FSRs (96.4%) better than it does lower FSR banks (91.7%) using training subsample1. Consequently, the ACC rate for training

subsample1, for which data are used to fit a model, is 94.0% ((110+111)/235), which is lower

than the ACC rates associated with CHAID (97.4%) and CART (97.02%) using the same subsample1.

Accordingly, the EMC associated with the MLP model using training subsample1 is 0.528. It

is the most costly EMC of those associated with the two other machine-learning techniques using same subsamples1. Apparently, the high Type II error rate (8.3%) associated with the

MLP model enlarges the overall EMC of the model.

Table 5.13: Classification results for MLP neural network using testing subsample1 Actual Group Membership No. of Cases Predicted Group Membership

High FSR Low FSR High FSR 58 53 5 91.4% 8.6% Low FSR 58 11 47 19% 81%

180

Table 5.13 shows the classification results for testing subsample1 using the MLP neural

network model. The classification matrix in Table 5.13 indicates that the ACC rate for testing subsample1, for which the data are used only to test the predictive power of the model, is

86.21% ((53+47)/116). The MLP neural network model predicts high FSRs (91.4%) better than it does low FSRs (81%). The EMC associated with the MLP model using testing subsample1 (1.181) is much more expensive than the EMCs associated with CHAID (0.5) and

CART (0.362) using same testing subsample1. Apparently, the high Type II error rate (19%)

associated with the MLP model enlarges its EMC.

5.4.3 Subsample2: 2001-2006 training subsample and 2007-2009 testing subsample

The same validation technique used for the entire data set and subsample1 is repeated for

subsample2 using the original 17 independent variables.

Table 5.14: Classification results for MLP neural network using training subsample2 Actual Group Membership No. of Cases Predicted Group Membership

High FSR Low FSR High FSR 105 100 5 95.2% 4.8% Low FSR 130 7 123 5.4% 94.6%

Source: Developed by the researcher (based on the statistical output).

Table 5.14 summarises the results for MLP bank FSR group membership model using training subsample2.The ACC rate using training subsample2 is 94.9% ((100+123)/235),

which is lower than the ACC rates using same training subsamples2 for both CHAID (97.4%)

and CART (97.02%). The MLP model, using training subsample2, predicts high FSRs

(95.2%) better than it does low FSRs (94.6%). The EMC for training subsample2 is 0.379,

which is significantly more expensive than the EMCs associated with both CHAID (0.072) and CART (0.217) using same training subsamples2.

181

Table 5.15: Classification results for MLP neural network using testing subsample2 Actual Group Membership No. of Cases Predicted Group Membership

High FSR Low FSR High FSR 67 54 13 80.6% 19.4% Low FSR 49 9 40 18.4% 81.6%

Source: Developed by the researcher (based on the statistical output)

As shown in Table 5.15, the MLP bank FSR group membership model, using testing subsample2, predicts low FSRs (81.6%) slightly better than it does high FSRs (80.6%), which

is different from results reported previously for testing subsample1. The ACC rate using

testing subsample2 is 81% ((54+40)/116), which is lower than the ACC rate for testing

subsample1 (86.21%). This is supported by the fact that the predictive capability of the MLP

model for high FSRs using testing subsample2 (80.6%) declined significantly in contrast to

that using testing subsample1 (91.4%).

As illustrated in Figure 5.3, the difference in the ACC rates between both testing subsamples can be observed in the gains charts for testing subsample1 and testing subsample2. Finally, the

EMC for testing subsample2 is 1.04 that is relatively less costly to that for testing subsample1

182

Figure 5.3: Gains charts for testing subsample1 and testing subsample2 using MLP neural

network