4.0 Introduction
This chapter analyses the results from the AIE model to answer the research questions listed in section 3.8. Chapter 5 discusses the findings of this chapter within the context of the literature. Hence this chapter refrains from discussing the literature to provide an uncluttered presentation of the results.
4.1 Research Questions
This section consists of seven subsections that correspond to a research question of the same number. Table 4–1 shows the results from running the AIE model. Portions of Table 4–1 are analysed under each of the research questions.
Table 4–1 shows the model variance of the various AIE models and their benchmarks. The table shows three phases, the calibration, prediction and evaluation. The calibration phase for the
‘runtime-weighted model-averaging’ and ‘optimal-calibration model-averaging’ require some form of optimisation to produce an optimal ‘runtime-weighted constant’ c value or ‘number of runs’, respectively. These optimal values are used in the prediction phase. The evaluation phase tests the efficacy of the model-averaging technique. The evaluation phase compares the prediction model variance, using the optimal values found in the calibration phase with optimising the model within the prediction period.
The two major divisions of the AIE models are aggregated and disaggregated. All the models in the Table 4–1 use the short calibration period, except for the aggregated version of the AIE model labelled ‘long calibration’. This ‘long calibration’ period result is compared with the ‘short calibration’ period results to answer research question one.
The adaptive-expectations model forms a benchmark for the AIE model and also has both an aggregated and disaggregated version. The adaptive-expectations model is the AIE model with the interactive-expectations component set to zero. REH provides another benchmark for the AIE model, where a single REH benchmark provides for both aggregated and disaggregated versions. The disaggregated AIE model uses a ‘link-intensity matrix’ that represents the intensity of interactions among firms of differing divisions. There are three types of ‘link-intensity matrix’, based on an ‘input-output table’, its transpose, and a ‘matrix of ones’ that represents the default.
Table 4–1 The results of the AIE simulation for the aggregated and disaggregated versions
Calibration Prediction Evaluation Version of AIE
Model
Model-averaging
Method SSE/T Runs or c SSE/T SSE/T Runs or c Optimal-calibration 37 6 runs 248 190 1 run Runtime-weighted 37 0 229 208 c = 2.04 Bayes-factor 37 229 Equal-weighted 41 252 Long Calib rat ion Single run 42 190
Optimal-calibration 17 9 runs 67 49 17 runs Runtime-weighted 18 c = 0 67 66 c = 2.01 Bayes-factor 18 67 Equal-weighted 21 62 Short Cali brat ion Single run 19 78 Optimal-calibration 30 8 runs 114 87 29 Runtime-weighted 32 c = 6.4 99 98 c = 0 Bayes-factor 32 98 Equal-weighted 41 102 Aggre gated Adaptive - exp e ctatio ns Single run 32 93
Optimal-calibration 16 2 runs 59 52 5 runs Runtime-weighted 18 c = 5.2 68 65 c = 122 Bayes-factor 18 70 Equal-weighted 22 62 Input-O utput Single run 21 107
Optimal-calibration 17 5 runs 80 65 46 runs
Runtime-weighted 18 c = 0 79 60 18 Bayes-factor 18 79 Equal-weighted 22 68 Tran sp ose Single run 20 83
Optimal-calibration 17 7 runs 54 36 2 runs Runtime-weighted 17 c = 7.02 72 62 c = 0 Bayes-factor 18 62 Equal-weighted 21 65 Link-inten sity Matrix Matrix of ones Single run 19 96
Optimal-calibration 27 8 runs 99 96 2 runs Runtime-weighted 28 c = 0 103 103 c = 0 Bayes-factor 28 103 Equal-weighted 28 101 Disagg reg a te d Adaptive - exp e ctatio ns Single run 30 126 Short period 201 93 REH Long period 326 93
4.1.1 Do the profit expectations undergo a significant structural change or phase shift during the quarter ending March 2000?
This question compares the predictive performance of the aggregated AIE model calibrated over a short and a long period. Table 4–1 shows that the model variances for prediction, calibrated over the short period, is about a third of that calibrated over the long period. This shows that profit expectations have undergone a significant structural change or phase shift during the quarter ending March 2000. The cause of change in profit expectations during the March 200 quarter cannot be ascertain from the results, whether it is due to structural change or a phase shift. Because calibrating over the short period provides much better predictive performance, the remaining questions only address models calibrated over the short period, unless otherwise noted. Sections 5.6.5 and 5.6.7, in further research, further discuss the structural change and phase shift issue, in particular, calibrating AIE in the pre March 2000 period to compare with a post March 2000 calibration.
4.1.2 Does ‘optimal-calibration model-averaging’ improve predictive performance over ‘equal-
weighted model-averaging’ and ‘Bayes-factor model-averaging’ for the AIE model?
This question compares the model variance of the predictions of the ‘optimal-calibration model- averaging’ against two benchmarks, ‘Bayes-factor model-averaging’ and ‘equal-weighted model- averaging’. Table 4–2 provides the information from Table 4–1 in a form that more readily addresses this question.
Table 4–2 Comparing the model variance of the prediction of model-averaging techniques Version of AIE model
Aggregated Disaggregated Link-intensity
Model Variance (SSE/T)
Long Calibration Short Calib
ration Adaptive -e x p e ctation s
Input-Output Transpose Matrix of O
n es Adaptive -e x p e ctation s Optimal-calibration 248 67 114 59 80 54 99 Runtime-weighted 229 67 99 68 79 72 103 Bayes-factor 229 67 98 70 79 62 103 Model- averaging Techniques Equal-weighted 252 62 102 62 68 65 101
Table 4–2 shows that the model variance of the ‘optimal-calibration model-averaging’ is less than
the ‘Bayes-factor model-averaging’ for the disaggregated AIE versions with a ‘matrix of ones’ and
‘input-output table’ and to a lesser extent the disaggregated adaptive-expectations model.
Additionally, the model variance of the ‘optimal-calibration model-averaging’ is less than the ‘
equal-weighted model-averaging’for the disaggregated AIE versions with a ‘matrix of ones’ and ‘
input-output table’ and to a lesser extent the following two adaptive-expectations versions, aggregated with a long calibration and disaggregated with short calibration.
Conclusion
‘Optimal-calibration’ provides better predictive performance than ‘equal-weighted’ or ‘Bayes- factor model-averaging’ for the disaggregated AIE versions with a ‘matrix of ones’ and ‘input- output table’. Noting that, these two AIE versions provide the best predictions, which indicates that, the ‘optimal-calibration’ technique is more usefully applied to models that have a history of good predictive performance, which require recalibrating, rather than use the technique on unproven or newly developed models.
4.1.3 Does ‘runtime-weighted model-averaging’ improve predictive performance over ‘equal-
weighted model-averaging’ and ‘Bayes-factor model-averaging’ for the AIE model?
This question compares the predictive performance of the ‘runtime-weighted model-averaging’
against two benchmarks, ‘Bayes-factor model-averaging’ and ‘equal-weighted model-averaging’. Note that the ‘Bayes-factor model-averaging’ is the ‘runtime-weighted model-averaging’ less the runtime-weighted component. Table 4–2 shows that variance of the ‘runtime-weighted model- averaging’ is less than the ‘Bayes-factor model-averaging’ for the AIE version with an ‘input- output table’ only and by a margin of 2 only. Additionally, the model variance of the ‘runtime- weighted model-averaging’ is less than the ‘equal-weighted model-averaging’ for the long calibration AIE version and aggregated adaptive-expectations model only. Noting that, these two models are the worst and second or third worst predicting models.
Conclusion
Except for the AIE version with an ‘input-output table’, the ‘runtime-weighted model-averaging’
fails to improve the prediction performance over the ‘Bayes-factor model-averaging’ and makes the prediction performance worse for the AIE version with a ‘matrix of ones’ and the aggregated adaptive-expectations model.
4.1.4 Does the interactive-expectations network improve the predictive power of the AIE model?
This question benchmarks the AIE model against the adaptive-expectations model that is the AIE model less the interactive network. Table 4–1 shows that the model variance for all AIE versions for both calibration and prediction phases is less than that for all adaptive-expectations models. Section 5.6.11, in further research, discusses using the random seed function in NetLogo to improve the adaptive-expectations model as benchmark.
Note that a slightly lower model variance for the aggregated adaptive-expectations model was found by setting the network topology to values other than L = 1 and ρ = 0, as stipulated in the methodology. Since I = 0, these alternate network topology settings only indirectly affect the model variance calculations because the random functions in the model are affected by using different parameters. Table 4–1 shows these lower model variances to provide a tougher benchmark for the AIE model.
Conclusion
The interactive network improves predictive performance.
4.1.5 Does the subjective approach of the aggregated AIE model improve predictive performance over the objective approach of REH?
This question benchmarks the AIE against REH. Table 4–1 shows that the model variances for both calibration and prediction of all AIE versions are lower than the model variances of REH, except for the prediction of the aggregated AIE model calibrated over the long period, which was rejected in research question one. An interesting point is that the predictive performance of REH is better than the adaptive-expectations models.
Conclusion
The AIE model provides better predictive performance than REH that in turn provides better predictive performance than the adaptive-expectations model.
4.1.6 Does introducing an input-output table link-intensity matrix improve the predictive performance of the disaggregated AIE model?
This question compares the predictive performance of introducing the ‘link-intensity matrix’ into the disaggregated AIE model, based on an ‘input-output table’ and its transpose against the benchmark a ‘matrix of ones’. Table 4–2 clearly shows that the predictive performance of the transpose ‘link-intensity matrix’ is less than that of the ‘matrix of ones’. However, it is difficult to
decide which ‘link-intensity matrix’ has the better predictive performance between the ‘input-output table’ and ‘matrix of ones’.
Conclusion
There is an ambiguous effect on prediction performance from introducing the ‘input-output table’. However, there is a clear reduction in predictive performance from introducing the transpose.
4.1.7 Does disaggregating the AIE model improve predictive performance?
This question compares the predictive performance of the aggregated and disaggregated AIE versions. Table 4–2 clearly shows that the disaggregate AIE version with a transposed ‘input- output table’ is unsuitable. However, Table 4–2 shows also the disaggregated AIE versions with an
‘input-output table’ and ‘matrix of ones’ provide better predictive performance than the aggregated AIE version for ‘optimal-calibration model-averaging’. ‘Runtime-weighted model-averaging’ is exclude form discussion because it has proven ineffective. However, the ‘Bayes-factor model- averaging’ and ‘equal-weighted model-averaging’ results are ambiguous, that the disaggregated AIE version combines data from both ‘input-output tables’ and the D&B (2008) expectations survey, which introduces more error into the disaggregated model and makes uncertain whether the ambiguous results are due to a poor model fit or compounding error.
Conclusion
The results are ambiguous but ‘optimal-calibration’ favours the predictive power of the disaggregated version over the aggregated version, which suggests that alternative ways to model disaggregated AIE be tried. Section 5.6.3, in further research, discusses an alternative way to develop the ‘link-intensity matrix’.
4.2 Conclusion
This chapter analyses the results from running the AIE model to address the research questions without reference to the literature, where Chapter 5 relates the findings to the literature.