Results at Default of the Contract

The out-of-time performance measurements at default of the contract are shown in Table 3.9. In contrast to the execution of the contract the results differ signifi- cantly between both companies. While for company A both methods again yield more accurate LGD estimations by incorporating the macroeconomic factors, the opposite holds for company B.

A closer study of the outcomes for company A shows that particularly the LGD estimations performed by the nonlinear regression spline model benefit from the inclusion of macroeconomic factors for all forecast periods. The predictions of the linear regression improve consistently less. In particular, for the linear regression the benefit of using macroeconomic factors decreases towards the end of the observation period. This is a possible indication that a linear estimation model for the LGD can hardly consider the serious changes of the macroeconomic factors that occurred in the wake of the financial crisis (see Section 3.2.4). In this situation it seems to be more reasonable to extrapolate the training data with a nonlinear model.

Comparing the out-of-time results for company A at execution and default of the contract, we recognize that the improvement of the estimation accuracy by in-

Company A Company B

Linear Regression Regression Spline Linear Regression Regression Spline

Year FIRM FIRM+MACRO FIRM FIRM+MACRO Year FIRM FIRM+MACRO FIRM FIRM+MACRO

MAE 2005 0.2958 0.2941 0.2965 0.2893 2005 0.3064 0.3120 0.3018 0.2985 RMSE 0.3731 0.3708 0.3701 0.3648 0.3680 0.3738 0.3661 0.3674 REC Area 0.7058 0.7076 0.7047 0.7119 0.6711 0.6715 0.6973 0.7021 MAE 2006 0.2634 0.2603 0.2667 0.2617 2006 0.3179 0.3212 0.3167 0.3209 RMSE 0.3291 0.3294 0.3328 0.3328 0.3770 0.3827 0.3778 0.3784 REC Area 0.7378 0.7409 0.7179 0.7396 0.6826 0.6794 0.6846 0.6802 MAE 2007-2009 0.3321 0.3319 0.3334 0.3229 2007 0.3634 0.3641 0.3646 0.3647 RMSE 0.4105 0.4112 0.4285 0.4208 0.4352 0.4392 0.4573 0.4567 REC Area 0.6702 0.6706 0.6733 0.6836 0.6404 0.6395 0.6418 0.6414 MAE 2008 0.3410 0.3444 0.3411 0.3558 RMSE 0.4073 0.4109 0.4085 0.4258 REC Area 0.6595 0.6561 0.6595 0.6453 MAE 2009 0.3213 0.3241 0.3184 0.3649 RMSE 0.3838 0.3893 0.3825 0.4396 REC Area 0.6792 0.6764 0.6820 0.6365

Table 3.9: Out-of-time performance measurements at default of the contract by company and forecast period. The estimates were carried out each with the nonlinear regression spline model and the linear regression model. FIRM represents that only idiosyncratic factors were used as explanatory variables, FIRM+MACRO implies that additionally macroeconomic factors were included, in each case the better result is underlined. REC Area is defined as the area under the regression error characteristic curve, MAE is the mean absolute error, and RMSE is the root mean squared error. For the REC Area higher, for MAE and RMSE lower outcomes are preferable.

corporating macroeconomic factors is significantly more pronounced at execution of the contract. This observation is in line with the in-sample results. Moreover, it is noticeable that at default of the contract the improvement of the estimation accuracy by the inclusion of macroeconomic factors is mainly reflected by the out- comes of the MAE and the REC Area and only partially by the outcomes of the RMSE. One explanation for this observation is provided by the respective REC curves, displayed in Figure 3.4. In particular the shape of the REC curves for the forecast periods 2006 and 2007-2009, presented in Figure 3.4b and Figure 3.4c, shows that the proportion of observations with minor deviations between the real- ized and the predicted LGD is higher if macroeconomic factors are incorporated. This increasing proportion of observations with minor deviations between the re- alized and the predicted LGD is characteristic if the LGD estimation benefits from the use of macroeconomic factors. This feature is also observed at execution of the contract for both companies. However, in contrast to the situation at execution of the contract, the REC curves of the models that use macroeconomic factors undercut the respective REC curves of the models without macroeconomic factors

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Error tolerance Accuracy

Linear Regression: FIRM Linear Regression: FIRM+MACRO Regression Spline: FIRM Regression Spline: FIRM+MACRO

(a) Forecast period: 2005

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Error tolerance Accuracy

Linear Regression: FIRM Linear Regression: FIRM+MACRO Regression Spline: FIRM Regression Spline: FIRM+MACRO

(b) Forecast period: 2006 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Error tolerance Accuracy

Linear Regression: FIRM Linear Regression: FIRM+MACRO Regression Spline: FIRM Regression Spline: FIRM+MACRO

(c) Forecast period: 2007-2009

Figure 3.4: Regression error characteristic (REC) curves of the out-of-time loss given default (LGD) estimations at default of the contract for company A by forecast period. The estimates were carried out each with the nonlinear regression spline model and the linear regression model. FIRM represents that only idiosyncratic factors were used as explanatory variables, FIRM+MACRO implies that additionally macroeconomic factors were included.

with increasing error tolerance at default of the contract for company A. This im- plies a higher proportion of observations with substantial deviations between the realized and the predicted LGD for the models that incorporate macroeconomic factors. For this reason, the difference between the models with and without macroeconomic factors is less in terms of the RMSE compared to the MAE, be-

cause the RMSE penalizes in particular large deviations between the realized and the predicted LGD.

Analogously to the out-of-time results at execution of the contract, for company A at default of the contract the nonlinear regression spline model primary achieves a higher estimation accuracy than the linear regression if macroeconomic factors are considered. This might be another indication that the relationship between the LGD and the macroeconomic explanatory variables is nonlinear for company A. In-sample, however, the nonlinear model consistently performs better than the respective linear model.

For company B the out-of-time outcomes at default of the contract show a com- pletely different picture than for company A. In contrast to the out-of-time esti- mation at execution of the contract, the consideration of macroeconomic factors does not lead to more accurate estimations of the LGD at default. In particular, it should be highlighted that this finding holds for all forecast periods. Consequently, the different results for both companies are consistent and are not the outcome of, e. g., biased data at the end of the observation period. The linear regression yields consistently better predictions without incorporating the macroeconomic factors. For the nonlinear regression spline model the inclusion of the macroe- conomic factors worsens the LGD estimation particularly towards the end of the observation period. Even a simple historical average provides more accurate esti- mations than the regression spline model which considers macroeconomic factors for these forecast periods.

At first glance, for company B the out-of-time results seem to contradict the in-sample results, because the latter also indicate an improvement of the estima- tion accuracy by considering macroeconomic factors at default of the contract. An explanation is provided by the analysis of the effect of the macroeconomic factors on the LGD, discussed in Section 3.4.2. While a clear link between the economic situation and the LGD can be observed for both companies at execution

of the contract, this applies only for company A at default of the contract. In contrast, for company B the influence of the macroeconomic factors on the LGD is extremely volatile at default of the contract and does not follow a clear structure. In particular, it is not possible to connect changes of the LGD directly to the impact of the financial crisis. In other words, although in-sample a higher estima- tion accuracy is achieved by the inclusion of the macroeconomic factors, a direct relationship between the LGD and the economic situation cannot be established. Consequently, it can be assumed that the relationship between the LGD and the macroeconomic explanatory variables is not identical over the entire observation period. This implies that extrapolating the training data is hardly possible, es- pecially towards the end of the observation period when serious changes of the macroeconomic factors occurred in the wake of the financial crisis. In this regard, it should also be borne in mind that the serious changes of the macroeconomic factors are only reflected by moderate changes of the level of the LGD (see Ta- ble 3.2). The negative influence of the inclusion of the macroeconomic factors on the estimation accuracy especially towards the end of the observation period is more pronounced for the nonlinear regression spline model, because this model achieves a higher adaption to the training data and it can be assumed that it attaches more importance to the serious changes of the macroeconomic factors.