test error for the 5-year data set is higher than that of the 10-year data set. Therefore, more information help to get a better result for the data with 10 grades.
4.3
Overlapping Data
Our sample size is really small, so we plan to use as much information as possible. We use all 10-years data, by starting with the first 5-year data (2008- 2012) then the next 5-year data (2009-2013) and keep going until the last 5- year (2013-2017). For the data within each 5-year period we apply the tech- nique described in Chapter 3 to get the first 20 PCs. The final data set consists of 120 PCs and the 2017 credit rating. For the data with 10 categories we use the Mean and Standard Deviation scaling method. The results are shown in Table 4.17. The training error average per company is 1.242, which means that the average distance between the true and the predicted credit ratings is 1, i.e., it a one-grade difference. The test error average per company is 2.58, which means that the distance between the true and the predicted credit rat- ings is about 3, i.e., it is a three-grade difference. The average training error Total 0 (24.8) and training error Total 1 (5.4) indicate that for the training data there are close to 25 companies for which the predicted result is the same as the true credit rating and there are 5 companies for which the predicted credit rating has a one-grade difference with the true credit rating. The average test error Total 0 (1) and test error Total 1 (2.4) mean for the test data there is 1 company for which the predicted credit rating is the same as the true credit rating and there are 2 companies for which the predicted credit rating has a one-grade difference with the true credit rating.
TABLE4.17: Overlapping data with 10 Grades
Training Error Test Error
Total Total 0 Total 1 Total Total 0 Total 1
Test 1 56 25 6 36 0 2 Test 2 76 18 6 26 1 1 Test 3 40 27 5 29 1 3 Test 4 47 25 7 15 1 3 Test 5 42 29 3 23 2 3 Average 52.2 24.8 5.4 25.8 1 2.4
Ave per Company 1.242 0.59 0.13 2.58 0.1 0.24
Comparing the results from the overlapping data with the results of All Vari- ables, the training error improves slightly but the test error does not improve.
With the data in 4 categories, we use the Max and Min scaling method. The results are shown in Table4.18. The average per company of training error is 1.152, which means the credit rating can be accurately predicted. The av- erage per company of test error is 1.36, which means the credit rating can be accurately predicted. The training error Total 0 (24.6) and training Total 1 (2) indicate that for about 25 companies in the training data set the predicted results are the same as the true credit ratings and for two companies the pre- dicted credit rating is one grade away from the true credit rating. The test error Total 0 (6.4) and test error Total 1 (0.8) indicate that for 6 companies in the test data set the predicted group credit ratings are the same as the true group credit ratings and for close to one company the predicted credit rating is one grade away from the true credit rating.
Both 10 categories’ and 4 categories’ prediction accuracy from the overlap- ping data do not improve comparing with the AV method.
4.4. 104 Companies 57
TABLE4.18: Overlapping data with 4 Grades
Training Error Test Error
Total Total 0 Total 1 Total Total 0 Total 1
Test 1 46 31 0 19 7 0 Test 2 67 23 2 19 5 1 Test 3 26 34 0 14 6 1 Test 4 50 26 4 2 9 1 Test 5 53 9 4 14 5 1 Average 48.4 24.6 2 13.6 6.4 0.8
Ave per Company 1.152 0.59 0.05 1.36 0.64 0.08
4.4
104 Companies
From Section4.3, when incorporating more information, the prediction ac- curacy does not improve significantly, we hypothesized whether this may be caused by the small sample size. We want to increase the sample size in order to improve prediction accuracy. For each company, we use the first 5 years (2008-2012) with the 2012 credit rating as one sample, and then the last 5 years (2013-2017) with the 2017 credit rating as another sample. We do the same thing for all 52 companies, so we can double the sample size to have 104 companies. We can do this because most of the missing values occur in the first 5 years. After cleaning the original data set, the variables in first 5 years and last 5 years are not entirely same. We can treat them as two differ- ent samples. The results for 104 companies with 10 categories are shown in Table4.19.
The training error average per company is 1.532, which means that the av- erage distance between the true and the predicted credit ratings is about 2, i.e., it is a two-grade difference. The test error average per company is 2.44, which means that the distance between the true and the predicted credit rat- ings is 2, i.e., it is a two-grade difference. The average training error Total 0 (43.8) and training error Total 1 (18) indicate that for the training data there
are close to 44 companies for which the predicted result is the same as the true credit rating and there are 18 companies for which the predicted credit rating has a one-grade difference with the true credit rating. The average test error Total 0 (1) and test error Total 1 (2.8) indicate that for the test data there is one company for which the predicted credit rating is the same as the true credit rating and there are close to 3 companies for which the predicted credit rating is a one-grade difference with the true credit rating.
TABLE4.19: 104 companies with 10 Grades
Training Error Test Error
Total Total 0 Total 1 Total Total 0 Total 1
Test 1 136 47 16 11 2 5 Test 2 169 38 21 28 1 1 Test 3 112 47 20 36 0 2 Test 4 134 49 15 23 2 4 Test 5 169 38 18 24 0 2 Average 144 43.8 18 24.4 1 2.8
Ave per Company 1.532 0.47 0.19 2.44 0.1 0.28
We also performed the same test for 4 grades. We used the Max and Min scaling method. The results are shown in Table4.20. The training error av- erage per company is 1.08, which means the credit rating can be accurately predicted. The test error average per company is 1.2, which means the credit rating can be accurately predicted. The average training error Total 0 (69.6) and training error Total 1 (3.4) mean that for the training data there are about 70 companies for which the predicted result is the same as the true credit rating and there are 3 companies for which the predicted credit rating has one-grade difference with the true credit rating. The average test error Total 0 (7.2) and test error Total 1 (1) indicate that for test data there are 7 compa- nies for which the predicted credit rating is the same as the true credit rating and there is one company for which the predicted credit rating has one-grade away from the true credit rating.