Since the cases have ratings in four levels A, B, C, D, which are given by the current rating system. The scoring model generated by the M5 model tree algorithm is taken as an example to show how the cases with different ratings are classified by the scoring models. We apply the finally derived model tree to all the available cases in the data set. Figure 5.4/1,Figure 5.4/2,Figure 5.4/3 are the distribution of the scores of A/B , C and D companies.
0,0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 score 10% 20% 30% 40% 50% Percent
Figure 5.4/1: The distribution of the scores of A/B companies
0,0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 score 10% 20% 30% 40% 50% Percent
Figure 5.4/2: The distribution of the scores of C companies
0,0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1,0 score 10% 20% 30% 40% 50% 60% 70% Percent
It is clearly shown in the figures that the most of A/B companies have scores lower than 0.3 and the D companies have scores higher than 0.9, while the scores of C companies are scattered in the range of the scores.
The attractive character of the scoring model is the continuous scores assigned to each case. The risk levels of each case are measured in a continuous scale. In practical application, several-classes can be constructed through dividing the scores into several segments. Credit decisions are made flexibly by changing the division values of scores. A flexible way of using the scores produced by the model is as follows:
With two threshold values T1 and T2, the cases can be divided into three classes: class 1 (score<T1), class 2 (T1<=score<T2), and class 3 (score>=T2). The following cross tables (See Table 5.4/1) show the distribution of cases in the ratings and score classes with different T1 and T2. 2 1 3 66,7% 33,3% 100,0% 46905 1155 269 48329 97,1% 2,4% ,6% 100,0% 1248 2181 7939 11368 11,0% 19,2% 69,8% 100,0% 1 5666 5667 ,0% 100,0% 100,0% 48156 3336 13875 65367 73,7% 5,1% 21,2% 100,0% Count % Count % Count % Count % Count % A B C D Ratings Total 1 2 3 Score classes (T1=0,3 T2=0,97) Total 2 1 2 66,7% 33,3% 100,0% 46905 1350 74 48329 97,1% 2,8% ,2% 100,0% 1248 3837 6283 11368 11,0% 33,8% 55,3% 100,0% 1 5666 5667 ,0% 100,0% 100,0% 48156 5187 12024 65367 73,7% 7,9% 18,4% 100,0% Count % Count % Count % Count % Count % A B C D Ratings Total 1 2 3 Score classes (T1=0,3 T2=0,985) Total
2 1 3 66,7% 33,3% 100,0% 46905 1413 11 48329 97,1% 2,9% ,0% 100,0% 1248 8154 1966 11368 11,0% 71,7% 17,3% 100,0% 1 5666 5667 ,0% 100,0% 100,0% 48156 9568 7643 65367 73,7% 14,6% 11,7% 100,0% Count % Count % Count % Count % Count % A B C D Rating Total 1 2 3 Score classes (T1=0,3 T2=0,99) Total
Table 5.4/1: Crosstabulation of ratings and score classes
It is shown that most of A/B companies are classified into the first class, and most of D companies are classified into the third class. That means the most of A/B companies and D companies can be identified by the scoring model.
The C companies, whose risk levels are difficult to be decided, are classified into three classes with different percentages. The C companies in class 3 have the similar patterns as D companies, while the C companies in class 1 have the similar patterns as A/B companies. The C companies that classified into class 2 are ambiguous. The credit decision with them should be made individually by the experts.
The threshold values T1, T2 can be decided flexibly by the users. If a larger T2 is used then more C companies are classified into class 2 and need to be individually reviewed by experts.
6 Summary and conclusion
This paper focused on the problem of the evaluation of classification models for credit scoring. The reason of the model evaluation is to find the optimal solution from various classification models generated in an iterated and complex model building process.
Classification accuracy is the basic criterion to measure the performance of a model. Classification error rates can be estimated from the test sample. In order to get an accurate estimation, the sample of test data has some requirements: independent from training data, representative for the underlying population, and large enough.
Because credit scoring model should be effectively applied to real world problems, other criterions for evaluating practical scoring models are also important under some circumstances: the speed, the transparency and the simplicity of the models.
Usually credit scoring models classify credit customers into two classes: 'good' and 'bad'. The two types of error rates are used to assess the performance of models. There are different evaluation methods to describe these two types of errors. The most simple and direct method is confusion matrix. ROC curve is the graphic representation of the tradeoff between two types of errors. If the cost function, which weighs the two kinds of errors, is available, the optimal threshold value on the ROC curve can be determined and the cost minimization model is obtained. Some practical ways of selecting threshold values are discussed.
With the help of a learning curve, incremental analysis can be carried out in order to get the proper model structures and train sample size. There are some basic phenomena in the incremental cases analysis and incremental complexity analysis.
An example illustrated the process of using five algorithms to build credit scoring models, then evaluating and comparing them. Decision tree method M5 has a higher classification accuracy than other methods. The second accurate model is produced by the multi-layer perceptron neural network method, but its outputs can not be understood intuitively.
Finally, the scores obtained from models are compared with the ratings given by the current system. A flexible way of using the scores produced by scoring models is explained.
Appendices
Appendix A. Explanations of some variables in the data set
The variables listed here can help to understand the explanations of the models in this paper. The information comes from the German credit information agency 'Die Vereine Creditreform'. To keep their original meanings the explanations listed here are in German. A.1 Variable 'ZAHLWEISE'
01 Zahlungserfahrungen liegen noch nicht vor
02 bisher keine ausreichenden Erfahrungen, Beanstandungen liegen nicht vor 11 Skontoausnutzung
12 pünktlich, meist Skonto
13 zumeist Skonto, teils Zielinanspruchnahme bis zu 30 Tagen 21 innerhalb vereinbarter Ziele
22 innerhalb vereinbarter Ziele von 30 bis 60 Tagen 23 innerhalb vereinbarter Ziele von 60 bis 90 Tagen 29 soweit bekannt, pünktlich
31 ohne Beanstandungen bei gelegentlichen Zielüberschreitungen 32 meist innerhalb vereinbarter Ziele, teils auch länger
33 bisher ohne Beanstandungen, in letzter Zeit gelegentlich Zahlungserinnerungen 39 ohne Beanstandungen
41 Zielüberschreitung bis zu 30 Tagen
42 teils innerhalb vereinbarter Ziele, teils mit Zielüberschreitung
43 langsam mit Zielüberschreitungen, mehrfach Zahlungserinnerungen 44 langsam mit Zielüberschreitungen, Wechselprolongationen
51 erhebliche Zielüberschreitungen bis zu 3 Monaten 52 erhebliche Zielüberschreitungen von mehr als 3 Monaten 53 schleppend, mehrfach Mahnung, auch Scheckrückgaben
54 wird beanstandet, termingebundene Verpflichtungen wurden nicht eingelöst 55 langsam und schleppend, Creditreform-Inkasso-Dienst wurde eingeschaltet
61 Haftanordnung zur Abgabe der eidesstattlichen Versicherung
62 Abgabe der eidesstattlichen Versicherung über die Vermögensverhältnisse 63 Antrag auf Eröffnung des gerichtlichen Vergleichsverfahrens
64 Eröffnung des gerichtlichen Vergleichsverfahrens 65 Antrag auf Eröffnung des Konkursverfahrens 66 Eröffnung des Konkursverfahrens
67 Konkursverfahren mangels Masse abgelehnt 68 Eröffnung des Vorverfahrens
A.2 Variable 'AUFTRAGE'
10 sehr gute Auftragslage 1
11 sehr guter Geschäftsgang 20 gute Auftragslage 2 21 guter Geschäftsgang 30 zufriedenstellende Auftragslage 3 31 zufriedenstellender Geschäftsgang 40 rückläufige Auftragslage 4 41 rückläufiger Geschäftsgang 50 mäßige Auftragslage 5 51 mäßiger Geschäftsgang 60 schlechte Auftragslage 6 61 schlechter Geschäftsgang
A.3 Variable 'ENTWICKLUNG'
0 01 Geschäftsentwicklung bleibt abzuwarten 10 expansive Unternehmensentwicklung 1 11 expansive Geschäftsentwicklung 20 positive Unternehmensentwicklung 2 21 positive Geschäftsentwicklung 30 konstante Unternehmensentwicklung 3 31 konstante Geschäftsentwicklung
40 stagnierende Unternehmensentwicklung 4 41 stagnierende Geschäftsentwicklung 50 rückläufige Unternehmensentwicklung 5 51 rückläufige Geschäftsentwicklung
60 stark rückläufige Unternehmensentwicklung 6
61 stark rückläufige Geschäftsentwicklung
A.4 Variable 'BONITAETINDEX'
'BONITAETINDEX' is the Creditreform credit-standing index. It is given in the form of a three- digit figure (from 100 to 600). The first digit corresponds to a ranking aligned to the German system of school grades (where 1 is very good and 6 is fail); the second and third digits permit further differentiation (more information see CreditR02).
Appendix B. Brief summary of software and toolsets used in this study
Weka (Decision tree M5, Neural networks and k-NN): Weka is a software workbench which integrates many different Machine learning methods as well as some useful preprocessing methods (more information see WiFr00).
SPSS (Logistic regression and Linear discriminant analysis): SPSS is an integrated statistic tool. It provides statistical classification algorithms as well as tools for descriptive analysis, sampling, and graphics.
Appendix C. The derived model tree
Tree structure: ZAHLWEISE <= 29.5 : | BONITAETINDEX <= 224 : | | ZAHLWEISE <= 12.5 : | | | GRUNDJAHR <= 11.5 : | | | | GRUNDJAHR <= 4.5 : | | | | | GRUNDJAHR <= 2.5 : LM1 | | | | | GRUNDJAHR > 2.5 : LM2 | | | | GRUNDJAHR > 4.5 : LM3 | | | GRUNDJAHR > 11.5 : LM4 | | ZAHLWEISE > 12.5 : | | | UM23 <= -2.5 : | | | | UM12 <= -8.5 : LM5 | | | | UM12 > -8.5 : LM6 | | | UM23 > -2.5 : | | | | GRUNDJAHR <= 8.5 : | | | | | GRUNDJAHR <= 4.5 : | | | | | | GRUNDJAHR <= 2.5 : LM7 | | | | | | GRUNDJAHR > 2.5 : LM8 | | | | | GRUNDJAHR > 4.5 : LM9
| | | | GRUNDJAHR > 8.5 : LM10 | BONITAETINDEX > 224 : | | UM12 <= -2.5 : | | | UM12 <= -9.5 : | | | | UM23 <= -9.5 : LM11 | | | | UM23 > -9.5 : LM12 | | | UM12 > -9.5 : LM13 | | UM12 > -2.5 : | | | GRUNDJAHR <= 4.5 : | | | | GRUNDJAHR <= 2.5 : LM14 | | | | GRUNDJAHR > 2.5 : LM15 | | | GRUNDJAHR > 4.5 : | | | | BONITAETINDEX <= 256 : LM16 | | | | BONITAETINDEX > 256 : | | | | | BONITAETINDEX <= 278 : | | | | | | GRUNDJAHR <= 13.5 : LM17 | | | | | | GRUNDJAHR > 13.5 : | | | | | | | ENTWICKLUNG <= 3.5 : LM18 | | | | | | | ENTWICKLUNG > 3.5 : LM19 | | | | | BONITAETINDEX > 278 : | | | | | | ENTWICKLUNG <= 3.5 : LM20 | | | | | | ENTWICKLUNG > 3.5 : LM21 ZAHLWEISE > 29.5 : | ZAHLWEISE <= 41.5 : LM22 | ZAHLWEISE > 41.5 : LM23 Linear models at the leaves:
LM1: Score = 0.873 - 0.00158V1 - 0.0011V2 - 1.1e-4V3+ 1.11e-4V4 - 6.49e-6V6 + 1.51e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 0.0848V12 + 0.00128V13 - 0.00274V14 + 0.00166V15 + 7.59e-5V16.
LM2: Score = 0.458 - 0.00158V1 - 0.0011V2 - 1.1e-4V3+ 1.11e-4V4 - 6.49e-6V6 + 1.51e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 0.102V12 + 0.00128V13 - 0.00274V14 + 0.00166V15 + 7.59e-5V16.
LM3: Score = 0.0349 - 0.00158V1 - 0.0011V2 - 1.1e-4V3+ 1.11e-4V4 - 6.49e-6V6 + 1.51e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 0.00367V12 + 0.00128V13 - 0.00274V14 + 0.00166V15 + 7.59e-5V16. LM4: Score = 9.43e-4 - 5.5e-4V1 - 0.0011V2 - 1.1e-4V3+ 1.11e-4V4 - 6.49e-6V6 + 1.51e-4V8 + 2.75e-5V9 +
7.46e-5V10 + 2.32e-6V11 - 1.66e-5V12 + 0.00128V13 - 0.00274V14 + 0.00166V15 + 7.59e-5V16.
LM5: Score = 0.712 - 1.31e-4V1 - 5.83e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 4.22e-4V6 - 3.88e-4V7 + 1.05e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 4.82e-6V12 + 0.0174V13 - 0.0191V14 + 0.00288V15 + 7.59e-5V16.
LM6: Score = 0.0894 - 1.31e-4V1 - 5.83e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 1.33e-4V6 - 1.09e-4V7 + 1.05e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 4.82e-6V12 + 0.00694V13 - 0.00931V14 + 0.00288V15 + 7.59e-5V16.
LM7: Score = 0.927 - 1.31e-4V1 - 5.83e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 6.87e-6V6 + 1.05e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 0.051V12 + 0.00118V13 - 0.00237V14 + 0.00133V15 + 7.59e-5V16.
LM8: Score = 0.216 - 1.31e-4V1 - 5.83e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 6.87e-6V6 + 1.05e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 0.0341V12 + 0.00118V13 - 0.00237V14 + 0.00133V15 + 7.59e-5V16.
LM9: Score = 0.107 - 1.31e-4V1 - 5.83e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 6.87e-6V6 + 1.05e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 0.00237V12 + 0.00118V13 - 0.00237V14 + 0.00133V15 + 7.59e-5V16.
LM10: Score = 0.0788 - 1.31e-4V1 - 5.83e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 6.87e-6V6 + 1.05e-4V8 + 2.75e-5V9 + 7.46e-5V10 + 2.32e-6V11 - 1.09e-5V12 + 0.00118V13 - 0.00237V14 + 0.00133V15 + 7.59e-5V16.
LM11: Score = 0.903 - 1.89e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 3.24e-4V6 + 8.65e-6V7 + 7.81e-5V8 + 2.75e-5V9 + 0.00161V10 + 9.63e-5V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16. LM12: Score = 0.632 - 1.89e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 3.24e-4V6 + 8.65e-6V7 + 7.81e-5V8 +
2.75e-5V9 + 0.00161V10 + 9.63e-5V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16. LM13: Score = 0.194 - 1.89e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 7.49e-4V6 + 8.65e-6V7 + 7.81e-5V8 +
2.75e-5V9 + 0.00161V10 + 9.63e-5V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16. LM14: Score = 0.88 - 0.00201V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 4.34e-6V6 + 7.49e-6V7 + 7.81e-5V8 +
LM15: Score = 0.132 - 0.00201V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 4.34e-6V6 + 7.49e-6V7 + 7.81e-5V8 + 2.75e-5V9 + 0.00213V10 + 1.03e-4V11 - 0.0144V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16. LM16: Score = -0.0756 - 6.85e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 4.34e-6V6 + 3.66e-6V7 + 7.81e-5V8 +
2.75e-5V9 + 0.0492V10 + 8.23e-5V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16. LM17: Score = 0.0597 - 6.85e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 + 3.71e-9V5 - 4.34e-6V6 + 3.66e-6V7 +
7.81e-5V8 + 2.75e-5V9 + 0.00532V10 + 1.16e-4V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16.
LM18: Score = -0.0424 - 6.85e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 + 1.16e-8V5 - 4.34e-6V6 + 3.66e-6V7 + 7.81e-5V8 + 2.75e-5V9 + 0.0364V10 + 1.16e-4V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16.
LM19: Score = 0.218 - 6.85e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 + 1.16e-8V5 - 4.34e-6V6 + 3.66e-6V7 + 7.81e-5V8 + 2.75e-5V9 + 0.0756V10 + 1.16e-4V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16.
LM20: Score = 0.0747 - 6.85e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 4.34e-6V6 + 3.66e-6V7 + 7.81e-5V8 + 2.75e-5V9 + 0.0273V10 + 1.16e-4V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16. LM21: Score = 0.46 - 6.85e-4V1 - 5.03e-4V2 - 1.1e-4V3+ 1.11e-4V4 - 4.34e-6V6 + 3.66e-6V7 + 7.81e-5V8 +
2.75e-5V9 + 0.0517V10 + 1.16e-4V11 - 1.65e-6V12 + 4e-4V13 - 7.86e-4V14 + 5.3e-4V15 + 7.59e-5V16. LM22: Score = 0.984 - 8.01e-5V1 - 1.59e-4V3 + 1.59e-4V4 - 1.41e-4V7 + 1.16e-4V8 + 3.95e-5V9 - 5.81e-6V11
- 2.38e-6V12 + 1.09e-4V16.
LM23: Score = 0.997 - 8.01e-5V1 - 1.59e-4V3 + 1.59e-4V4 - 4.27e-7V7 + 1.15e-4V8 + 3.95e-5V9 - 5.81e-6V11 - 2.38e-6V12 + 1.09e-4V16. V1:JAHRNUM V2: RECHTSFORM=77,46,78,24,76,79,0 V3: RECHTSFORM=46,78,24,76,79,0 V4: RECHTSFORM=24,76,79,0 V5: STAMMKAP V6: UM12 V7: UM23 V8: ZAHLWEISE V9: AUFTRAGE V10: ENTWICKLUNG V11: BONITAETINDEX V12: GRUNDJAHR V13: BRANCH=O,A,I,H,F,M,Q V14: BRANCH=A,I,H,F,M,Q V15: BRANCH=I,H,F,M,Q V16: BRANCH=H,F,M,Q
Literature
AdHa00 Adams, N. M./Hand, D. J.: Improving the practice of classifier performance assessment. In: Neural Computation, 12(2000), 2, P. 305-311.
AdHa99 Adams, N. M./Hand, D. J.: Comparing classifiers when the misallocation costs are uncertain. In: Pattern Recognition, 32(1999), P. 1139-1147.
Baet95 Baetge, J./Hüls, D./Uthoff, C.: Früherkennung der Unternehmenskrise. In: Forschunfsjournal Westfälische Wilhelms-Universität Münster, 2(1995), P. 21-29. Baet98 Baetge, J.: Empirische Methoden zur Früherkennung von Unternehmenskrisen.
Nordrhein-Westfälische Akademie der Wissenschaften, Natur-, Ingenieur- und Wirtschaftswissenschaften, Vorträge, Nr. 432, Westdeutscher Verlag, 1998.
Boge67 Bogess, W. P.: Screen -Test your Credit Risks. In: Harvard Business Review, 45(1967), 6, P. 113-122.
CreditR02 Die Vereine Creditreform. Company information, The credit-standing index.
http://www.creditreform.com/eng/services/company_information/index.php, 21.05.02. Dink95 Dinkelmann, R.: Kriterien und Instrumente zur Risikofrüherkennung im
Firmenkundengeschäft der Banken. Verlag Paul Haupt Bern, Stuttgart, Wien, 1995. Feid92 Feidicker, M.: Kreditwürdigkeitsprüfung - Entwicklung eines Bonitätsindikators,
dargestellt am Beispiel von Kreditversicherungsunternehmen. IDW-Verlag GmbH, Düsseldorf, 1992.
Fran97 Frank, E./Wang, Y./Inglis, S./Holmes, G./Witten, I. H.: Using model trees for classification. In: Machine Learning 32(1998), 1, P. 63-76.
FrHo98 Fritz, S./Hosemann, D.: Behaviour Scoring for Deutsche Bank’s German Corporates. In: Gholamreza Nakhaeizadeh, et al. (eds.), Application of machine learning and data mining in finance, 10th European Conference on Machine Learning, Workshop notes; 24. April 1998, TU Chemnitz, P. 179-194.
GaTa97 Galindo, J./Tamayo, P.: Credit Risk Assessment using Statistical and Machine Learning Basic Methodology and Risk Modeling Application. In: Proceedings of Computational Economics’97 conference, 1997. Downloaded from: http://otn.oracle.com/products/ datamining/pdf/credit_risk.pdf, 4.12.2001
HaHe97 Hand, D. J./Henley, W. E.: Statistical Classification Methods in Consumer Credit Scoring: a Review. In: J. R. Statistic Soc. A (1997), 160, Part 3, P. 523-542.
Hand97 Hand, D. J.: Construction and Assessment of Classification Rules. Wiley series in probability and statistics, John Wiley & Sons, 1997.
Hech91 Hecht-Nielsen, R.: Neurocomputing, Reading, Mass., Addison-Wesley, 1991.
HeSc99 Heimann, J./Schumann, M.: Intelligent Agents for Automated Credit Decision. In: Tagungsband (Hrsg.): The Third International Stockholm Seminar on Risk Behaviour and Risk Management, 1. Aufl., Stockholm, 1999.
Heno83 Heno, R.: Kreditwürdigkeitsprüfung mit Hilfe von Verfahren der Mustererkennung. Verlag Paul Haupt Bern und Stuttgart, 1983.
Hofm90 Hofmann, V. H. J.: Die Anwendung des CART-Verfahrens zur statistischen Bonitätsanalyse von Konsumentenkrediten. In: Zeitschrift für Betriebwirtschaft, 60(1990), 9, 941-962.
Joos98 Joos, P./Vanhoof, K./ Ooghe, H./Sierens, N.: Credit classification: a comparison of logit models and decision trees. In: Gholamreza Nakhaeizadeh et al. (eds.), Application of machine learning and data mining in finance: European Conference on Machine Learning (ECML'98), Workshop notes; 24. April 1998, TU Chemnitz, P. 59-70.
Krah98 Krahl, D./Windheuser, U./Zick, F.K.: Data Mining: Einsatz in der Praxis. Addison- Wesley-Longman, Sparkassen-Informationszentrum, Bonn, 1998.
Krau93 Krause, C.: Kreditwürdigkeitsprüfung mit Neuronalen Netzen. IDW-Verlag GmbH, Düsseldorf, 1993.
Krus99 Kruse, A.: Antragsprüfung und Kartenüberwachung von privaten Kreditkartenkunden mit Künstlichen Neuronalen Netzen. Verlag Dr. KOVAC, Hamburg, 1999.
LiMo98 Liu, H./Motoda, H.: Feature selection for knowledge discovery and Data Mining. Kluwer Academic Publishers, London, 1998.
Liuy02 Liu, Y.: An framework of data mining application process for credit scoring. Research paper, Institute of Information Systems, University of Goettingen, Nr. 02/2002, Göttingen.
Lohr94 Lohrbach, T.: Einsatz von Künstlichen Neuronalen Netzen für ausgewählte betriebswirtschaftliche Aufgabenstellungen und Vergleich mit konventionellen Lösungsverfahren. Göttinger Wirtschaftsinformatik, Band 10, Göttingen, 1994.
MiAr95 Milton, J. S./Arnold, J. C.: Introduction to probability and statistics: Principles and Applications for Engineering and the Computing Sciences. McGraw-Hill, Inc., New York, 1995.
Mich94 Michie, D./Spiegelhalter, D. J./Taylor, C. C.: Machine Learning, Neural and Statistical Classification. Horwood, New York, 1994.
Müll96 Müller, J.: DV-gestützte Systeme zur Kreditwürdigkeitsprüfung bei Kreditversicherungen. Göttinger Wirtschaftsinformatik, Band 23, Göttingen, 1997. NaTu97 Nayak, G. N. / Turvey, C. G.: Credit risk assessment and the opportunity costs of loan
misclassification. In: Canadian Journal of Agricultural Economics 45 (1997), P. 285-299. Nieh87 Hiehaus, H.: Früherkennng von Unternehmenskrisen. IDW-Verlag GmbH, Düsseldorf,
1987.
Quin92 Quinlan, J. R.: Learning with continuous classes. In: Proceedings Australian Joint Conference on Artificial Intelligence, 1992, World Scientific, Singapore. P. 343-348. RoGl94 Rosenberg, E./Gleit, A.: Quantitative methods in credit management: A Survey. In:
Operations Research, Vol. 42, No. 4, 1994, P. 589-613.
Spac89 Spacman, K. A.: Signal detection theory: valuable tools for evaluating inductive learning. In: Proceedings of the sixth international workshop on machine learning. Morgan Kaufmann Publishers, INC. San Mateo, California, 1989, P. 160-163.
Schu91 Schumann, M.: Neuronale Netze zur Entscheidungsunterstützung in der Betriebswirtschaft, in: Biethahn, J./Bloech, J./Bogaschewski, R./Hoppe, U. (Hrsg.),
Wissenbasierte Systeme in der Wirtschaft - Anwendungen und Tools, Wiesbaden 1991, P. 23-50.
Utho97 Uthoff, C.: Erfolgsoptimale Kreditwürdigkeitsprüfung auf der Basis von Jahresabschlüssen und Wirtschaftsauskünften mit Künstlichen Neuronalen Netzen. M&P Verlage für Wissenschaft und Forschung, Stuttgart, 1997.
WaWi97 Wang, Y./Witten, I. H.: Introduction of model trees for predicting continuous classes. In: Proceedings of the poster papers of the European Conference on Machine Learning, 1997, University of Economics, Faculty of Informatics and Statistics, Prague. Downloaded from: http://www.cs.waikato.ac.nz/~ml/publications.html 14.08.2002. WeIn98 Weiss, S. M./Indurkhya, N.: Predictive Data Mining: A Practical Guide. Morgan
Kaufmann Publishers, Inc. San Francisco, California, 1998.
WeKu91 Weiss, S. M./Kulikowski, C. A.: Computer systems that learn: Classification and prediction methods from statistics, neural nets, machine learning, and Expert Systems. Morgan Kaufmann Publishers, Inc. San Mateo, California, 1991.
West00 West, D.: Neural network credit scoring models. In: Computer & Operations research, 27(2000), P. 1131-1152.
WiFr00 Witten, I. H./Frank, E.: Data Mining: Practical machine learning tools and techniques with Java implementations. Morgen Kaufmann Publishers, San Francisco, 2000.