Credit Scoring Methods A Comparative Analysis

aaaaaaa 1

RMG

Credit Scoring Methods

A Comparative Analysis

North American Power Credit Organization

Scottsdale: January 2013

RMG Financial Consulting, Inc. 813 East Ballard

Colbert, WA, 99005

Phone: (509) 468-2956 Fax: (509) 468-3217 Cell: (509) 990-0894

Usefulness of Company Scoring

 Credit scoring allows for consistency in counterparty evaluations

 Provides indicative ratings for non-rated counterparties

 Adds supporting data inputs for:

– Estimation of counterparty creditworthiness – Credit limit determination

– Estimation of credit reserves – Capital usage calculations – Capital adequacy calculations – Other portfolio metrics

Scoring Wholesale vs. Retail

Credit scoring for retail accounts (individuals and small businesses) tends to have fundamental differences from scoring wholesale accounts:

 Wholesale energy transactions tend to be of high dollar value and spread across a small group of counterparties.

 Sample sizes are much smaller and do not often lend themselves well to statistical analysis.

 Counterparties are fragmented into several distinct company types.

 Each industry sector tends to have specific financial profiles.

 Financial data and ratings tend to clump together in narrow ranges.

Common Methods of Credit Scoring

Fundamental analysis

Linear Regression

• Linear regression refresher (least squares) • Linear regression as descriptive analysis • Bankruptcy predictive regression

Asset volatility (EDF)

Multivariate weighted average scoring

Peer group comparison

Fundamental Analysis

RMG

Fundamental Analysis

Banks and rating agencies employ fundamental analysis when analyzing financial strength. Both banks and rating agencies tend to have open access to company management and specific financial information and forecasts that may not be disclosed in public filings.

 “Bottom up” approach – reviewing the individual financial status of each company in detail as a stand alone entity

 “Top down” approach – putting each company in the context of its industry and the overall economy

 Banks and rating agencies use both quantitative and qualitative elements in their process of evaluating company creditworthiness

There is no true substitute for thorough due diligence of one’s trading counterparties and the more you know the better able you are to make informed credit decisions.

Fundamental Analysis

We all must now be financial analysts – whether we like it or not.

We need to be able to source and evaluate many types of market and financial information in determining the creditworthiness of our trading counterparties.

10-K’s and 10-Q’s are the most detailed source of information generally available on a company’s financial standing, and we all need to be able to read, understand and adjust this information as necessary in performing adequate due diligence.

Company management has incentives to smooth financial data to meet earnings expectations and debt covenants.

Accounting rules and reported financial statement data does not necessarily reflect a company’s true economic condition.

Relying on Debt Ratings is Not Enough

Rating agencies are hesitant to change company ratings due to the possible effects on market participants. A recent study determined that on average, downward rating

changes lagged the market by approximately six months and upward rating changes lagged by approximately nine months.

RMG

CDS

Company Name Moody S&P Fitch Spread

1 Progress Energy, Inc. Baa2 BBB BBB 18.1

2 Duke Energy Corp Baa2 BBB BBB+ 30.0

3 Dominion Resources Inc Baa2 A- BBB+ 39.3

4 American Electric Power Company, Inc. Baa2 BBB- BBB 46.3

5 Northeast Utilities Baa2 A- BBB+ 46.6

6 TECO Energy Inc. Baa2 BBB BBB 55.0

7 Exelon Corp. Baa2 BBB BBB+ 74.8

8 Pepco Holdings Inc. Baa3 BBB BBB 80.4

9 NiSource Inc. na BBB- BBB- 82.2

10 Southern California Edison Co. A3 BBB+ A 88.5

11 DTE Energy Co. Baa2 na BBB 91.3

12 CenterPoint Energy, Inc. Baa3 BBB BBB 91.8

13 SCANA Corp Baa3 BBB BBB+ 101.8

14 CMS Energy Corp Ba1 BB+ BB+ 115.6

Linear Regression

RMG

Part II

•

Basic Least Squares Regression Model

•

Regression as Descriptive Statistical Analysis

•

Regression as Predictive Statistical Analysis

aaaaaaa 10

Basic Linear Least Squares Model

Y = a + b₁X₁ + b₂X₂ + b₃X₃ +…b_nX_n + E Assumptions:

 Causality is pre-existing in functional form

 Form of relationship is indeed linear

 Independent variables are not cross correlated

 Sample used is non-biased

 Errors are independent and have mean value of zero Causes for error / bias:

 Errors of measurement / missing data

 Omitted Variables

 Incorrect functional form

 Multicollinearity (non-independence of variables)

 Autocorrelation (non-independence of errors)

 Heteroskadasticity (non-constant variance of errors)

aaaaaaa 11

Basic Linear Least Squares Model

A few of the key resulting statistics:

R

R2 _{= 1 - Explained Sum of Squares}

Total Sum of Squares

F Statistic – Used to test the hypothesis that all coefficients in the

regression taken together are not statistically significant at a predetermined level of probability.

Standard Errors – Used to test the hypothesis that individual coefficients are not statistically significant at a predetermined level of probability

(….there are plenty of others)

Basic Linear Least Squares Model

Basic Linear Least Squares Model

RMG

Basic Linear Least Squares Model

RMG

So, let’s look at a simple regression for Investor Owned Utilities: Rating = a +b1(PTROA) + b2(EBIT/Int) + b3(TD/TC) +E

RMG

Linear Regression as Descriptive Analysis

Stats

Return on

Assets

Interest

Cov'g

Total

Debt/Cap Intercept

(x1)

(x2)

(x3)

b

mn..., b

0.238

0.298

-0.038

13.267

sen…, se

0.114

0.144

0.029

2.195

r

, se

0.362

2.901

#N/A

#N/A

F, df

15.3

81

#N/A

#N/A

ss

, ss

387

682

#N/A

#N/A

RMG

Linear Regression as Descriptive Analysis

Bankruptcy Predictive Regression

Attempts to use regression analysis on historical financial data to determine a predictive model for company failure.

Altman (1968): Multivariate analysis

Z

= a

x

+ a

X

+ … + a

x

where: x_i = financial ratios

successful company: Z_i => z

failure: Z_i <= z

The Altman Z-score was initially based on a sample of 66 manufacturing companies, 33 of which had filed for bankruptcy during the period 1946 through 1965. Altman’s original model correctly identified 79% of the sample one year prior to failure.

Bankruptcy Predictive Regression

RMG

“Zone of Ignorance”

aaaaaaa 20

Altman’s Model

Z = 0.012 X₁ + 0.014X₂ + 0.033X₃ + 0.006X₄ + 0.999X₅

Where:

X₁ = working capital / total assets X₂ = retained earnings / total assets X₃ = EBIT / total assets

X₄ = market value of equity / book value of liabilities X₅ = sales / total assets

Z-scores of greater than 2.99 clearly represent non-failure Z-scores of less than 1.81 clearly represent failure

Altman subsequently developed a revised Z-score model which dropped variables X₄ and X₅ and replaced them with a new variable, X₄ = net worth (book value / total liabilities). Sales / total assets was dropped to minimize potential industry effects relating to asset turnover.

Altman (continued)

In 1977 Altman developed a private company, ZETA Services, Inc. that sold a new model touted as being, “far more accurate in bankruptcy

classification…” While the coefficients were not specified, the model was based on the following factors (see: Altman, E. I. (2000): “Predicting

Financial Distress of Companies: Revisiting the Z-Score and ZETA Models.” Stern School of Business, New York University.):

 Return on assets

 Stability of earnings

 Debt service

 Cumulative profitability

 Liquidity / current ratio

 Capitalization (five year average of market value)

 Size (total tangible assets)

Other Bankruptcy Prediction Models

Assumes that net assets follow a random walk process with some fixed probability of a negative cash flow in each period. For long enough periods there is a probability for a clustering of net negative cash flows that will

exceed net assets and the ability to borrow. Ohlson’s O-Score (1980)

Ohlson employed logit regression on a much larger sample size of 105 bankrupt and 2,058 non-bankrupt companies.

Support Vector Machine Model – (several authors)

Employs statistical learning theory and artificial neural networks approach to finding specific solutions – while complex the approach works well with small sample sizes

Linear Regression and Bankruptcy

Prediction Models

The usefulness of regression analysis in determining default, while interesting, is questionable. The usefulness of fail / non-fail prediction models as described by Ohlson:

“…real world problems concern themselves with choices which have a

richer set of possible outcomes. No decision problem I can think of has a payoff space which is partitioned naturally into the binary status of bankruptcy versus non-bankruptcy…Most of the analysis should

simply be viewed as descriptive statistics – which may, to some extent, include estimated prediction error-rates, and no ‘theories’ of bankruptcy or usefulness of financial ratios are tested.”

RMG

Asset Volatility Models

RMG

aaaaaaa 25

Asset Volatility Models

Asset volatility models are thought to be more accurate than the older

probability of bankruptcy (PB) model’s, such as Altman and Ohlson since: • While PB estimates are statements of the likelihood of future events they

rely on accounting information that is designed to measure past performance

• Financial statement info is formulated based on the accounting principle of ongoing-concern which assumes a company won’t go bankrupt, thus the PB model is limited by design

• Accounting principles incorporate conservatism and can cause assets to be underestimated, injecting bias into the structure of the PB model

• PB models do not incorporate asset volatility which captures the

likelihood that the value of a firm’s assets may decline (two companies with the same Debt/Capital ratio may have significantly different

likelihoods of default)

Asset Volatility Models

Asset volatility models use a Black/Scholes/Merton (BSM) option pricing theory to determine a debt/equity-based estimate of default probability as of a point in time.

Asset volatility assumes that a company becomes more likely to default as the market value of the company converges to the value of the company’s debt - much like an option is assumed more likely to be exercised as the market price converges to the strike price of the option.

Asset volatility models employ market data (share price volatility) and financial data (debt levels) to calculate estimated default frequencies (EDF).

Sources for EDF’s include: - Moody’s KMV - Bloomberg - CreditGrades.

aaaaaaa 27

Asset Volatility Models

Asset values are unobservable, but using the equity value today (which is observable) and the Black-Scholes model, the asset value (and asset volatility) today can be estimated. Standard Black-Scholes-Merton: VE = N(d1) – e-rTXN(d2), where: d1 =(ln (VA/X) + (r + ½ σA2/2)T) / σAT-½, and d2 = d1 - σAT-½ DD = (ln (V_A/X) + (μ + ½ σ_A2/2)T) / σ AT-½ P_def = N(-DD)

VE = equity value X = book value of debt r = risk free rate

VA = asset value T = time to maturity of longest debt

σA = asset volatility N = normal distribution

DD = Distance to default P_def = Probability of default

Relying on Market Indicators is Not Enough

Market indicators, such as estimated default frequencies (EDF’s), credit swap rates and bond spreads provide good leading indicators of

deteriorating or improving credit quality, but market indicators don’t tell you WHY a company’s credit quality is improving or declining.

Market indicators such as EDF’s are subject to share price volatility and market perceptions and tend to overstate downturns and understate upturns in credit quality through a business cycle.

Bond spreads and share price volatility may reflect more factors than simply credit premiums / credit risk.

Market indicators assume market and price transparency – some bonds or equities may have limited transactions: not all markets are fully transparent.

Multivariate Weighted Average Models

RMG

Multivariate Weighted Average Models

Multivariate weighted average (MWA) scoring models use financial and non-financial indicators to produce company rankings.

 Weighting factors and scoring tables act to compare companies to their peers within industry groups.

 Employs a straightforward methodology which is easily understood.

 Scoring results can easily be mapped to common financial scales such as debt ratings.

 A well designed and well tested MWA model can produce consistent and accurate results.

aaaaaaa 32

Basic MVWA Model

Assumptions:

 Causality is pre-existing in functional form

 Choice of ratios are significant (appropriate)

 Weightings are appropriate Causes for error / bias:

 Errors of measurement / missing data

 Omitted or incorrect ratios

 Incorrect weightings

RMG

(Ranked AAA to D)

Ratio (1) Scoring Vector (1) Weighting (1)

Ratio (2) Scoring Vector (2) Weighting (2)

Ratio (3) Scoring Vector (3) Weighting (3)

Score = . X . X .

. . .

. . .

MVWA Model Issues

Choice of weighting factors:

 Which factors should be used?

 How should they be weighted?

 How to make an informed decision?

Development of scoring tables:

 How to determine discrete scoring vectors?

 Ranges of sample data may be narrow

 Samples sizes are often small

 Data may not be statistically significant

When initial results are not statistically significant, the “art” of financial analysis becomes as useful as the “science” of statistical analysis.

MVWA Model Issues

Choice of financial ratios:

Pearson correlation results of Debt Ratings to various financial ratios

RMG

Sample Size (Over 5 year-ends and 1 Q3) 20 69 126 125

Ratio Merchants IOU Publics Corp

1 Pre Tax Return On Equity 0.11 0.23 0.01 0.02

2 Pre Tax Return on Assets 0.40 0.43 0.03 0.44

3 Operating Income / Sales 0.30 0.29 0.04 0.29

4 TNW / Total Assets 0.36 0.41 0.09 0.09

5 Operating Cash Flow / Total Debt 0.38 0.30 0.05 0.12

6 Funds from Operations / Total Debt 0.40 0.43 0.03 0.20

7 EBIT / Interest Expense 0.09 0.18 0.01 0.43

8 EBITD / Total Debt 0.35 0.34 0.03 0.21

9 Total Debt / Total Assets -0.70 -0.52 -0.13 -0.14

MVWA Model Issues

RMG

MVWA Model Issues

RMG

MWA Model Issues

Limited scope and range of data sets:

RMG

Peer Group Analysis

RMG

aaaaaaa 39

Basic Peer Group Model

Assumptions:

 Causality is pre-existing in functional form

 Choice of ratios are significant (appropriate)

 Weightings are appropriate Causes for error / bias:

 Errors of measurement / missing data

 Omitted or incorrect ratios

 Incorrect weightings

RMG

(Ranked by quartile)

Ratio (1) Quartile Averages (1) Weighting (1) Ratio (2) Quartile Averages (2) Weighting (2) Ratio (3) Quartile Averages (3) Weighting (3)

Score = . X . X .

. . .

. . .

aaaaaaa 40

Peer Group Analysis

Peer group analysis is a simple statistical approach to multivariate weighted average scoring.

For a quick and easy way to build scoring tables why not just compare a company’s financial ratio results to other companies in its SIC code? Method:

- Define the SIC codes for each company type (peer groups) - Access financial data for all companies within each SIC code

- Choose a few financial ratios and run them for all companies within each SIC code

- Stratify the resulting peer group data around mean values

And you have an easy way to build discrete scoring vectors to score companies against, … right?

aaaaaaa 41

Peer Group Analysis - Limitations

 Data may fall within a limited range.

 Compressed strata may bias results. Choice of peer groups

 SIC code groupings should be screened.

 Some companies within an SIC Code Classification may not belong. Peer group size

 Peer group must be at least large enough to adequately calculate scoring vector.

Market cyclicality and peer group bias

 Peer groups may trend together over time and through a business cycle. Omission bias

 Missing data will imply bias in calculating scoring vectors and applying rankings.

Peer Group Analysis - Summary

Peer group comparison is only useful if one has: - Well thought out methods

- Well defined peer groups - Large sample sizes

- Broad ranges of sample data

- A complete data set for each peer group - Data that doesn’t track together over time

So…peer group analysis is only useful if you have thoroughly analyzed the financial profile of each company type and the data being used and back tested your model results.

Other Topics

Unadjusted Financial Data is Not Enough

Company management has incentives to smooth financial data to:

 Meet earnings expectations

 Satisfy debt covenants

 Qualify for bonuses

 Just because they can

Accounting rules do not necessarily reflect a company’s true economic condition – an issue of “Accounting vs. Economic Value.”

Accounting value follows GAAP accounting rules that allow for timing adjustments, deferrals of expenses and the matching of asset value to revenue recognition.

Economic value looks to the underlying market value of a company’s assets, liabilities and earnings independent of GAAP.

Model Validation

 Any scoring model should be well documented and well tested.

 All assumptions should be clearly defined.

 Data sources should be tested and verified.

 A scoring model should be thoroughly tested for accuracy and

completeness, and this process of validation should be re-run and updated on a regular basis.

– How can you tell if your model’s scoring results are accurate and meaningful?

– What baseline do you compare your model’s results against?

– How do you best quantify the accuracy (or lack thereof) of your model’s results?

Model Validation - Back Testing

RMG

Adjusted Std Error = 0.65 Standard Error = 1.02 Sample Size = 97

Model Validation - Back Testing

RMG

Adjusted Std Error = 0.30 Standard Error = 0.68 Sample Size = 15

Model Validation - Back Testing

RMG

Sample Size = 128

Standard Error = 0.98

Model Validation - Back Testing

RMG

Sample Size = 19

Standard Error = 2.50

Summary

Credit scoring models can provide useful and straightforward methods for ranking wholesale counterparties; however, your model must be:

- Well thought out and documented - Based on prior financial research

- Able to score different company types - Thoroughly tested and documented - Not over-relied upon

- Not used as a substitute for thorough due diligence

Bibliography:

E. Altman and H. Rijken: (2006) “A Point in Time Perspective on Through-the-Cycle Ratings,”, Financial Analysts Journal, Volume 62 No.1, January/February

L. Revsine, D. Collins and W. Johnson: (1998): “Financial Reporting and Analysis,” Prentice Hall

A. Sondhi and D. Fried: (1994) “The Analysis and Use of Financial Statements,” G. White, John Wiley & Sons, Inc.

Moody’s Investor Service: (July 2005) “Moody’s Approach to Global Standard

Adjustments in the Analysis of Financial Statements for Non-Financial Corporations – Part 1,”

Maddala, G.S.(1983): Limited dependent and qualitative variables in econometrics, Cambridge University Press, Cambridge

Maddala, G.S.(1987):”Limited Dependent Variable Models Using Panel Data”, The Journal of Human Resources..Vol.3,p.307-337

aaaaaaa 52

Bibliography:

Hillegeist, Cram, Keating and Lundstedt (2003): “Assessing the Probability of Bankruptcy,” Review of Accounting Studies.

Wlcox, J. W., (1971): “A Gambler’s Ruin Prediction of Business Failure Using Accounting Data,” Sloan Management Review, Vol. 12.

Wilcox, J. W., (1973): “A Prediction of Business Failure Using Accounting Data,” Journal of Accounting Research.

Begley, J., Ming, J., and Watts, S., (1997): “Bankruptcy Classification Errors in the 1980s: An Empirical Analysis of Altman’s and Ohlson’s Models,” Review of

Accounting Studies.

Scott, J., (1981): “The Probability of Bankruptcy: A Comparison of Empirical Predictions and Theoretical models,” Journal of Banking and Finance.

Chung, H. M., and K. Y. Tam (1992): “A Comparative Analysis of Inductive-Learning Algorithms,” Intelligent Systems in Accounting, Finance and Management.

Johnson, C. G., (1970): “Ratio Analysis and the Prediction of Firm Failure,” Journal of Finance.

Kelejian, H., and Oates, W. (1974): “Introduction to Econometrics, Principles and Applications,” Harper and Row

aaaaaaa 53

RMG

Credit Scoring Methods

A Comparative Analysis

North American Power Credit Organization

Scottsdale: January 2013

RMG Financial Consulting, Inc. 813 East Ballard

Colbert, WA, 99005

Phone: (509) 468-2956 Fax: (509) 468-3217 Cell: (509) 990-0894