Evaluation of Credit Guarantee Policy using Propensity Score Matching

Evaluation of Credit Guarantee Policy using Propensity Score Matching

Inha Oh ^a, Jeong-Dong Lee ^a, Gyoung-Gyu Choi ^b and Almas Heshmati ^c

aTechno-Economics and Policy Program, College of Engineering, Seoul National University,

San 56-1, Shillim-Dong, Gwanak-Gu, Seoul 151-742, KOREA

b Department of Business Administration, Dongguk University,

26, 3-Ga, Pil-Dong, Chung-Gu, Seoul 100-715, Korea

C Seoul National University and University of Kurdistan - Hawler Last Update: September 10, 2006

Abstract

In the aftermath of the Asian financial crisis in Korea, the credit guarantee policy was used as an instrument by the government to support small and medium enterprises (SMEs). However, the effect of the credit guarantee has not been carefully studied and the policy has been criticized for impairing development of innovative private financial sector and for making SMEs highly dependent on government policy. In this paper, we evaluate the effect of the credit guarantee policy by comparing a large sample of guaranteed firms and matched non-guaranteed firms from 2000 to 2003. The sample firms are compared with respect to growth rates of different performance indicators including: productivity, sales, employment, investment, R&D, wage level and survival of firms in the post crisis period. To avoid the selectivity problem, recently developed propensity score matching methodologies are adopted. Results suggest that the guarantee provision influenced significantly for supported firms to maintain their size in terms of sales and employment, and increase their survival rate, but credit guarantees did not help firms to increase their R&D and investment and hence, growth in productivity. Moreover, the selection of firms to receive guarantee funds was not linked to the productivity of the supported firms.

Keywords: credit guarantee, selection bias, propensity score matching, SME.

JEL Classification Numbers: C40, H43, H81, L25, L53

1. Introduction

The credit guarantee was used as a policy instrument by the Korean government in its support towards small and medium enterprises (SMEs) on the afterward of the Asian financial crisis. However, the credit guarantee programs has been criticized by many domestic and foreign scholars for its negative effects by impairing development of innovative private financial sector and for making SMEs highly dependent to government policy measures (Kang, 2005; Lee, 2006; IMF, 2005). Even though there have been some qualitative remarks on the effectiveness of the policy, evaluation of credit guarantee policy has not been conducted systematically in terms of methodology and data.

This study aims at filling the gap in the existing literatures. In that, we evaluate the effect of the credit guarantee policy in terms of growth of productivity, sales, employment, investment, R&D, wage level of the supported firms and their survival rates in the post crisis period. The reliability of measures of effectiveness of the policy is hampered by the ‘selectivity’ problem, which implies that public funding goes to firms having a priori favorable conditions to correspond to the policy objectives (Jaffe, 2002; Blundell and Costa Dias, 2000). In this paper, we deal with the ‘selectivity’ issues by employing recently developed propensity score matching methodologies.

1.1 Impact of Asian financial crisis on Korean manufacturing sector

Manufacturing is a major contributor to the Korean Economy. It is highly export oriented and as such subject to both domestic and international competition and various market related risks. The Asian financial crisis, which occurred from mid-1997 to 1998, can be a representative example of such external risk. In 1998, the gross domestic product (GDP) level decreased by 6.7% and fixed investment decline was as much as 40% (Borensztein and Lee, 2002). Although the Korean economy severely suffered from the Asian financial crisis, it is well known for its rapid recovery from the crisis (Koo and Kiser, 2001). During the process of recovery, the transparency and the general market confidence for the Korean economy has significantly improved (Hong and Lee, 2000). However, one can expect that the impact of crisis on firms would be different

depending on the firms’ characteristics, e.g. size classes.

During and after the crisis, the International Monetary Fund (IMF) set a large number of guidelines mainly regarding restructuring and downsizing the conglomerates labeled as Chaebols¹ and large scale enterprises (LSEs). Revision of bankruptcy-related laws and implementation of the outside courts workout programs mainly for Chaebols were also executed. As a result, some notable changes happens; for example, a number of Chaebols became bankrupt and average debt/equity ratio of LSEs has gone down dramatically.

The impact of the economic crisis on SMEs was more disastrous than on the LSEs counterpart. For example, the number of SME bankruptcies in 1998 reached 22,800, while its corresponding number was only 11,600 in 1996 (Gregory et al., 2002). After the Asian financial crisis, SMEs in Korea faced great difficulties to regain its position due to decreased productivity and profitability, inefficient restructuring plans, slowdown in demand and the rise of the Chinese manufacturing sectors (Kim and Lee, 2002).

Table 1 compares status of LSEs and SMEs throughout the crisis periods and confirms arguments discussed above. Especially, the profit ratios and debt equity ratios clearly indicate the gap between the two size classes after the crisis; i.e. increased profit ratio and stabilized debt equity ratio for LSEs which are in concordance with IMF restructuring plan. The difference in R&D intensity for LSEs and SMEs persisted, which implies that performance gap between the two groups is rather difficult to be further reduced.

LSEs succeeded in the required restructuring during the aftermath of the financial crisis and grew rapidly in terms of profitability and output, particularly in the export market.

However, SMEs were still suffering from the effects of the recession and, in particular, in the domestic market. The proportion of SMEs making loss increased from 17.7 percent in 2002 to 21.3 percent in 2003 (IMF, 2005). It seems that the restructuring of SME industries was less successful and insufficient. This led to the fact that many SMEs managed to survive with the financial support program provided by the government.

Oh et al. (2006), by comparing plant turnover and productivity dynamics of Korean

manufacturing industry by size in pre- and post-crisis periods, observed the undesirable exit of SMEs with higher productivity, especially pronounced in the post crisis period.

The study confirmed the existence of ‘zombie firms’ which were kept alive by support from the public financial sector as one of the sources of the negative exit effect.

1. 2. The credit guarantee policy

SMEs face great difficulties to finance investment due to asymmetric information which arises from the lack of financial information and standardized financial statements.

Stiglitz and Weiss (1981) showed that in equilibrium a loan market may be characterized by credit rationing. In perfect economic system, if prices do their duty, rationing should not exist. However, credit rationing does in fact exist due to an excess demand for loanable funds. In loan markets, there exists residual imperfect information after the evaluation of loan applications. Due to imperfect information and the adverse selection, it is often difficult for SMEs to borrow funds even at higher interest rates.

The belief that capital markets do not provide adequate funds for new businesses is also one of the rationales for government loan assistance programs to SMEs (Evans and Jovanovic, 1989). Many governments provide subsidized loans and loan guarantees to SMEs for start-up and expansion. The USA, UK, France, Belgium, the Netherlands and others have adopted financial assistance programs for unemployed workers who start businesses, let alone the efficiency of the programs (Bendick and Egan, 1987).

The Korean government in its efforts tried to avoid systemic risk bringing about a contagious failure of solvent but temporarily illiquid SMEs in the event of a credit crunch. In order to prevent the decline in bank loans to SMEs immediately after the crisis, the government sharply expanded its credit guarantees schemes. The purpose of credit guarantee was to provide financial support to SMEs suffering from insufficient investment from private financial institutions due to market failures, to enhance competitiveness of SMEs and finally to increase SMEs’ accessibility to private financing. The credit guarantee gives warranty to private investors by reliving the risks of lending to SMEs.

The credit guarantee scheme was one of the most influential public support policies

aimed at SME sector for the case of Korean economy. The amount of government contribution to credit guarantee funds was about 1 trillion KRW² in 2003, which will increase total guarantee balances by nearly 11 trillion KRW, considering the operation multiple³ which lies between 9.7 and 16. And large portion of SMEs lending money from banks are utilizing the credit guarantee scheme (1/4 for the case of deposit banks and 1/3 for the case of commercial banks). On the other hand the amount of other public supporting policy measures to SMEs was about 6 trillion KRW in the same period (Lee, 2006).

There exist two major public credit guarantee institutions in Korea⁴; the Korea Credit Guarantee Fund (KCGF) and the Korea Technology Credit Guarantee Fund (KOTEC).

KCGF is a pubic financial institution established in 1976 under the provision of the Korea Credit Guarantee Fund Act. The objective of KCGF is to lead the balanced development of the national economy by extending credit guarantee services for the liabilities of promising enterprises which lack tangible collateral and stimulating sound credit transactions through the efficient management and use of the credit information.

KOTEC was founded in 1989 under the Financial Assistance to New Technology Businesses Act which went through a full-scale revision and was newly titled Korea Technology Credit Guarantee Fund Act in 2002. The mission of KOTEC was to contribute to the national economy by providing credit guarantees to facilitate financing for new technology-based enterprises while promoting the growth of technologically advanced SMEs and venture businesses.

Even though both institutions are providing public guarantee service to mainly SMEs, but the missions are different. KCGF was aimed to provide credit guarantee to firms with competitive power, but lack of credit ratings or tangible collateral to pull investment, while KOTEC was developed to provide guarantee funds to technology oriented firms in SME sectors. Figure 1 shows the development pattern of credit guarantee balance for the two guarantee institutions. As can be seen in the Figure 1, the remaining balance of credit guarantee increased steeply after the crisis periods (after 1998), which implies that government used these two credit guarantee institutions as a policy instrument to give support to the SME sectors during and after the financial crisis.

The amount of credit guaranteed by government soared and it reached almost 6 to 8

percent of GDP, which is higher than other countries, e.g. 0.1 percent in USA, 0.02 percent in UK, 0.2 percent in Germany in 2004. Even for the case of Taiwan, where SMEs play an important role in the overall national economy, the ratio of credit guarantee balance to GDP is only less than 3 percent (Kang, 2005).

Right after the crisis, government prepared large amount of public funds and a major portion of which was given as a public support to temporary illiquid SMEs in broad sense. One can easily imagine that the two guarantee institutions might lack experience or capacity of conducting detailed investigation for all applications and of selecting eligible firms. KCGF has adopted the sophisticated corporate credit rating system (CCRS) only after 1999. Based on the accumulated financial data of firms and considering potential risks on management and operation, CCRS mechanically calculate credit rating of each firm which will be used for screening process to select firms to be supported. However, after the crisis the workload for employees of KCGF who are in charge of screening process increased significantly due to soaring number of applicants received by the guarantee funds and increased amount of guarantee supply. Compared to 1997, the amount of guarantee supply per examiner was 3.63 times higher and the number of firms per examiner was 2.53 times higher in 2001 (Lee, 2006). Besides KCGF has utilized circulation of job mission of employees inside the institution for every 2 or 3 years, which seemed not to be very suitable to train examiners, specialized in evaluation and screening processes. On the other hand, KOTEC aimed to give credit primarily to technology based newly founded firms, but as pointed out in Kang et al.

(2006) and shown in Table 2, the amounts of credit guarantee supply based on the detailed technology assessment was less than 4 percent in 2001. Thus, we can infer that most of screening process to pick firms to receive support was insufficient.

Figure 3 shows the increasing trend of guarantee default in 2003 for both KCGF and KOTEC. The average length of guarantees provided by both of institutions was around five years, implying that most of firms subject to guarantee default are the ones originally selected in 1998 when guarantees were expanding in the wake of the crisis.

That is to say, the increasing default rates can be attributable to the termination of guarantees which were assigned right after the crisis. This fact partly explains that, right after the Asian financial crisis, lots of marginal firms were selected in the guarantee

policy, due to lack of capacity to evaluate credit condition of firms, and sudden increase in demand for both institutions.

There exist few recently published studies analyzing credit guarantee policy, which are mostly qualitative critics about the policy scheme. According to Lee (2006), the object of government intervention in SME loan market was to compensate for market failure due to information asymmetry and external effects. However, as an agency of government, guarantee institution had its own objective to maintain operational efficiency and long-term sustainability. Moreover the incentive structure among the three key players, guarantee institutions, banks, and SMEs, is inappropriately coordinated, since most of risk should be handled by the guarantee institutions. And no one was really taking responsibility for losses from the guarantee defaults. The study indicated that the divergence of objectives between government and institutions and inappropriately coordinated incentive structure lead to insufficient policy performance, and eventually to financial burden for the government. Kang (2005) pointed that the non-selective government support of SMEs was one of the key sources of the sluggish SME restructuring process after the financial crisis. It worsened the SME market environment in two ways: the first was the effect of crowding out the private financial.

The second and long-term negative effect was to make SMEs become more dependent on public support. The survival of uncompetitive SMEs with the help of government support might result in a decrease in market share and profits of competitive firms. Thus, uncompetitive firm might replace competitive one with the public intervention.

IMF (2005) described the behavior of government intervention in Korea as ‘the ubiquitous hand’. In the review of IMF, the policy intervention after the crisis brought about financial difficulties of SMEs and provided negative incentive larger corporations for investment. IMF shares the point raised by Kang (2005) that inappropriate public intervention toward SMEs through non-selective guarantee support hinder the voluntary market based restructuring of SMEs. Based on the observations, IMF suggested reducing the credit balance ratio to GDP by 1 percent per annum for the next 5 years.

We know that several researches agree the common point that the public intervention including credit guarantee program has had negative effect. However, not many quantitative researches have been done especially for the effectiveness of the credit

guarantee policy mainly due to the unavailability of consistent data and appropriate methodological framework.

In this research, we evaluate the effect of the credit guarantee policy in terms of employment, investment, survival and productivity in the post crisis period. The amount of funds allocated to the credit guarantee is huge and the number of targeted firms is large, then we need to investigate its effectiveness and to provide background information for further evolution of the policy.

Methodologically, reliable policy evaluation should solve the ‘selectivity’ problem, which implies that public funding may be allocated to proposals judged in advance with high probability to succeed. If we cannot control selectivity problem, we might over- or under-estimate the true effect (Jaffe, 2002; Blundell and Costa Dias, 2000). To deal with the selectivity issue, we adopt propensity score matching techniques, which has been developed in labor economics (Dehejia and Wahba, 1999, 2002; Frölich et al., 2004;

Heckman et al., 1997; Smith, 2000), and recently has been applied to firm-level studies (Arnold and Hussinger, 2005; Yasar and Rejesus, 2005; Lööf and Heshmati, 2005).

This study has a number of contributions to the existing literature. First, it utilizes original dataset that covers all manufacturing firms with 5 more employees in Korea and all the guaranteed firms. Especially the data set for guaranteed firms are unique since all the guaranteed firms are included and non-financial characteristics, in addition to financial ones, are covered. Second, we analyze the effectiveness of the public policy in terms of survival of firms as well as traditional growth measures. Third, we adopt the most up-to-date statistical methodology in the field of propensity score matching.

The remainder of this study is organized as follows. Section 2 reviews the methodologies of propensity score matching. Section 3 describes the data and presents key descriptive statistics. The empirical results are discussed in Section 4. The final Section 5 summarizes and concludes this study.

2. Propensity Score Matching

In this section we introduce the propensity score matching technique. The method has been developed in the field of labor economics and adopted in many field for policy

evaluation. We review the most frequently used matching methods and briefly argue for factors in the support of their use, the strengths and weaknesses of such methods in evaluation of public support programs. We also elaborate with properties of the estimates of treatment effects.

2.1 The difficulties in evaluation studies - selection bias

Since Rubin (1974), the effect of a program or a support is defined as a created value added by participating in a program. In other words, the effect of program can be defined as ‘what would have happened to those who, in fact, did receive treatment, if they had not received treatment (or vice versa)?’ (Rubin, 1974). Thus, we have to compare the factual and (hypothetical) counterfactual situations. Unfortunately, we cannot observe this counterfactual in real situation, since participants are observed in only one factual state. Mere comparison between supported and non-supported groups can not identify the exact additional effect of the support program, since their characteristics before participation in the supporting program were different already.

The concept of treatment effect defined by the additional value added should, thus, be based on the appropriate construction of a counterfactual. In other words, the policy treatment effect can be defined as the difference between real outcome and hypothetical outcome represented by the counterfactual. Modern evaluation methods are focusing on estimating this counterfactual (Blundell and Costa Dias, 2000).

One other issue, except for the construction of counterfactual, is the well-known

‘selectivity’ problem, which implies the recipient firms are eligible before the implementation of the policy program. Jaffe (2002) expressed this issue that the projects that are the best candidates to be funded - in the sense of maximizing the impact of public support - are also the projects that would have the highest expected output in the absence of funding (Jaffe, 2002).

Given the counterfactual and selection problems, the most appropriate measure of the effectiveness of government support might be comparing the performances of two firms with same characteristics, assuming that one received support (or treatment) and the other did not. However, it is hard to find appropriate comparison groups which can

represent non-supported firms in evaluating the program.

In this study we apply the recently developed propensity score matching (PSM hereinafter) methodology (Rosenbaum and Rubin, 1983; Heckman et al., 1998; Dehejia and Wahba, 2002), which allows to construct a comparison group by matching twin units based on the propensity score in the population of untreated groups. With this modern approach, we expect to solve the selectivity problem and to compare the factual and counterfactual as a result.

2.2 PSM methodologies

PSM methodology was first introduced by Rosenbaum and Rubin (1983). The concept of PSM is based on the strongly ignorable treatment assignment assumption. It means that conditioned on the observable characteristics (X variables) of possible participants, the decision for participation of the program should be independent of the outcome measures. This is so-called conditional-independence assumption (CIA). CIA in this respect can be written as following:

(1) ^{( , )Y Y0 1 ⊥T X}

where Y means the outcome in the treated state and ¹ Y denotes the outcome in the ⁰ untreated state. T is an indicator variable denoting participation in the program.

Another condition is that the probability to participate in the program for program group and comparison group should lie in the same domain, which is called common support condition.

If these assumptions and conditions are satisfied and when there exist sufficient number of observable variables related to the characteristics of participants to a program, it is theoretically possible to obtain unbiased estimation of effect of a program.

Propensity score indicates a conditional probability of applicants to participate in a program when observable characteristics of applicants are given. In other words,

(2) Propensity score= =P P X( ) Pr(= T =1 )X .

Roesenbaum and Rubin (1983) proved the following two lemmas under CIA and the common support condition:

Lemma 1: If P(X) is the propensity score, then (3) ^{X ⊥T P X( )}.

Lemma 2: Under CIA and Lemma 1, then the conditional independence result extends to the use of the propensity score as:

(4) ( , )Y Y_{0 1} ⊥T P X( ).

Based on the above lemmas, for a population of units denoted by i, we can define the policy impact, which is defined as the difference between real and counterfactual outcomes, as the average effect of treatment on the treated (ATT) as follows:

(5)

{ }

( )

{ }

( ) ( )

{ }

1 0

1 0

CIA

( ) 1 0

- 1

- 1, ( )

1, ( ) - 0, ( ) 1

i

i i i

i i i i

P X i i i i i i i

ATT E Y Y T

E E Y Y T P X

E E Y T P X E Y T P X T

= =

= =

= = = =

where the outer expectation is taken over the distribution of P(Xi) in the population of participants, Ti=1.

We can estimate the propensity score and test the balancing hypothesis (3), known as balancing test, according to the iterative algorithm suggested by Dehejia and Wahba (2002) and Becker and Ichino (2002)⁵.

Once the balancing test is satisfied, we can confirm that the selected observed variables and their interaction and square terms reflect the assignment mechanism sufficiently and the use of calculated propensity score as a conditioning variable is acceptable.

However, estimation of the propensity score only, is not enough to estimate the ATT of interest using equation (5). This is because the probability of observing two units with exactly the same value of the propensity score is in principle zero, since P(X) is a continuous variable. Various methods have been proposed in the literature to overcome this problem, and three of the most widely used are (i) nearest neighbor matching, (ii) radius matching and (iii) kernel matching, which are summarized as follows⁶.

2.2.a. The Nearest Neighbor Matching

The nearest neighbor matching method is the most common form of matching in the statistics literature. In this method, each treated unit is matched to an untreated unit with the nearest propensity score. Let T be the set of treated units and C the set of control units, then:

(7) ( ) min _{i j}

C i = j P−P

Here Pi and Pj are the propensity scores of treated and untreated units, respectively.

Once each treated unit is matched with an untreated unit, the difference between the outcome of the treated units and the outcome of the matched untreated units can be computed. The ATT is then obtained by averaging these differences as follows:

(8)

( )

( ) ( )

1

1 1 1

T C

i ij j

T

i T j C i

T C T C

i ij j i ij j

T T T

i T i T j C i i T j C i

ATT Y w Y

N

Y w Y Y w Y

N N N

∈ ∈

∈ ∈ ∈ ∈ ∈

⎡ ⎤

= ⎢ − ⎥

⎣ ⎦

⎡ ⎤

= ⎢ − ⎥= −

⎣ ⎦

∑ ∑

∑ ∑ ∑ ∑ ∑

.

In the case of the nearest neighbor matching method all treated units can find a match of their own. However, it is obvious that some of these matches are fairly poor because for some treated units the nearest neighbor may have a very different propensity score.

Nevertheless this seemingly inappropriately matched pair would contribute to the estimation of the treatment effect, which results in unreliable estimate of ATT (Becker and Ichino, 2002).

2.2.b. The Radius Matching

With radius matching each treated unit is matched only with the untreated units whose propensity score falls within a pre-specified range of neighborhood of the propensity score of the treated unit. If the range of the neighborhood, i.e. the radius, is set to be very small it is possible that some treated units are not matched because their neighborhood does not contain any untreated units. On the while, the smaller the size of the neighborhood the better is the quality of the matches. However, by using more

untreated observations, one can increase the precision of the estimates, but at the cost of increased bias (Caliendo and Kopenig, 2005).

In radius matching the untreated unit is defined as:

(9) ^{C i( )=}

{

}

that is, all the untreated units with estimated propensity scores falling within a radius of r from Pi are matched to the treated unit i. The radius matching shares the attractive feature of over sampling mentioned above in the nearest matching, but avoids the risk of attaining bad matches. We can also obtain ATT from (8).

2.2.c. The Kernel Matching

With the kernel matching all treated units are matched with a weighted average of all untreated units with weights that are inversely proportional to the distance between the propensity scores of treated and untreated units such that:

(10)

1

j i

C j

j C n

T T i

i T k i

k C n

P P

Y G h

ATT Y

N P P

G h

∈

∈

∈

⎧ ⎛ − ⎞⎫

⎪ ⎜ ⎟⎪

⎪ ⎝ ⎠⎪

= ⎨⎪⎪⎩ − ⎛⎜⎝ − ⎞⎟⎠ ⎬⎪⎪⎭

∑ ∑

∑

where G(·) is a Gaussian kernel function and h_n is a bandwidth parameter. Under standard conditions on the bandwidth and kernel, a consistent estimator of the counterfactural outcome Y0 is given by:

(11)

j i

C j

j C n

k i

k C n

P P Y G h

P P G h

∈

∈

⎛ − ⎞

⎜ ⎟

⎝ ⎠

⎛ − ⎞

⎜ ⎟

⎝ ⎠

∑

∑

2.3 Choice of methodology for empirical analysis

Under some conditions of large sample, similar population distribution, and so on, the

alternative algorithms listed above are expected to provide similar result. However, in practice, the results seem to be sensitive to the matching estimators selected (Heckman et al., 1997). It should be clear that there is no winner for all situations and the choice of the estimator crucially depends on the situation at hand, especially data structure (Zhao, 2004). Few comparative researches exist which deals with the issue of choice among matching algorithms. Caliendo and Kopenig (2005) reported that trade-off relation exists among estimators in terms of efficiency and bias, and generally the more untreated observations are used to make untreated groups for treated groups, the estimator become more efficient in terms of variance but bias becomes larger. Frölich (2004) reported that, one to one nearest neighbor matching is obviously inefficient and suggested use of kernel matching (Heckman et al., 1997) or ridge matching which was first suggested in the same paper. Frölich in his work conducted Monte-Carlo simulation varying untreated to treated ratio and the linearity of distribution between propensity score and actual selection to the program. The ridge matching dominated in terms of efficiency measured by variance in most of cases. However, when the untreated-to- treated ratio was high (more than four times), the kernel matching often resulted in the best estimation of the results.

Testing the statistical significance of treatment effects and computing their standard errors is not straightforward procedure to do. The simple, but incorrect way to do it might be doing nearest neighbor matching with or without replacement and then to take mean differences between the outcomes in the treatment sample and the matched comparison group sample, using the usual t-test for the variance of a difference in means. The problem is that the estimated variance of the treatment effect should also include the variance due to the estimation of propensity score and the common support imputation. Smith (2000) reported that, in the case of nearest neighbor matching with one nearest neighbor, treating the matched observation as mentioned above will understate the standard errors. The usual method employed in practice is bootstrapping, though no published work demonstrates the validity of the bootstrap for matching.

Recently, Abadie and Imbens (2006) showed that bootstrapping is not valid for non- parametric matching estimators as nearest neighbor matching and radius matching due to the lack of smoothness, although their criticism is not applied to smooth non-

parametric regression methods as kernel matching (Todd, 2006).

Based on the literatures on the selection and implementation of matching estimator that we investigated above, we decided to analyze the effect of credit guarantee with kernel matching which showed good precision when the untreated-to-treated ratio is high (in our cases, it is about ten) and can avoid the criticism expressed by Abadie and Imbens (2006), which pointed out the problem of non-smoothness in calculating standard errors with bootstrapping. To show the robustness of the estimation, we also conducted the same analyses with nearest neighbor matching with replacement and radius matching differing the radius of caliper for comparison. The nearest neighbor matching with replacement is known to have the lowest bias but efficiency is lacking and radius matching has good properties of over sampling mentioned above, but on the other hand it avoids the risk of using inappropriate matches.

3. Data and Variables

3.1 The data

The data used in this study was obtained from the unpublished plant-level data assembled from the Annual Report on Mining and Manufacturing Survey in Korea. The data covers all plants with five or more employees in 580 manufacturing industries at the KSIC (Korean Standard Industrial Classification) five-digit level. It was an unbalanced panel data with approximately 95,000 to 109,000 plants for each year from 2000 to 2003, which covers the post economic crisis period. We identified all manufacturing firms which received credit guarantees supplied by KOTEC and KCGF during the period from 2001 to 2002. During the whole process of research, the actual names of firms were kept confidential. Public guarantee funds distributed to the manufacturing industry covers 32 to 35 percent of total credit guarantee balance throughout 2001 to 2002, according to the annual report form guarantee funds. The entry and exit of plants are identified based on the plants appearance and disappearance in the survey data. Entry and exit of plants due to spin-off, split, merger, and acquisition could not be identified with available plant level data base. So, further analyses were

conducted with the firms which do not have multi-plants. This is to avoid confusion between firm turnovers and behaviors of multi-plant firms, or between firm level analyses and plant level analyses. Among firms which received credit guarantee from KCGF, 5.3% of the firms are classified to have multi-plant, and the corresponding was 4.4% for KOTEC. The multi-plant firms mainly consisted of LSEs, so the mean level of output and number of employee for the entire firm population decreased significantly as a result of removal of multi-plant firms.

With the data set we had, we created two balanced panels. In order to measure differences in performance of guaranteed and non-guaranteed firms, we excluded firms which did not existed for four consecutive years (2000-2003) and computed the difference of growth in performance of firms which received guarantee in 2001 and 2002. The number of observation was 42,213 firms. To investigate the effect of credit guarantee on survival, we excluded firms which did not existed for three consecutive years (2000-2002) and observed whether the firms survived in 2003. In this case, the number of observation was 48,540. As the description of data in Table 3 indicates, some firms received credit guarantee from both KCGF and KOTEC, which are indicated as BOTH⁷. We intend to observe comparative performance of the two institutions and investigate the substitutability or complementarities between them.

3.2 Definition of variables

We evaluated the effect of credit guarantees by observing various aspects of a firms operation concerning changes in firm status and performances. The outcome variables considered are: TFP growth, growth in employment, sales, wage level and investment intensity, and the change in R&D status.

The variables as growth in sales and number of employee will indicate the overall growth of the firm size. The growth in wage level will represent improvement in skill or quality of employees. Variables indicating changes in R&D status and intensity in investment in fixed capital are introduced to observe if guarantee funds are used to enhance future productivity and expanded production facility. The growth of productivity might be an ultimate goal of such government support policy. These

outcomes were related to the variable used in estimating propensity scores but not exactly same, since the firm status changes and performances are calculated by comparing data in 2000 and 2003. We also investigated the effect of credit guarantee on the survival of firms. Here, the dataset are slightly different from the one used above to see the differences in outcomes, as explained in section 3.1. For the firms existed through 2000 to 2002, we gave zero value to the firms exited in 2003 and one to the firms survived in 2003.

The chosen variables to be used to estimate propensity score should be related to the outcomes as well as participation decision based on economic theory and previous empirical findings (Caliendo and Kopeinig, 2005). Among the variables in the Survey, we utilized sales, fixed capital, number of laborer, investment intensity in fixed capital, age of firm, and dummy variable for existence of R&D expenditure. These variables will surely affect their own growth rate and survival (Oh et al., 2006). The definition of variables is explained in the Appendix A. We also added four dummy variables according to the management structure of firms. Industrial sector was divided by four categories based on OECD international standard industrial classification (2001) and Kim and Lee (2002). Because the survey do not contain information related to profitability or productivity of firms, we calculated total factor productivity (TFP) of each firms using chained-multilateral index developed in Good (1985) and applied by Aw et al. (2001). The methodology adopted is explained in the Appendix B.

4. Estimation Results

The characteristics of supported and non-supported groups, before they received guarantees, are different as shown in Table 3, which implicates that we need sophisticated matching technique. The table shows the data in 2000 for non-supported firms and firms which received credit guarantee from one of the two guarantee funds or both denoted as KOTEC, KCGF and BOTH during 2001 and 2002. The relative size of supported firms is significantly larger than that of non-supported firms in terms of sales and number of workers, but their fixed capital does not show significant difference, except for firms which received guarantees from both institutions. The firm size of

KOTEC and KCGF supported firms seems to be similar but the size of firm receiving guarantee from BOTH institutions was much larger. However the firms which received guarantee from KOTEC and from BOTH seem to have lower TFP level then non- supported firms which suggest that the guarantee fund was not provided such that the decision being affected by the level of productivity. The wage level of guaranteed firms was higher than the non-guaranteed firms and their investment intensity was slightly higher only for KOTEC firms. However the ratio of firms investing in fixed assets is higher for firms which receive guarantee. Younger firms are more guaranteed by KOTEC while KCGF focused on slightly older firms. This may come from the fact that the aim of KOTEC was to fund technology oriented newly founded firms. The guaranteed firms are involved in R&D more aggressively than non-guaranteed firms but the effect was higher for KOTEC firms. When we look at the management structure, firms owned by a company corporation was much more involved in the guarantee funds and when we look at the industries, firms in high technology industrial sectors received guarantees more frequently.

The descriptive statistics of outcome variables before matching are presented in Table 4 and 5. The performance variables and survival rate were also statistically different among the two groups before matching. However, these differences should not be taken at face value due to problem of selection bias outlined earlier, which requires us to apply advanced matching technique.

4.1 Estimation of propensity score and balancing test

The estimation of propensity score was conducted by applying the procedure explained in Section 2. As we are dealing with two populations of firms and three alternative treatment forms (KOTEC, KCGF and BOTH), we conducted probit estimation for six cases and the variables included were slightly different for all six cases⁸. As in Dehejia (2005), different specifications for variables are needed to be included in probit estimation for each combination of untreated and treated groups. Estimation of the probit model was conducted by using population of non-supported firms and each one of supported firms by KOTEC, KCGF and BOTH. It is to be noted that each time, the

untreated groups are only made up of non-supported firms, and hence the number of observation in regression also differ by the size of the treated group.

When we applied the balancing test described in Section 2, almost in all cases, the means were equal at the 1% significant level (0.5% for few dummy variables), and none of the covariates systematically failed the mean equality test in all the blocks. While this is a high level of significance, implying differences would have to be quite large in order to facilitate failing the balancing test, this is justified for two reasons (de Boer, 2003; Becker and Ichino, 2002). The first is that the observations involved tend to be large (>1000) increasing the likelihood of insubstantial differences with statistically significant difference. The second reason is that the test is applied to each variable, which means that, if the balancing test is strictly adhered to, it would fall if only one of these differences was significant. There is an increased statistical probability of this occurring as the number of variables increase⁹.

The results are presented in Table 6 and 7. Results in Table 6 was estimated to investigate the performance difference during 2000-2003 while those in Table 7 was estimated to investigate the difference in survival rate in 2003 for firms survived during 2000-2002. Since the parameters in Table 6 and 7 showed similar results, we will concentrate on interpretation of Table 6. When we look at Table 6, the parameter for sales, number of employees and age showed positive signs but all squared terms showed negative signs, which implies that there exist some optimal levels for these variables.

Firms with less fixed capital received credit guarantee, since credit guarantee funds were aimed to help SMEs with small collateral. R&D Dummy was positive and significant for assignment to KOTEC and BOTH but insignificant for KCGF, which implies KCGF funds was not very sensitive to R&D status of firms to be supported.

Firms taking the form of company corporation which belong to technologically advanced industrial sectors tended to receive guarantee funds with a higher probability.

However, we could observe significant and negative parameter for TFP which implies that it is hard to find any evidence that productivity of firms was taken into account in the selection procedure. The same pattern was also found for the descriptive statistics reported in Table 3.

4.2 Estimation of treatment effect

To compute ATT accurately, one should match the treated and untreated groups (supported and non-supported firms) precisely on the basis of the propensity score. In practice, it is never possible to match the scores precisely and therefore in this study, three alternative matching methods of nearest neighbor matching (NNM), radius and kernel matching methods were used and compared. Here the radius matching estimator was conducted with different radius (0.00005 and 0.00001). It should be noted that all the analyses were based on implementation of common support, so that the distribution of treated and untreated units were located in the same domain. However, only a few observations were discarded, and given the large samples, the number of excluded observations is relatively small. Standard errors for treatment effects for all cases were calculated by the bootstrapping method by using 200 replications.

The effect of credit guarantee provided by the two institutions in terms of performance (growth in TFP, number of employees, sales, R&D status, investment intensity and wage level) are presented in Table 8.a, 8.b and 8.c. In these tables we report ATT for firms guaranteed by KOTEC, KCGF or by BOTH institutions, respectively.

When we look at the ATT for KOTEC firms in Table 8.a, we can see the effect of KOTEC guarantee on growth of sales, number of employees and wage level is large and statistically significant. The difference of employee growth rate was about 8.5-9.4 percent and the difference of sales growth rate for guaranteed firms and non guaranteed firms was about 26.7-28.8 percent. ATT for wage level was estimated to be in the interval of 7.2 to 8.4%. Changes in R&D status were positive but only significant for NNM and kernel matching methods. However these changes in R&D was also negative for KOTEC firms, but compared to non-supported firms, the rate of decrease was lower.

Difference in growth in the ratio of investment to sales was found to be negative and significant for one radius and kernel matching. It may result from the rapid growth of sales of KOTEC firms. It might also partly reflect the fact that KOTEC fund did not effect positively firms’ investment in fixed capital. ATT for TFP growth was insignificant for all matching estimators.

When we look at the ATT for KCGF firms in Table 8.b, we can find significant effects

of credit guarantee in terms of growth in number of employees, sales, wage level.

However, ATT for TFP growth, changes in R&D status and investment intensity were found to be insignificant. Difference in growth of employee was 5.5-6.2 percent, sales 16.3-20.3 percent and wage level 3.3-4.2 percent, ignoring the insignificant ATT from NNM for wage level. These growth rates are slightly lower compared to those of KOTEC firms. Additionally, in the case of KCGF firms, changes in R&D status turned to insignificant.

Comparing the ATT for KOTEC firms and KCGF firms, it is certain that differences in growth rate for sales, employee and wage level is higher for KOTEC firms. As can be seen in Table 3, KOTEC firms consist of firms that are younger and conduct more R&D compared to KCGF firms. In Table 6, we can observe that the assignment to KOTEC fund was highly influenced by R&D status of firms. Becchetti and Trovato (2002) in their analysis on Italian SMEs argued that, small surviving firms are characterized to have a higher growth potential than average firms and are not fully utilizing their growth potential due to the scarce availability of external finance. The KOTEC funds, which mainly aimed to provide financial aid to technology oriented start-ups, may have helped to reduce the scarcity of external finance resulting in a high growth rate in their size in terms of sales and employee, as well as maintaining their R&D status. Another explanation for a higher performance of KOTEC firms can be found from Table 10. As Table 10 indicates, an average amount of guarantees given to an average KOTEC guaranteed firm was 0.185 to 0.2 billion KRW while it was only 0.086 and 0.096 for an average KCGF guaranteed firm from 2001 to 2002.

Investment in R&D is very costly decisions, which requires hiring more skilled employees and involves investment in research facilities. So there might be some level of threshold in investment amount to start up R&D. If certain amount of investment is collected to allow firms to overcome fixed R&D startup cost, then the firm will start to invest or maintain their investment amount on R&D (David et al., 2000). It seems probable that the guarantee amount for KCGF firms had been insufficient for firms to exceed the limit of fixed R&D startup cost and the guarantee funds and investment were mainly used as marketing cost to increase their sales, hiring more skilled employees and to increase their wage level, hence also to promote welfare of employees. However for

KOTEC firms, though their investment intensity decreased, they managed to maintain their spending on R&D. However, the returns from R&D are still not realized in terms of increase TFP growth rate.

When we look at the ATT for firms receiving guarantee from both of the institutions in Table 8.c, we can observe similar trend for growth in employees, sales, and wage level, which reached 4.4-8.1 percent, 25.4-32.6 percent and 8.1-10.7 percent, respectively except for the insignificant wage effect for the case of NNM. ATT for changes in R&D status turned out to be positive by 4.5–5.7 percent. ATT for TFP growth was positive, while investment intensity growth was insignificant in all cases. The amount of ATT in this BOTH case seems to be larger or similar to KOTEC firms, but less than the simple sum of ATTs from KOTEC and KCGF. Bearing in mind that these BOTH firms were consisted of much larger firm compared to KOTEC firms and KCGF firms as shown in Table 3, (the sales of BOTH firms was 1.74 times larger than KOTEC firms and 1.66 times larger than KCGF firms) not the growth rate but the absolute increase might be bigger. According to Table 10, the average amount of guarantee given to BOTH firms was from 0.246 to 0.267 billion KRW during 2001 to 2002, which is as large as the sum of an average KOTEC guaranteed and average KCGF guaranteed firm. With these large amounts of credit guarantees coming from overlapped support and followed investment from private sector, it seems that BOTH firms succeeded to maintain or to increase their R&D, which finally resulted in the growth in TFP.

The effects of credit guarantee of two institutions in terms of survival are presented in Tables 9.a, 9.b and 9.c, for the case of KOTEC, KCGF, and BOTH, respectively.

The differences of survival rate 1.7-2.5 percent for KOTEC, 2.4-2.9 percent for KCGF and 3.2–3.9 percent for BOTH. Not surprisingly, the firms guaranteed from both of institutions showed highest ATT on survival and the credit guarantee showed a positive effect for survival in all cases. Here, though effects on performance were superior for KOTEC firms compared to KCGF firms, the ATT on survival of KCGF firms is better than or as good as KOTEC firms. The aim of KCGF institutions was to provide firms under a temporary credit crunch with cheap credit, hence to tide over the situation. It seems that in terms of survival, KCGF worked well and in closer correspondence with the mission of the guarantee policy.

5. Summary and Discussion of Results

This research evaluates the impact of public policy, i.e. credit guarantee policy, for recipient firms by controlling the selection bias with the up-to-date propensity score matching technique.

The results are based on investigation of the differences in the performance indicators from 2000 to 2003, for the firms receiving credit guarantee in 2001 and 2002. The empirical analysis for the Korean credit guarantee policy showed that the policy affected positively the growth of sales, employment, wage level as well as survival rate of supported firms. A positive effect is interpreted as the guarantee fund has helped firms to increase or maintain their size in terms of sales and employment and to hire more skilled employees or it has helped to promote welfare of the employees.

The size of the effects were different for the two credit guarantee institutions, however it was more pronounced when the firm received guarantee from both of institutions.

KOTEC firms showed changes in R&D status and firms receiving guarantee from BOTH institutions showed positive effects on their TFP growth rates. Considering the amount of guarantee in each case, it seems that certain level of guarantee fund given to a firm would be needed to stimulate investment in R&D, which will eventually result in growth in productivity. The credit guarantee funds also prevented firms from bankruptcies by providing financial aid.

However, we could not find evidences that the selection of firms to receive guarantees were affected by their productivity considerations. The size, age, management structure and industrial sectors of firms seemed to influence more on the selection firms to be guaranteed.

We also could not find sufficient evidences that the funds have affected positively TFP growth, R&D and investment intensity. The criticism of credit guarantee policies was mainly about its lack of effective selection mechanism, interference in the market mechanism and its disturbing impacts on the restructuring plans which resulted in overcapacity of SME sectors in the aftermath of the financial crisis (IMF, 2005; Kang, 2005). The results obtained from our analysis are in correspondence with their criticism

and provide quantification of the effects of credit guarantees on various firm performance and survival indicators.

Notes:

1 Chaebol is a Korean corporate form consisting of a pyramid of subsidiary firms operated by a single family line.

2 Korean Won, US Dollar (USD) 1$=1037 KRW, in November 2005.

3 Operation multiple is defined as the outstanding guarantee balance divided by basic funds of the guarantee institution.

4 Also there exist 14 regional credit guarantee institutions founded by local governments, and their aggregated guarantee balance compared to the total guarantee balance provision from all guarantee institutions was 3.1% and 3.7% for year 2001 and 2002, respectively. We could not identify firms supported from regional credit guarantee institutions. However more than half of firms supported by regional credit guarantee institutions were also beneficiaries of KOTEC and KCGF (Lee, 2006).

5 The detailed algorithms of balancing tests are well presented in the appendix of Dehejia and Wahba (2002).

6 It is to be noted that, in this study, we do not consider an alternative stratification and interval matching, which is also noted as blocking and sub classification in Rosenbaum and Rubin (1983), since it is ‘group to group’ matching and different scheme with other matching methodologies which are based on matching of pair of observation.

7 Hereafter the KOTEC firms, KCGF firms and BOTH firms means the firms received credit guarantee from one of KOTEC, KCGF and both of institutions, respectively.

8 Matching with multiple treatments applied in Frölich et al. (2004) could be implemented in this kind of evaluation that there exist more than two institutions working under the same policy. However we are using seven performance variables including survival rate, and comparing all these by institutions will result in too many combinations, so here we concentrated about comparing supported firms with non- supported firms by institutions.

9 For example, assume that the significance level is set to 0.01. Taking ten variables and five intervals of the propensity score would require 50 mutually independent tests of mean differences. For this balancing test, there is a probability of

1 49

50 0.01 0.99× × =0.306 that one of these tests reports significant difference although the balancing test is true.