Outlier Detection and Bootstrapping Procedures

A data robustness check is appropriate considering the fact that DEA is a non-parametric, deterministic, linear programming-based frontier approach. The results may be severely biased by the presence of extreme observations, or outliers in the data. Outliers are atypical observations and some of them might arise from measurement errors and should be corrected or deleted from the data sample because these observations may disproportionally, perhaps misleadingly, influence the evaluation of other companies’ performance. Generally, two streams of approaches dealing with those observations have been proposed. The first approach is to identify outliers in the data and correct them (if possible) or delete them if they are based on corrupted data; a number of specific techniques for detecting outliers have been developed (e.g., Wilson, 1993). The second approach is based on recently developed robust estimators, which offer additional advantages.

A commonly used statistical method of detecting outliers in the production data set ex ante was developed by Andrews and Pregibon (1978), who derived a statistic for the case of only one output. Wilson (1993) extended the approach for the multiple-output case. Generally, the approach employs an influence function based on a geometric volume spanned by the sample observations and the sensitivity of the volume with respect to deleting singletons, pairs, triplets, etc. from the sample (Simar and Wilson, 2008). In other words, outliers are identified by comparing the geometric volume of a subset (S-L(i)) of the data with the entire data set S where the excluded set L(i) contains i observations. This implies that outliers can be defined by looking at the values of the geometric volume produced with different sets of observation L(i) deleted from the sample. If (S-L(i)) produces small values of the geometric volume, the observations i in L(i) are considered as outliers.

The robust estimators are based on a concept of “partial” frontiers within the DEA calculation procedure in contrast to the “full” frontier that envelops all observations. Basically, “partial” frontiers do not calculate the absolute lowest level of inputs but rather a value lying nearby resulting in less restrictive estimations. Moreover, two concepts of estimating such a “partial” frontier have been suggested: The order-m frontiers have been proposed by Cazals, Florens, and Simar (2002) and follow the idea of comparing the actual input usage of a company with its expected minimum input usage, where the expected minimum is obtained by drawing k samples of m companies producing at least as much output as the company of interest. Thus, m can be seen as a trimming parameter that allows tuning the percentage of points that will lie above the order-m frontier. Order-α quantile frontiers proposed by Aragon, Daouia, and Thomas-Agnan (2005) are analogous to traditional quantile

21

functions, which have been adapted to frontier problems. This approach first fixes the probability (1–α) of observing points above the corresponding order-α frontier and then calculates the frontier itself. These partial frontiers are consistent estimators of the full frontier by allowing the order of the frontier (m or α) to grow with increasing sample size. In finite samples, this theoretical result may be of limited use but these estimators do not envelope all observations, and thus, the frontier is more robust when detecting outliers and extreme value than traditional DEA (Daraio and Simar, 2007, Simar and Wilson, 2008).

Another advantage of the robust non-parametric frontiers is that they do not suffer from the “curse of dimensionality” and “sample size bias”. The first problem describes the circumstance that in order to avoid large variances and wide confidence interval estimates a large quantity of data is needed. The second one refers to biased efficiency estimates that can emerge by comparing the results of different sample sizes. This bias arises because DEA indicates that certain units are fully efficient (the peers). The extent of the bias, which is upward, will vary with the sample size because other efficient companies may be included, which may result in shifting the frontier upwards. This could be quite important for small samples (Bonaccorsi and Daraio, 2004). In other words, the DEA estimators measure efficiency in relation to an estimate of an unobserved true frontier. It is conditional on observed data sets of a specific size resulting from an underlying data-generation process (DGP), the process through which inputs are obtained. It is also conditional on outputs and the input proportions. Associated with this estimation, there is a difficulty in making statistical inference, mainly based on the non-parametric approach with very few assumptions. Recent developments have suggested solutions to enable statistical inference by means of asymptotic results or bootstrapping.22 The use of asymptotic results is potentially appropriate for the calculation of asymptotic bias and variance, as well as asymptotic confidence intervals. However, they remain asymptotic, which may result in problems with small samples (Bonaccorsi and Daraio, 2004).

A useful alternative is the bootstrapping procedure, originally developed by Efron (1979, 1982) and Efron and Tibshirani (1993). The idea behind bootstrapping is that the underlying DGP for a sample of observations is not completely known and the essence of the bootstrap approach is to approximate the sampling distribution of interest by simulating (or mimicking) the DGP. Therefore, the given sample is used to generate a set of bootstrap samples from which the parameters of interest can be calculated. An empirical sampling distribution of the variables of interest is then constructed by means of repeated sampling of the original data series. The first use of frontier models was made by Simar (1992). The normal (or naïve) bootstrap just draws randomly (independently, uniformly, and with replacement) a specific number of observations from the original sample forming the bootstrap sample and then calculates the corresponding DEA efficiency estimates. The construction of a bootstrap sample and the efficiency estimation based on this sample is conducted multiple times providing information about the distribution of the efficiency estimate and the bootstrap bias associated with the estimates of the observed data sample. As the normal bootstrap estimator is not consistent, Simar and

Wilson (1998, 2000a) suggested two approaches to overcome this problem: the smoothed homogenous and the heterogeneous bootstrapping procedure. These approaches resample from a semi-artificial sample generated from a density function for the inefficiency values (that is the same for all values in the homogeneous case) based on a smoothed representation of the observed distribution of efficiency estimates (Simar and Wilson, 2008).23

In our empirical analysis we apply the outlier detection procedure of Wilson (1993) and the homogenous bootstrapping procedure of Simar and Wilson (1998) because of their easy computing. The combination of both methods ensures that the resulting efficiency scores are less likely to be prone to distortions from outliers, extreme values, a finite sample size, and the inherent upwards bias of DEA efficiency estimates.