This section describes a number of experiments to assess the sensitivity of our results to several issues. These include a possible relation between funds’ alphas and active management, the level of noise in the simulated fund returns, alternative factor models and return smoothing.
7.1. Are the Simulated Returns too Noisy?
Kowsowski et al. (2006) find that the best performing funds have significant positive alphas, but Fama and French (2010) do not find significant performance. One of the differences between the studies is the method of simulating fund returns. Fama and French and the preceding tables resample, under the null hypothesis that the true alphas are zero, from the vector of factors and the actual fund returns minus their estimated alphas. Kowsowski et al. resample from the factor model residuals. Fama and French argue that this approach understates the sampling error by ignoring the sampling variation in the factors. However, the more conservative approach suggested by Fama and French does not account for the fact that the alphas have been estimated with error, because in each simulation trial and for each fund the estimated alphas are treated as constants. In the cross-section, the alphas are random variables subject to estimation error. We conduct an experiment which accounts for the fact that the
estimated alphas contain measurement error. We adjust the simulations to account for estimation error in the alphas.
Instead of subtracting the estimated alpha from a fund in the simulations as if it was constant, we subtract a normally distributed random variable, with mean equal to the point estimate of alpha and standard deviation equal to the heteroskedasticity- consistent standard error of the alpha estimate. The results of this experiment are summarized in the Internet Appendix, Table A.3, Panel E.
good alpha estimate is larger by about 0.05% per month and the bad alpha estimate is smaller by a similar amount. The power and confusion parameters are similar. The fraction of zero-alpha hedge funds is now estimated to be larger than zero, at 0.02% to 12.4% depending on the test size, but remains within one standard deviation of zero. The BSW estimates are similar to the previous results.
7.2 Are the Alphas Correlated with Active Management?
Studies suggest that more active funds have larger alphas (e.g. Cremers and Petajisto (2009), Titman and Tiu (2011), Amihud and Goyenko (2013) and Ferson and Mo (2015)). In particular, funds with lower market model regression R-squares are found to have larger alphas. We examine the correlations between the factor model R- squares and the estimated alphas and find a correlation of -0.015 in the mutual fund sample and -0.111 in the hedge fund sample. The mixtures of distributions simulated above do not accommodate this relation.
We modify the simulations to allow a relation in the cross-section of funds’ returns, between alpha and a fund’s active management measured by the R-squares in the factor model regressions that deliver the alphas. We sort the funds in the original data by their factor model R-squares, group them into three groups with the group sizes determined by the -fractions at any point in the simulations, and assign the good alpha first to the low R-square group and the bad alpha first to the high R-square group. The simulations draw the vector of factors and fund returns, so they preserve the relation between the random part of fund returns and the factors. Thus, this approach builds in a relation between the alpha and active management, measured by the factor model R- squares.
The results from this variation on the simulations are summarized in the Internet appendix, Table A.3, Panel D. We find that this modification also improves the
funds are similar to those in the original design, but the estimate of the bad alpha is smaller, at -0.35% to -0.37%, while in the original design it was about -0.1% per month. Fewer fractions of the hedge funds are estimated to have this more pessimistic bad alpha, and as a result the fraction of zero alpha hedge funds is increased to 30%-44%, which is a significant positive fraction when the test size is 10%. The BSW estimates are similar to those in the original design.
This experiment illustrates that when the probability model incorporates an association between the hedge funds’ alphas and the cross-section of the regression R- squares, the best-fitting alpha parameters are further out in the tails, and in particular in the left tail, where it moves about ¼ of a percent to the left. This suggests that the
relatively poor performance of the high R-square hedge funds is the dominant part of the relation between the R-squares and performance. This result may not be surprising, but it does suggest that future work on estimation by simulation might profit from building in associations between other fund characteristics and the performance groups.
7.3 Alternative Alphas
While the Fama and French (1996) three-factor model is less controversial for fund performance evaluation than for asset pricing, it is still worth asking if the results are sensitive to the use of different models for alpha. We examine two alternatives for mutual funds: one with fewer factors and one with more factors. The first is the Capital Asset Pricing Model (Sharpe, 1964), with a single market factor and the second is the Carhart (1997) model, which adds a momentum factor. For the hedge fund sample we use the multifactor model of Fung and Hsieh (2001, 2004) in the main tables and try the Fama and French three factor model as a robustness check. Results using the
7.4 Return Smoothing
Return smoothing tends to reduce the standard errors of fund returns, can
increase alphas by lowering the estimated betas, and may be important for hedge funds (e.g. Asness et al., 2000). We address return smoothing by replacing the estimates of the betas and alphas with Scholes-Williams (1977) adjusted estimates. Here we include the current and lagged values of the factors in the performance regressions and the beta is the sum of the coefficients on the lagged and contemporaneous factor. The first thing to check is the impact on the t-ratios for alpha in the original data, since the simulations assume the returns are independent over time and will not be affected except by the additional noise from having more regressors. The results are presented in the Internet Appendix, Table A.1. The effect of using the Scholes-Williams betas on the alpha t-ratio distributions is small, so we do not further investigate the issue.