Alternative measures of forecast errors

Two alternative measures of unexpected earnings for short-term, medium-term, and long-term forecast horizons were used to test the robustness of the results reported earlier. Recall that the Feltham-Ohlson model uses forecast and actual EPS and BVS to estimate abnormal earnings. The associated forecast errors (UEPSST, UEPSMT, UEPSLT) are henceforth referred to as the “model” variables, and were calculated as follows:

UEPSitl = (AEPSit - re ABVit) - (FEPSitl - re FBVitl)

where

UEPSitl = unexpected abnormal earnings per share for firm i at time t with lag l

(l relates to the ST, MT, or LT forecast)

AEPSit = actual earnings per share reported by VL for firm i at time t

ABVit = actual beginning-of-year book value reported by VL for firm i at time t

FEPSit = VL-forecasted earnings for firm i for time t with lag l

FBVit = VL-forecasted beginning-of-year book value for firm i for time t with lag l

re = assumed cost of common equity capital

The first alternative set of forecast error variables is referred to as the “simple” errors. In computing these variables, each firm-year’s book value and any anticipated return on that book value were ignored, and only the firm-year’s earnings per share was used, as follows: SAEPSitl = AEPSit - FEPSitl

where

SAEPSitl = simple abnormal earnings for firm i at time t with lag l

AEPSit = actual earnings per share reported by VL for firm i at time t

FEPSit = VL-forecasted earnings for firm i for time t with lag l

For short-term (i.e., one year), medium-term (i.e., two years), and long-term (i.e., three or more years) forecast horizons, the VLIS firm-specific earnings forecasts are compared to the EPS reported for each firm-year in the relevant subsequent years. This set of simple forecast error variables (SAEPSST, SAEPSMT, SAEPSLT) still includes an element of the forecast ability of trained professionals, but the variables do not explicitly consider the effects of book values on any so-called “normal” level of earnings.

The second set of alternative forecast error variables, or the "naïve" errors, ignores any potential biases that analysts may incorporate into their forecasts. A "naïve" forecast error is

the difference between the EPS reported for the current year and the EPS reported for the related subsequent year (i.e., one, two, and three years out), as follows:

NAEPSitl = AEPSit - AEPSi(t-1)

where

NAEPSitl = naïve abnormal earnings for firm i at time t with lag l

AEPSit = actual earnings per share reported by VL for firm i at time t

AEPSi(t-1) = actual earnings per share reported by VL for firm i at time t-1

Calculating these errors yielded three additional variables (NAEPSST, NAEPSMT, NAEPSLT) for each firm-year.

Table 4.10 reports comparisons of the mean errors among the C, M, and L firm-years to find any significant differences using the three sets of earnings forecast error variables. The overall results of these additional tests lead to several generalized conclusions. First, and probably most importantly, the mean differences in forecast errors are always smallest for C firm-years and largest for L firm-years, with M firm-years always falling somewhere in the middle. These differences are nearly always strongly significant for comparisons involving L firm-years. The differences are almost always insignificant, however, for comparisons of C and M firms.

These results lead to the strong conclusion that analysts do considerably worse at forecasting earnings for firms using liberal portfolios of accounting methods. This poorer forecast ability exists over short-term, medium-term, and long-term forecast horizons and does not appear to be driven by the definition of the forecast error. It may also be true that analysts do a better job of forecasting for firms using a conservative portfolio of accounting methods, but the differences shown in these results are not statistically significant (although they are consistent).

The second general result from these tests relates to the relative ability of analysts to predict earnings. Although not based on a statistical comparison, it is true that the mean forecast errors are smallest for the three model variables and largest for the three Naïve variables. Two exceptions to this general conclusion bear comment. First, the Naïve medium-term variable (NAEPSMT) for M firm-years “beats” the related simple variable (SAEPSMT); that is, the Naïve mean difference is smaller than the Simple mean difference. This exception is probably not an important finding – it is an anomaly related to firm-years

not using either of the “extreme” portfolios of accounting methods, and it does not occur in short-term or long-term forecast horizons.

The second exception is potentially more embarrassing for the proponents of financial forecasting. In these results, the Naïve forecast errors are considerably smaller than the Simple forecast errors for all long-term forecasts. Putting that another way, if one desires a long-range forecast of a firm’s EPS, a “forecast” equal to the current year’s reported earnings (i.e., a random walk) is likely to be more accurate than an analyst’s (here, VLIS) forecast. In support of the industry, however, it is also true that when analysts consider a firm’s book value and a return on the book value in the calculation of unexpected earnings (as shown in the model variables), the analyst’s forecasts are more accurate than the Naïve forecasts.

These results reflect a change in forecast accuracy over time. Brown et al. (1985) compared consensus and individual analysts’ forecast errors to mechanical models of earnings predictions and also summarized prior research on the topic. They report that, in the research reviewed, consensus estimates are consistently more accurate than forecasts from individual analysts, which are consistently more accurate than any of several mechanical, or naïve, models. In the present research, Value Line Investment Survey underperforms the naïve model if EPS forecasts are considered apart from other forecast indicators (i.e., the Simple variables). The analysts at VLIS outperform the naïve model, however, when the VLIS forecast of book values and earnings (i.e., the Feltham-Ohlson model variables) are both considered.

We have seen the third overall result before. Generally, in the short-term, forecast error differences are insignificant. Only for the Naïve short-term forecast error variable (NAEPSST) do we see any statistically significant differences. Those differences are consistent with the other results shown above – forecast errors are larger for firms using liberal accounting methods.

Table 4.10 Summary of differences in EPS forecast errors in C, M, and L Variable C mean C # M mean M # L mean L # Overall F- statistic Overall p-value Model UEPSST $0.189 120 $0.644 751 $0.846 205 1.902 0.15 UEPSMT 0.498 121 0.793 752 1.618 205 24.894 <0.01 UEPSLT 1.200 121 1.420 752 2.116 205 9.811 <0.01 Simple SAEPSST $0.43 113 $0.87 726 $1.10 202 1.723 0.18 SAEPSMT 1.06 120 1.29 739 2.23 203 18.113 <0.01 SAEPSLT 2.82 121 3.00 752 4.00 205 9.38 <0.01 Naïve NAEPSST $0.64 109 $0.89 660 $1.68 163 20.423 <0.01 NAEPSMT 1.11 109 1.19 663 2.56 163 27.604 <0.01 NAEPSLT 1.45 110 1.46 662 2.87 164 23.331 <0.01

Model variables: forecast errors derived from the Feltham and Ohlson model using actual and forecast EPS and BV

Simple variables: forecast errors calculated as the difference between the VLIS forecast EPS and the actual EPS

Naïve variables: forecast errors calculated as the difference between the prior year and current year EPS

Table 4.10 (continued) Pairwise comparisons of EPS forecast errors in C, M, and L

C vs. L M vs. L Variable t statistic p-value t statistic p-value Model

UEPSST N/A N/A N/A N/A

UEPSMT -5.964 <0.01 -6.394 <0.01 UEPSLT -3.677 <0.01 -4.065 <0.01 Simple

SAEPSST N/A N/A N/A N/A

SAEPSMT -4.834 <0.01 -5.628 <0.01 SAEPSLT -3.34 <0.01 -4.124 <0.01 Naïve NAEPSST -5.431 <0.01 -5.854 <0.01 NAEPSMT -5.456 0.01 -7.249 <0.01 NAEPSLT -4.788 0.01 -6.728 <0.01

* N/A means “not applicable” and is included any time the initial comparison showed no overall differences.