Evaluating Forecast Accuracy - What Do We Know about Recent Exchange Rate Models? In-Sample Fit

The Diebold-Mariano statistics (Diebold and Mariano, 1995) are used to evaluate the forecast performance of the different model specifications relative to that of the naive random walk.

Given the exchange rate series x and the forecast series _t y , the loss function L for the _t mean square error is defined as:

(A1) L(y_t)=(y_t −x_t)².

Testing whether the performance of the forecast series is different from that of the naive random walk forecast z , it is equivalent to testing whether the population mean of the loss differential series _t

d is zero. The loss differential is defined as t

(A2) d_t =L(y_t)−L(z_t).

Under the assumptions of covariance stationarity and short-memory for d , the large-sample _t statistic for the null of equal forecast performance is distributed as a standard normal, and can be expressed as observations. Different lag-window specifications can be applied, such as the Barlett or the quadratic spectral kernels, in combination with a data-dependent lag-selection procedure (Andrews, 1991).

For the direction-of-change statistic, the loss differential series is defined as follows: d _t takes a value of one if the forecast series correctly predicts the direction of change, otherwise it will take a value of zero. Hence, a value of d significantly larger than 0.5 indicates that the forecast has the ability to predict the direction of change; on the other hand, if the statistic is significantly less than 0.5, the forecast tends to give the wrong direction of change. In large samples, the studentized version of the test statistic,

(A4)

T d

/ 25 . 0

5 .

−0

, is distributed as a standard Normal.

Table 1.A: Full-Sample Estimates of Sticky-Price Model, in Levels

sign BP/$ Can$/$ DM/$ SF/$ Yen/$

Coint (asy.) 1,1 3,1 0,0 1,1 1,1

Coint (f.s.) 0,0 1,0 0,0 0,0 0,1

money [+] -2.89* 1.10* 2.14* 3.61* 1.29

(1.01) (0.25) (0.74) (0.74) (0.96)

income [-] 1.64 9.70* 0.93 -1.10 0.77

(3.94) (1.87) (1.87) (1.72) (1.97) interest rate [-] -19.49* -6.44* -5.86 2.09 -17.11*

(4.01) (3.27) (4.14) (5.73) (4.72) inflation rate [+] -7.11 10.74* 24.29* 40.96* 26.56*

(4.60) (3.11) (4.27) (6.79) (4.03)

Notes: Long-run cointegrating estimates from Johansen procedure (standard errors in parentheses), where the VECM includes 2 lags of first differences. The rows “coint” indicate the number of cointegrating vectors implied by the trace and maximal eigenvalue statistics, using the 5% marginal significance level. “asy.” denotes asymptotic critical values and “f.s.” denotes finite sample critical values of Cheung and Lai (1993) are used. “Sign” indicates coefficient sign implied by theoretical model. * indicates significantly different from zero at the 5% marginal significance level. Estimates for DM include shift and impulse dummies for German monetary and economic unification.

Table 1.B: Full-Sample Estimates of Sticky-Price Model, in First Differences

sign BP/$ Can$/$ DM/$ SF/$ Yen/$

money [+] -0.21 -0.00 0.16 -0.02 0.44

(0.12) (0.06) (0.22) (0.14) (0.24)

income [-] -2.02* -0.48 -0.51 0.59 -0.00

(0.42) (0.29) (0.43) (0.52) (0.39) interest rate [-] 0.83* -0.42* -0.91* -0.82* -0.28

(0.41) (0.10) (0.45) (0.37) (0.33) inflation rate [+] -0.15 -0.07 1.26 1.29 0.32

(0.48) (0.20) (1.09) (0.81) (0.44)

Notes: OLS estimates (Newey-West standard errors in parentheses, truncation lag = 4). * indicates significantly different from zero at the 5% marginal significance level. Estimates for DM include impulse dummies for German monetary and economic unification.

Table 2.A: Full-Sample Estimates of Productivity Model, in Levels

Notes: Long-run cointegrating estimates from Johansen procedure (standard errors in parentheses) , where the VECM includes 2 lags of first differences. The rows “coint” indicate the number of cointegrating vectors implied by the trace and maximal eigenvalue statistics, using the 5% marginal significance level. “asy.” denotes asymptotic critical values and “f.s.” denotes finite sample critical values of Cheung and Lai (1993) are used. “Sign” indicates coefficient sign implied by theoretical model. * indicates significantly different from zero at the 5% marginal significance level. Estimates for DM include shift and impulse dummies for German monetary and economic unification.

Table 2.B: Full-Sample Estimates of Productivity Model, in First Differences

sign BP/$ Can$/$ DM/$ SF/$ Yen/$

Table 3.A: Full-Sample Estimates of BEER Model, in Levels

Notes: Long-run cointegrating estimates from Johansen procedure (standard errors in

parentheses), where the VECM includes 2 lags of first differences (4 lags for DM). The rows

“coint” indicate the number of cointegrating vectors implied by the trace and maximal

eigenvalue statistics, using the 5% marginal significance level. “asy.” denotes asymptotic critical values and “f.s.” denotes finite sample critical values of Cheung and Lai (1993) are used. “Sign”

indicates coefficient sign implied by theoretical model. * indicates significantly different from zero at the 5% marginal significance level. Estimates for DM include shift and impulse dummies for German monetary and economic unification.

Table 3.B: Full-Sample Estimates of BEER Model, in First Differences

sign BP/$ Can$/$ DM/$

Table 4: Uncovered Interest Parity Estimates

BP/$ Can$/$ DM/$ SF/$ Yen/$

horizon

3 month -2.19* -0.48* -0.70 -1.28* -2.99*

(1.08) (0.51) (1.09) (1.04) (0.96) Adj R² 0.04 -0.00 -0.01 0.01 0.06

SER 0.21 0.08 0.26 0.29 0.28

1 year -1.42* -0.61* -0.58* -1.05* -2.60*

(0.99) (0.49) (0.66) (0.52) (0.69)

Adj R² 0.06 0.03 0.00 0.04 0.17

SER 0.11 0.04 0.14 0.14 0.13

5 year 0.44 0.24 0.52 -1.18* 1.19

(0.36) (0.47) (0.75) (0.97) (0.38)

Adj R² 0.02 -0.00 0.02 0.04 0.13

SER 0.04 0.02 0.06 0.04 0.05

Notes: OLS estimates (Newey-West standard errors in parentheses, truncation lag = k-1). SER is standard error of regression. * indicates significantly different from unity at the 5% marginal significance level.

Table 5: The MSE Ratios from the Dollar-Based and Yen-Based Exchange Rates

Table 5 (Continued)

Note: The results are based on dollar-based and yen-based exchange rates and their forecasts. Each cell in the Table has two entries. The first one is the MSE ratio (the MSEs of a structural model to the random walk specification). The entry underneath the MSE ratio is the p-value of the hypothesis that the MSEs of the structural and random walk models are the same (Diebold and Mariano, 1995). The notation used in the table is ECM: error correction specification; FD: first-difference specification; S-P: sticky-price model; IRS-P: interest rate parity model; PROD: productivity differential model; and BEER: behavioral equilibrium exchange rate model. The forecasting horizons (in quarters) are listed under the heading “Horizon.” The forecasting period is 1993 Q1 – 2000 Q4. Due to data unavailability, the BEER model was not estimated for the Japanese Yen and Swiss Franc.

Table 6: Direction-of-Change Statistics from the Dollar-Based and Yen-Based Exchange Rates

Specification Horizon S-P IRP PROD BEER S-P IRP PROD

Table 6 (Continued)

Note: Table 2 reports the proportion of forecasts that correctly predict the direction of the dollar-based and yen-based exchange rate movements. Underneath each direction-of-change statistic, the p-values for the hypothesis that the reported proportion is significantly different from ½ is listed. When the statistic is significantly larger than ½, the forecast is said to have the ability to predict the direct of change. If the statistic is significantly less than 1/2, the forecast tends to give the wrong direction of change. The notation used in the table is ECM: error correction specification; FD: first-difference specification; S-P: sticky-price model; IRP: interest rate parity model; PROD: productivity differential model; and BEER: behavioral equilibrium exchange rate model. The forecasting horizons (in quarters) are listed under the heading “Horizon.” The forecasting period is 1993 Q1 – 2000 Q4. Due to data unavailability, the BEER model was not estimated for the Japanese Yen and Swiss Franc.

Table 7: Cointegration Between Exchange Rates and their Forecasts

Note: The table reports the Johansen maximum eigenvalue statistic for the null hypothesis that a dollar-based (or a yen-based) exchange rate and its forecast are no cointegrated. "*" indicates 10%

marginal significance level. Tests for the null of one cointegrating vector were also conducted but in all cases the null was not rejected. The notation used in the table is ECM: error correction

specification; FD: first-difference specification; S-P: sticky-price model; IRP: interest rate parity model; PROD: productivity differential model; and BEER: behavioral equilibrium exchange rate model. The forecasting horizons (in quarters) are listed under the heading “Horizon.” The forecasting period is 1993 Q1 – 2000 Q4. A "--" indicates the statistics are not generated due to unavailability of data.

Table 8: Results of the (1,-1) Restriction Test

Table 8 (Continued)

Note: The likelihood ratio test statistic for the restriction of (1, -1) on the cointegrating vector and its p-value are reported. The test is only applied to the cointegration cases present in Table 3. The notation used in the table is ECM: error correction specification; FD: first-difference specification; S-P: sticky-price model; IRP: interest rate parity model; PROD: productivity differential model; and BEER:

behavioral equilibrium exchange rate model. The forecasting horizons (in quarters) are listed under the heading “Horizon.” The forecasting period is 1993 Q1 – 2000 Q4.

1.2 1.6 2.0 2.4 2.8 3.2

1975 1980 1985 1990 1995 2000

Out-of-sample

Figure 1: German mark - US dollar exchange rate.

80 120 160 200 240 280 320

1975 1980 1985 1990 1995 2000

Out-of-sample

Figure 2: Japanese yen - US dollar exchange rate.

In document What Do We Know about Recent Exchange Rate Models? In-Sample Fit and Out-of-Sample Performance Evaluated (Page 26-41)