We are evaluating models with different numbers of factors (m = 1,2,3), weekly and biweekly prediction horizon h = 5,10, and with different numbers of assets (n = 5,10,15,22). Unfortunately, there is no agreement on how to evaluate whether the model prediction is good enough. We decided to use two alternative approaches. The first approach based on several statistical measures of constructed portfolio returns, i.e., it is a standardized approach based on the Conditional Value at Risk paradigm proposed by Engle and Manganelli (1999). The other approach is based on the ex- pected utility approach, and closely follows Bandi, Russell and Zhu (2005). We apply these two approaches to three portfolios: equally-weighted, minimum variance, and value-weighted portfolios.
4.5.1
Value-at-Risk approaches
Following Engle (2001), we use two methods to evaluate estimation performance. First method tests whether standardized by implied covariance portfolio variances is equal to one. Second is unconditional and conditional Value-at-Risk performance. We report results for DCC and FARCH model for different time horizons and different number of stocks. For this purpose we use three benchmark portfolios: minimum variance, equally
4Testing, not reported here, suggests that increasing this parameter beyond 50 does not change
results, which coincides with findings in Ghysels, Santa-Clara and Valkanov (2004a).
weighted, and value-weighted. The minimum variance portfolio is the most interesting in terms of correctness of the model specification. Since the weights of the portfolio depend on the estimated conditional volatility, a misspecified model will produce the worst results. Its time-varying weights are defined as follows:
wt=
Ht−+1h,t ι0H−1
t+h,tι
(4.9)
where Ht is one-step ahead prediction of volatility, conditional on the information set available to the researcher in period t −1, and ι is the vector of ones. Competitor number 2, the value-weighted portfolio, is constructed using the following formula
wt=
wt−1 ∗(1 +rt+h,t)
w0
t−1(1 +rt+h,t)
(4.10)
In this equation rt is for the k×1 asset returns, and ∗ denotes Hadamard product5. Initial weights are w0 = k−1ι. The last portfolio to consider is the equally-weighted
portfolio with the weights w0 =k−1ι.
If the conditional covariance is correctly specified, variance of the portfolio return in the period t+h, t will be st+h,t=wt0+h,tHt+h,twt+h,t, and
(bT /hc −1)ˆs2 = bXT /hc t=0 (r0 (t+1)h,thw(t+1)h,th)2 s(t+1)h,th ∼ χ2(bT /hc −1)
Under the null, ˆs2 should be centered around 1. To test this hypothesis, we construct
a symmetric confidence interval with probability α/2 in each tail. Too small ˆs2 will
indicate that there is some negative correlation in the standardized random variables. Too big ˆs2 will point to underestimation of serial correlation. Results are shown in
a Table 4.1 for h = 5 and in a Table 4.2 for h = 10. On average, ˆs2 obtained from
5If A=
{aij}and B={bij} are two matrices of the same dimensionn×m,A∗B =C is ann×m
matrix with elements{cij}={aijbij}.
Factor MIDAS models is closer to one than ˆs2 from DCC. But the difference is not
statistically significant. With an increase in the number of factors in Factor MIDAS models, ˆs2 becomes closer to 1.
As the number of assets increases, standard deviation of the standardized minimum variance portfolio increases for almost all Factor MIDAS, which indicates that diagonal Factor MIDAS with small number of factors is inappropriate for large portfolios because it does not capture all conditional variations of the model. The same trend is shown by a DCC minimum variance portfolio: ˆs2 tends to converge to 1. Standard deviations
of the other portfolios do not change as the number of assets increases. The Factor MIDAS and DCC consistently underestimate the standard deviation of the adjusted value-weighted portfolio. In addition, DCC consistently underestimates the equally- weighted portfolio.
The second measure of the empirical validity of the model is the HIT test, intro- duced by Engle and Manganelli (1999). This test is designed to evaluate Value-at-Risk performance of the models. The idea is the following: a series of HITt+h,t are defined as a binary random variable I{rt+h,t<V aR(q)} with rt+h,t - return on the portfolio and q
- quantile of the interest. Under the assumption of correct specification of Value-at- Risk, HITt+h,t should be independent of all information available upon period t −1 and should have mean q. They suggest running an artificial regression that could test the mean and the independence of this binary random variable jointly.
HITt+h,t−q=δ0+
r
X
i=1
δiHITt−(i−1)h,t−ih+δr+1V aRt+h,t+νt (4.11)
Under the null, all coefficients δ of this regression should be equal to zero, since
V aRt+h,t(q) depends only on the predicted portfolio variance and thus enters the t−1
information set. Normality assumption would imply thatV aR(q)t+h,t=−zqσˆt+h,t. For
example, ifd=.05, thenV aRt+h,t(.05) =−1.65ˆσt+h,t. However, using the Jarque-Bera
test at the 5% level, all individual stocks in the sample reject the null about normal- ity of returns. Thus, in the auxiliary regression, only the independence of HITt+h,t is tested, which is equivalent to the test that δi = 0,∀ i >0. Tables 4.3 and 4.4 present unconditional means for the 5% HIT variables. As the number of assets increases,
HIT variable estimated by Factor MIDAS minimum variance portfolio demonstrates robustness for five- and ten- day horizon. DCC results show an increase in the realized number of hits for the five-day horizon. TheHIT variable generated by MIDAS value- weighted portfolio decreases significantly on the ten-day horizon. In general, all models demonstrate larger deviations from the expected mean under the null with an increase in the number of assets. Table 4.5 provides the results from the auxiliary regression 4.11 for five- and ten-day time horizons and for minimum variance and equally-weighted portfolios. In absolute terms, for five to fifteen assets, MIDAS hits demonstrate strong support for acceptingH0 thatHITt+h,tdoes not depend on the information set available in period t−1. Evidence for twenty two assets is not that compelling. Also, there is no strong support for the fact that asymmetric models perform better than symmetric ones within this setup. DCC models do worse, on average, in comparison to MIDAS models. We can rejectH0 for five- and ten day horizon and twenty two assets portfolios.
Also, we can acceptH0 for a ten-day horizon. DCC equally weighted-portfolio only for
the case of a ten asset portfolio.