For this simulation study we used a geoadditive mixed model which contains a smooth spatial function and a random intercept in addition to six nonlinear functions of continuous covariates. For the nonlinear functions of continuous variables we used functions f1 to f6
from the simulation study in section7.1 which are shown in figure 7.1. The functions are indicated by the same numbers in the geoadditive simulation study. The smooth spatial function and the random effect are both shown in figure 7.22. For the spatial effect we used the 309 regions of West-Germany and created a two–dimensional function using the centroids (r1, r2) of the regions as variables. The spatial function is then given by
fspat = sin(r1·r2) + 0.1483,
where r1 is the value of a centroid in east–west direction and r2 its value in north–south
direction. Both variables r1 and r2 had been centered and standardised before. The
function is centered around zero by the value 0.1483. For each region we generated three observations so that we have 729 observations for the geoadditive simulation. Then, we generated a group variableindwith twenty individuals for a random effect. The individuals were randomly assigned to the observations in such a way that there are either 46 or 47 observations per individual. The random effect was created according to a normal distribution with mean zero and a standard deviation of 0.4.
The span between minimum and maximum of these two functions again amounts to 2 (like for the continuous variables) so that all functions have an equally strong influence on the predictor. The predictor takes the form
η=
6
X
j=1
fj(xj) +fspat(region) +frand(ind).
Additionally, we used six continuous covariates without effect. The number of replications isR = 250 and we assumed a Gaussian model with a standard deviation of σ= 1.1. For the modelling of the spatial function a Markov random field was used with possible degrees of freedom {0,10,20, . . . ,300} and df = 10 for the basis model. The effect of the continuous variables were represented by cubic P–splines with 22 basis functions and possible degrees of freedom {0,1,2, . . . ,21} where the linear fit df = 1 was used for the basis model. The random effect was represented by an i.i.d. Gaussian random effect with possible degrees of freedom{0,1,2, . . . ,19}. For the basis model we used a random effect withdf = 1. For all functions,df = 0 corresponds to the removal of the respective function from the model.
To analyse the results we computed average function estimates, empirical MSE, empirical bias and the ratio of AICimp values. We draw the following conclusions:
−1 −.5 0 .5 1 frand 0 2 4 6 8 10 12 14 16 18 20 ind Random Effect -0.9 0 1.2 spatial effect
Figure 7.22: True smooth spatial function fspat and random effect frand used in the geoad-
ditive simulation study.
0 .002 .004 .006 .008 AICimp−Ratio
adaptive adaptive/exact exact stepwise
Distribution of AICimp−Ratios −2.5 −2.25 −2 −1.75 −1.5 ln(MSE(eta))
adaptive adap./exact exact stepwise MCMC(true)
Distribution of the logarithmic MSE of the predictor
Figure 7.23: The left plot shows the distributions of ratio (7.1). The right plot shows the distributions oflog(MSE(η))for all different approaches. Here, the constant lines indicate the common minimum, median and maximum calculated over all approaches.
• In terms of ratio 7.1 of AICimp values shown in figure 7.23 the adaptive search per-
formed slightly worse than the exact and adaptive/exact search and even than the stepwise algorithm. For the adaptive search, the median of the distribution, however, is just about 0.00025 indicating that the difference to the best model is only 0.025% of the difference between the best and the empty model. Hence, in this respect, there is practically no difference between the algorithms.
−3.6
−3.2
−2.8
−2.4
log(MSE(region))
adaptive adap./exact exact stepwise MCMC(true)
Distribution of the logarithmic MSE of region
−5 −4.5 −4 −3.5 −3 log(MSE(ind))
adaptive adap./exact exact stepwise MCMC(true)
Distribution of the logarithmic MSE of ind
Figure 7.24: Distributions of the logarithmic MSE for the random effect and the spatial function. The constant lines indicate in each case the common minimum, median and maximum calculated over all approaches.
−1 −.5 0 .5 1 f(ind) −1 −.5 0 .5 1 f(ind)
Adaptive search: Random Effect
−1 −.5 0 .5 1 f(ind) −1 −.5 0 .5 1 f(ind)
MCMC (true): Random Effect
Figure 7.25: Estimated random effects (solid line) together with the true underlying random effect (dashed line). All functions are plotted against the true random effect.
0 .1 .2 .3 .4 Fraction 0 1 2 3 4 5
Number of wrongly identified variables
adaptive (mean = 1.428) 0 .1 .2 .3 .4 Fraction 0 1 2 3 4 5
Number of wrongly identified variables
exact (mean = 1.232) 0 .1 .2 .3 .4 Fraction 0 1 2 3 4 5
Number of wrongly identified variables
stepwise (mean = 1.084)
Figure 7.26: Histograms for the distribution of the number of wrongly identified covariates. Wrongly identified means in this case unimportant covariates that were included into the model as there were never any important variables removed.
true model by MCMC techniques yielded slightly better results.
• For the individual nonlinear functions f1 tof6, the logarithmic MSE values show no
difference between the different approaches (not even for MCMC techniques condi- tional on the true model) and, therefore, are not shown. The same applies to the logarithmic MSE for random effect and spatial function (compare figure 7.24). The only exception are the values of MCMC(true) for the spatial function which are in average slightly larger than for the other approaches.
• The average estimated functions ˆf1, . . . ,fˆ6are very similar to the respective estimated
functions of the additive simulation study shown in figures 7.8 and 7.9. Therefore, they are not shown. For some functions there is a small bias which is slightly larger for the adaptive search than for the true model. The largest bias was obtained for function f6 (peak). The average estimated random effects together with the true
random effect are shown in figure7.25. Here, the bias from the adaptive search is not distinguishable from the bias obtained from the true model. For the spatial effect, average estimated functions and empirical bias are shown in figure 7.27. The bias of the spatial function is slightly larger for the adaptive search than for MCMC(true).
• Figure 7.26 shows the number of unimportant variables which were wrongly added
to the model wheres neither approach removed important variables from the model. Again, the results are very similar where the adaptive search yielded slightly worse re- sults and the stepwise algorithm slightly better results than exact and adaptive/exact search. The results of exact and adaptive/exact search are identical.
• The computing times displayed in table 7.9 yielded greater differences between the selection algorithms than all other results. The adaptive search was by far the fastest approach whereas the stepwise algorithm again took the most time.
algorithm adaptive adaptive/exact exact stepwise mcmc (true) runtime 0:59 2:13 2:49 5:04 4:53
-0.9 0 1.2
Adaptive search: spatial effect
-0.9 0 0.9
Adaptive search: Bias for spatial effect
-0.9 0 1.2
MCMC: spatial effect
-0.9 0 0.9
MCMC: Bias for spatial effect
Figure 7.27: Average estimated spatial functions (left column) and their empirical bias (right column) for the adaptive search (top row) and the true model estimated by MCMC techniques (bottom row). Yellow indicates regions without bias.