Results - Simulation study - Robust methods in Mendelian randomization

3.5 Simulation study

3.5.3 Results

The mean proportion of variance in the risk factor explained by the genetic variants (R2

statistic), mean F statistic, and mean I2_{statistic are contained in Table 3.3 for scenarios}

1-4 for the null and positive causal effects by the number of invalid instruments. The mean R2 _{values were greater than 3% for all of the scenarios, and the minimum mean}

F-statistic was 20.8. The I2 _{statistic ranged from 39.1% to 77.5%. Since violations}

in the NOME assumption can lead to attenuation towards the null for the MR-Egger estimates, and this attenuation is approximately equal to the I2 _{statistic, we expected}

the MR-Egger estimates for the positive causal effect to be severely attenuated towards the null [49].

Table 3.3 Mean values of the R2 (%), F-statistic and I2 (%) for Scenarios 1-4 with a null

(θ = 0) or positive (θ = 0.3) causal effect by the number of invalid instrumental variables (IV).

No invalid IVs 1 invalid IV 3 invalid IVs 6 invalid IVs R2 _F _I2 _R2 _F _I2 _R2 _F _I2 _R2 _F _I2

Null causal effect: θ = 0

Scenario 1 3.0 20.8 39.6 - - - -

Scenario 2 - - - 3.0 20.8 39.6 3.0 20.8 39.3 3.0 20.8 39.5 Scenario 3 - - - 3.0 20.8 39.7 3.0 20.8 39.5 3.0 20.8 39.2 Scenario 4 - - - 3.4 23.6 56.5 4.2 29.3 70.7 5.4 37.7 77.5

Positive causal effect: θ = 0.3

Scenario 1 3.0 20.8 39.3 - - - -

Scenario 2 - - - 3.0 20.8 39.1 3.0 20.8 39.4 3.0 20.8 39.6 Scenario 3 - - - 3.0 20.8 39.9 3.0 20.8 39.7 3.0 20.8 39.6 Scenario 4 - - - 3.4 23.6 56.4 4.2 29.3 70.8 5.4 37.7 77.4

Results from the simulation study for the IVW model with: 1) the J genetic variants (IVW); 2) robust regression (Rr); 3) penalized weights (PW); 4) robust regression and penalized weights; and 5) the three sets of variants selected by LTS selection are provided in Table 3.4 (Scenario 1 only), Table 3.5 (null causal effect θ = 0), and Table 3.6 (positive causal effect θ = 0.3). Tables 3.4 to 3.6 also contain the results from Lasso selection with the heterogeneity stopping rule, simple (unweighted)

3.5 Simulation study 63

median, weighted median and MR-Egger methods, and for each method, information on the: mean estimate; mean standard error of the estimates; standard deviation of the estimates; coverage of the true causal effect of the 95% confidence interval; and power to detect the causal effect at the 5% significance level are provided. The power (at the 5% significance level) of the intercept test in the MR-Egger method for detecting directional pleiotropy and/or violation of the InSIDE assumption in all scenarios is provided in Table 3.7. The number of robust regression models that did not report a standard error (maximum of 2.6% across all of the scenarios considered) are given in the Table H.1. Apart from the calculation of the mean standard error, the robust regression models that did not report a standard error were included in the results, and the power calculations treated the standard error as infinite.

When all of the genetic variants were valid IVs (Table 3.4), all of the methods produced unbiased estimates of the null causal effect. With the exception of the IVW model with the h variants selected from LTS selection, the Type 1 error rates for the null causal effect were close to the nominal level of 5%. Apart from the simple median method, there was attenuation towards the null with a positive causal effect for all methods, and as expected, this was particularly evident for the MR-Egger method (also observed for Scenarios 2 and 3). Violation of the NOME assumption can lead to inflation of the intercept term in the MR-Egger method [49], and this was true for the simulation study where the power to detect the intercept term for Scenarios 1 and 2 was greater than 5% (Table 3.7). Only 7.5% of the MR-Egger models detected a positive causal effect, and apart from the median estimators and the IVW model with the h variants from LTS, all of the robust methods had approximately 95% power to detect the positive causal effect.

Although the mean estimates in Scenario 2 (Tables 3.5 and 3.6) were similar to those in Scenario 1, there were clear differences in the precision of the estimates for the null and positive causal effects, with most of the methods reporting larger mean standard errors under Scenario 2. With the exception of the IVW model with the h variants from LTS selection, where the mean standard error remained constant, the mean standard error increased as the number of invalid instruments increased for all methods. As seen in Scenario 1, the IVW model with the h variants from LTS selection had inflated Type I error rates and poor coverage. The IVW model with penalized weights had the most precise estimates, but suffered from inflated Type I error rates and poor coverage. The estimates from Lasso selection and the automated approach to LTS selection were almost identical for Scenarios 1 and 2. The simple and weighted

Table 3.4 Mean (standard error), standard deviation, coverage (%), and power (%) of the

estimates from the IVW model with: 1) the J genetic variants (IVW); 2) robust regression; 3) penalized weights; 4) robust regression and penalized weights; 5) the three sets of variants selected by the least trimmed squared (LTS) estimator; and 6) the genetic variants from the Lasso selection method with the heterogeneity stopping rule for Scenario 1 with a null (θ = 0) or positive (θ = 0.3) causal effect. Results from the simple (unweighted) median, weighted median and MR-Egger methods are also provided.

Null causal effect(θ = 0) Positive causal effect(θ = 0.3)

Estimate (SE) SD Cov. Pow. Estimate (SE) SD Cov. Pow.

Scenario 1. No pleiotropy, InSIDE automatically satisfied

IVW -0.001 (0.061) 0.058 95.7 4.3 0.287 (0.073) 0.069 95.5 98.2

Robust regression -0.001 (0.066) 0.060 95.1 4.9 0.287 (0.079) 0.072 94.7 94.8

Penalized weights -0.001 (0.060) 0.059 95.0 5.0 0.289 (0.072) 0.071 94.7 98.2

Robust regression with -0.001 (0.064) 0.061 94.5 5.5 0.288 (0.077) 0.073 94.1 95.7

penalized weights LTSa Variants from h -0.001 (0.078) 0.116 81.9 18.1 0.291 (0.092) 0.140 80.4 78.5 Variants from wLT S,2 -0.001 (0.061) 0.064 93.3 6.7 0.289 (0.073) 0.077 93.1 97.4 Automated approach -0.001 (0.060) 0.059 95.1 4.9 0.287 (0.072) 0.071 94.9 98.1 Lasso selection -0.001 (0.060) 0.059 94.8 5.2 0.287 (0.072) 0.071 94.6 98.0 Median Simple -0.002 (0.086) 0.074 97.9 2.1 0.301 (0.105) 0.090 98.0 86.9 Weighted -0.002 (0.080) 0.071 97.4 2.6 0.277 (0.097) 0.085 96.7 85.7 MR-Egger -0.001 (0.219) 0.207 96.1 3.9 0.143 (0.261) 0.251 91.0 7.5

Abbreviations: SE, standard error; SD, standard deviation; Cov., coverage; Pow., power; InSIDE, instrument strength independent of direct effect; IVW, inverse-variance weighted; LTS, least trimmed squares.

aThe following three sets of genetic variants were selected from the LTS estimator and included in the IVW model: 1) the h=8 variants used to estimate the initial LTS estimate ˆθLT S,h; 2) the variants with a weight of 1 in wLT S,2; and

3) the variants selected from the automated approach based on the heterogeneity stopping rule.

median estimators performed just as well, if not better, than the other robust methods for Scenario 2.

In Scenario 3 (directional pleiotropy, InSIDE satisfied), the IVW method produced biased causal estimates with inflated Type 1 error rates, and the degree of bias increased with the number of invalid IVs. With one invalid instrument, estimates from the robust methods were slightly biased and Type 1 error rates were fairly well controlled (with the exception of the IVW model with the h variants from LTS selection). As the number of instruments increased, bias in the estimates for the robust methods also increased, although the magnitude of bias was smaller than the IVW method, and Type 1 error inflation was less severe. The performance of the LTS selection method varied, and the estimates based on the h variants were the least biased across all of the robust methods, however, as with Scenarios 1 and 2, the estimates were too precise and had poor coverage. The estimates from the LTS selection method using the automated approach and Lasso selection produced almost identical results with the exception of 6

3.5 Simulation study 65

invalid instruments, where the LTS selection method produced less biased estimates. Robust regression with penalized weights performed reasonably well when there was 1 or 3 invalid instruments.

In Scenario 4 (directional pleiotropy, InSIDE violated), all of the robust methods produced biased estimates. When there was only one invalid instrument, the magnitude of bias from the robust methods was less severe than the IVW method, and this was particularly true for robust regression with penalized weights and LTS selection. Unlike robust regression with penalized weights, LTS selection suffered from poor coverage of the causal effect when there was one invalid IV. As the number of invalid IVs increased, the performance of the robust methods worsened, and there was little advantage in applying the robust methods compared to the median estimator in Scenario 4 when 6 of the 15 genetic variants were invalid IVs.

Table 3.5 Mean (standard error), standard deviation, coverage (%), and power (%) of the estimates from

the IVW model with: 1) the J genetic variants (IVW); 2) robust regression (Rr); 3) penalized weights (PW); 4) robust regression and penalized weights (Rr and PW); 5) the three sets of variants selected by the least trimmed squared (LTS) estimator; and 6) the genetic variants from the Lasso selection (LS) method with the heterogeneity stopping rule for Scenarios 2-4 with a null causal effect (θ = 0) by the number of invalid IVs. Results from the simple median, weighted median and MR-Egger methods are also provided.

1 invalid IV 3 invalid IVs 6 invalid IVs

Est. (SE) SD Cov. Pow. Est. (SE) SD Cov. Pow. Est. (SE) SD Cov. Pow.

Scenario 2. Balanced pleiotropy, InSIDE satisfied

IVW -0.002 (0.089) 0.092 94.7 5.3 0.000 (0.133) 0.136 93.4 6.6 0.000 (0.180) 0.183 93.0 7.0 Rr -0.002 (0.069) 0.065 94.3 5.7 0.000 (0.096) 0.087 94.5 5.5 0.001 (0.196) 0.173 94.3 5.6 PW -0.002 (0.062) 0.064 94.2 5.8 0.000 (0.066) 0.077 91.1 8.9 0.001 (0.075) 0.116 81.5 18.5 Rr and PW -0.002 (0.071) 0.065 94.6 5.4 0.001 (0.094) 0.078 94.7 5.2 0.001 (0.160) 0.119 91.5 7.3 LTSa h -0.002 (0.078) 0.115 82.2 17.8 0.001 (0.079) 0.113 83.7 16.3 0.001 (0.080) 0.118 86.2 13.8 wLT S,2 -0.001 (0.064) 0.066 94.2 5.8 0.001 (0.075) 0.086 92.0 8.0 0.002 (0.142) 0.180 85.2 14.8 Auto -0.002 (0.064) 0.064 94.6 5.4 0.000 (0.071) 0.081 91.8 8.3 0.001 (0.091) 0.136 83.5 16.5 LS -0.002 (0.063) 0.065 94.4 5.6 0.000 (0.071) 0.080 91.7 8.3 0.001 (0.088) 0.129 84.5 15.5 Median Simple -0.002 (0.090) 0.080 97.4 2.6 0.001 (0.097) 0.094 96.5 3.5 0.002 (0.115) 0.132 92.7 7.3 Weighted -0.001 (0.082) 0.076 96.9 3.2 0.000 (0.089) 0.090 95.2 4.8 0.000 (0.101) 0.133 88.9 11.1 MR-Egger -0.004 (0.317) 0.335 92.7 7.3 -0.009 (0.477) 0.496 92.7 7.3 -0.006 (0.646) 0.661 93.0 7.0

Scenario 3. Directional pleiotropy, InSIDE satisfied

IVW 0.064 (0.089) 0.064 94.8 5.2 0.194 (0.126) 0.076 76.0 24.0 0.388 (0.154) 0.089 16.1 83.9 Rr 0.010 (0.069) 0.064 94.3 5.7 0.069 (0.113) 0.083 93.9 6.1 0.335 (0.227) 0.105 63.6 36.4 PW 0.007 (0.062) 0.063 94.2 5.8 0.033 (0.067) 0.078 89.2 10.8 0.148 (0.082) 0.137 57.3 42.7 Rr and PW 0.005 (0.072) 0.065 94.8 5.2 0.025 (0.092) 0.079 93.2 6.7 0.115 (0.138) 0.147 78.6 20.9 LTSa h 0.000 (0.078) 0.115 82.6 17.4 0.007 (0.079) 0.116 83.0 17.0 0.030 (0.080) 0.143 84.1 15.9 wLT S,2 0.004 (0.064) 0.066 94.2 5.8 0.041 (0.076) 0.085 90.5 9.5 0.283 (0.130) 0.155 37.6 62.4 Auto 0.006 (0.063) 0.064 94.5 5.5 0.030 (0.071) 0.081 90.5 9.5 0.119 (0.093) 0.160 68.7 31.3 LS 0.006 (0.063) 0.065 94.2 5.8 0.031 (0.071) 0.080 90.3 9.7 0.164 (0.096) 0.146 60.5 39.5 Median Simple 0.021 (0.089) 0.077 97.3 2.7 0.074 (0.100) 0.086 92.9 7.2 0.224 (0.134) 0.124 64.3 35.7 Weighted 0.017 (0.082) 0.074 96.9 3.1 0.065 (0.090) 0.085 91.7 8.3 0.210 (0.110) 0.149 56.9 43.1 MR-Egger -0.003 (0.318) 0.334 92.9 7.2 -0.001 (0.450) 0.465 93.2 6.8 -0.004 (0.544) 0.562 92.4 7.6

Scenario 4. Directional pleiotropy, InSIDE violated

IVW 0.077 (0.070) 0.058 83.4 16.7 0.186 (0.075) 0.056 25.6 74.4 0.290 (0.071) 0.050 0.3 99.7 Rr 0.031 (0.085) 0.069 93.9 6.0 0.142 (0.127) 0.082 73.1 26.0 0.289 (0.079) 0.053 3.3 96.6 PW 0.021 (0.061) 0.070 89.2 10.8 0.083 (0.063) 0.091 64.1 35.9 0.231 (0.061) 0.092 12.2 87.8 Rr and PW 0.018 (0.071) 0.070 92.7 7.3 0.075 (0.084) 0.095 76.2 23.7 0.230 (0.074) 0.101 19.0 80.8 LTSa h 0.005 (0.078) 0.121 79.6 20.4 0.030 (0.076) 0.139 75.0 25.0 0.162 (0.066) 0.197 45.7 54.3 wLT S,2 0.017 (0.063) 0.074 89.7 10.3 0.090 (0.067) 0.103 61.5 38.5 0.266 (0.069) 0.088 5.8 94.3 Auto 0.025 (0.062) 0.073 88.1 11.9 0.105 (0.066) 0.108 51.9 48.1 0.265 (0.067) 0.103 6.7 93.3 LS 0.024 (0.062) 0.073 88.2 11.8 0.116 (0.066) 0.099 51.1 48.9 0.286 (0.066) 0.070 2.2 97.8 Median Simple 0.020 (0.089) 0.077 97.3 2.7 0.071 (0.092) 0.083 89.9 10.1 0.192 (0.088) 0.091 40.0 60.0 Weighted 0.055 (0.082) 0.077 91.0 9.0 0.198 (0.081) 0.097 34.8 65.2 0.343 (0.069) 0.074 0.5 99.5 MR-Egger 0.305 (0.214) 0.219 66.8 33.2 0.539 (0.197) 0.183 21.3 78.7 0.644 (0.182) 0.165 5.1 94.9

Abbreviations: IV, instrumental variable; Est. estimate; SE, standard error; SD, standard deviation; Cov., coverage; Pow., power; InSIDE, instrument strength independent of direct effect; IVW, inverse variance weighted; Rr, robust regression; PW, penalized weights; LTS, least trimmed squares; LS, Lasso selection; Auto, automated.

aThe following three sets of genetic variants were selected from the LTS estimator and included in the IVW model: 1) the h=8 variants used to estimate the initial LTS estimate ˆθLT S,h; 2) the variants with a weight of 1 in wLT S,2; and 3) the variants selected from the

3.5 Simulation study 67

Table 3.6Mean (standard error), standard deviation, coverage (%), and power (%) of the estimates from

the IVW model with: 1) the J genetic variants (IVW); 2) robust regression (Rr); 3) penalized weights (PW); 4) robust regression and penalized weights (Rr and PW); 5) the three sets of variants selected by the least trimmed squared (LTS) estimator; and 6) the genetic variants from the Lasso selection (LS) method with the heterogeneity stopping rule for Scenarios 2-4 with a positive causal effect (θ = 0.3) by the number of invalid IVs. Results from the simple median, weighted median and MR-Egger methods are also provided.

1 invalid IV 3 invalid IVs 6 invalid IVs

Est. (SE) SD Cov. Pow. Est. (SE) SD Cov. Pow. Est. (SE) SD Cov. Pow.

Scenario 2.Balanced pleiotropy, InSIDE satisfied

IVW 0.286 (0.097) 0.100 94.0 80.9 0.288 (0.139) 0.140 93.5 54.5 0.285 (0.184) 0.184 93.3 34.3 Rr 0.287 (0.084) 0.079 93.9 91.0 0.288 (0.116) 0.107 93.6 71.1 0.286 (0.193) 0.178 93.8 34.5 PW 0.289 (0.074) 0.079 93.2 96.5 0.291 (0.080) 0.098 88.6 91.8 0.295 (0.090) 0.147 78.4 80.5 Rr and PW 0.289 (0.083) 0.080 93.5 92.5 0.290 (0.100) 0.097 92.2 81.6 0.295 (0.145) 0.147 88.1 59.4 LTSa h 0.292 (0.092) 0.141 80.2 78.8 0.292 (0.093) 0.143 80.5 78.9 0.292 (0.094) 0.164 79.2 78.5 wLT S,2 0.288 (0.076) 0.081 93.2 95.1 0.289 (0.092) 0.108 90.2 83.6 0.287 (0.156) 0.192 85.4 46.5 Auto 0.287 (0.076) 0.079 93.5 95.3 0.289 (0.086) 0.103 89.6 87.2 0.289 (0.112) 0.178 79.0 67.1 LS 0.287 (0.076) 0.080 93.2 95.4 0.288 (0.085) 0.102 89.4 88.0 0.288 (0.108) 0.167 80.5 69.9 Median Simple 0.302 (0.109) 0.097 97.5 83.1 0.302 (0.118) 0.113 96.1 75.3 0.303 (0.136) 0.155 92.5 61.4 Weighted 0.276 (0.100) 0.091 96.1 81.9 0.277 (0.106) 0.108 94.0 75.6 0.277 (0.119) 0.152 88.2 63.2 MR-Egger 0.143 (0.349) 0.363 90.2 9.0 0.138 (0.495) 0.518 91.2 8.4 0.126 (0.657) 0.681 92.1 7.5

Scenario 3.Directional pleiotropy, InSIDE satisfied

IVW 0.353 (0.098) 0.075 96.1 97.8 0.482 (0.133) 0.087 81.5 99.3 0.673 (0.160) 0.101 28.3 100 Rr 0.306 (0.084) 0.077 95.1 94.8 0.383 (0.134) 0.099 93.9 86.2 0.631 (0.205) 0.112 60.8 90.8 PW 0.303 (0.074) 0.078 93.8 98.0 0.346 (0.081) 0.100 86.4 97.8 0.511 (0.098) 0.164 47.6 99.1 Rr and PW 0.300 (0.083) 0.080 94.1 93.5 0.335 (0.102) 0.102 91.3 88.8 0.485 (0.142) 0.179 66.2 86.9 LTSa h 0.297 (0.092) 0.144 79.3 79.2 0.308 (0.093) 0.148 80.2 81.4 0.366 (0.094) 0.214 75.1 86.4 wLT S,2 0.299 (0.076) 0.081 93.4 97.0 0.355 (0.092) 0.107 88.8 96.8 0.605 (0.142) 0.147 39.6 99.2 Auto 0.301 (0.076) 0.078 94.3 97.5 0.340 (0.087) 0.105 88.4 96.3 0.490 (0.114) 0.188 56.2 95.9 LS 0.301 (0.076) 0.079 94.0 97.4 0.340 (0.086) 0.104 88.2 96.5 0.513 (0.113) 0.168 53.8 98.5 Median Simple 0.329 (0.110) 0.095 97.5 89.5 0.393 (0.125) 0.108 93.5 93.2 0.572 (0.158) 0.150 61.8 97.2 Weighted 0.300 (0.100) 0.090 97.2 88.1 0.356 (0.111) 0.104 94.5 93.0 0.516 (0.131) 0.161 63.6 97.4 MR-Egger 0.142 (0.345) 0.353 90.9 8.7 0.138 (0.468) 0.485 91.3 8.1 0.137 (0.555) 0.576 91.9 8.0

Scenario 4.Directional pleiotropy, InSIDE violated

IVW 0.367 (0.080) 0.071 88.8 99.8 0.478 (0.084) 0.067 42.2 100 0.582 (0.078) 0.062 2.3 100 Rr 0.329 (0.100) 0.082 94.3 90.2 0.447 (0.128) 0.085 73.7 90.2 0.581 (0.087) 0.066 8.2 99.7 PW 0.323 (0.072) 0.086 88.6 98.2 0.403 (0.072) 0.102 61.2 98.9 0.546 (0.068) 0.092 11.2 99.9 Rr and PW 0.318 (0.085) 0.087 92.4 94.1 0.397 (0.095) 0.107 72.7 93.6 0.547 (0.077) 0.098 16.0 98.4 LTSa h 0.306 (0.091) 0.148 77.1 80.9 0.345 (0.087) 0.175 67.5 83.2 0.492 (0.075) 0.205 37.8 93.0 wLT S,2 0.316 (0.074) 0.091 88.2 97.1 0.406 (0.077) 0.112 60.9 98.3 0.567 (0.075) 0.089 6.8 99.6 Auto 0.324 (0.074) 0.088 88.1 97.8 0.424 (0.076) 0.112 52.5 98.4 0.570 (0.074) 0.094 5.4 99.2 LS 0.323 (0.073) 0.089 87.9 97.6 0.430 (0.076) 0.105 52.7 99.3 0.579 (0.073) 0.079 4.4 100 Median Simple 0.328 (0.108) 0.095 97.0 89.9 0.387 (0.111) 0.101 90.0 95.1 0.509 (0.101) 0.101 43.4 99.5 Weighted 0.344 (0.099) 0.095 94.0 94.4 0.496 (0.097) 0.108 47.2 99.7 0.625 (0.085) 0.087 3.4 100 MR-Egger 0.488 (0.254) 0.259 86.1 51.8 0.767 (0.233) 0.220 45.4 90.0 0.887 (0.214) 0.197 20.2 98.1

Table 3.7 Power (%) of the intercept test in the MR-Egger method for detecting directional

pleiotropy and/or violation of the InSIDE assumption for Scenarios 1-4 with a null (θ = 0) or positive (θ = 0.3) causal effect by the number of invalid instrumental variables (IV).

Null causal effect Positive causal effect No. invalid: 0 1 3 6 0 1 3 6

Scenario 1 3.7 - - - 8.7 - -

Scenario 2 - 7.2 7.5 7.0 - 9.4 8.5 7.8 Scenario 3 - 7.2 8.7 13.1 - 11.2 13.8 19.1 Scenario 4 - 8.6 26.2 32.0 - 22.8 49.9 55.9

The performance of the penalized weights, Lasso selection and LTS selection methods can also be evaluated by considering the mean number of genetic variants whose weights were penalized or were not selected for the IVW model (Table 3.8). With the exception of the scenario when there was only one invalid instrument, the mean numbers of penalized or not selected variants were noticeably smaller than the actual number of invalid instruments for all of the robust methods. There was little difference between the mean number of variants penalized or not selected for Scenarios 2 and 3 for the different methods. However, there were large reductions in the mean number of variants penalized or not selected for the IVW method for Scenario 4 compared to Scenarios 2 and 3. As the number of invalid IVs increased from 1 to 6, the percentage of simulated datasets that correctly penalized or did not include all of the invalid instruments decreased considerably. In terms of the mean number of variants penalized or not selected for the IVW method, and the frequency that all invalid instruments were correctly downweighted or not selected, the IVW method with penalized weights was generally the most effective method across the different scenarios for both the null and positive causal effects.

Results from applying robust regression and penalized weights to the MR-Egger method are provided in Table H.2. We had hoped that combining these robust methods with the MR-Egger method would provide additional robustness. In particular, we anticipated that there would be less bias in the estimates of the causal effect when the robust methods and MR-Egger were combined. However, the results were disappointing as there was no improvement in the performance of the methods when they were combined, and all of the models were affected by the violation of the NOME assumption.

Finally, results from the one-sample setting are provided in the Table H.3 and Table H.4. Bias in the direction of the observational association was observed in all methods. As with the two–sample setting, the IVW model with the h variants from

3.5 Simulation study 69

LTS selection produced the least biased estimates for all scenarios, and the IVW with penalized weights was the most precise.

Table 3.8 Mean estimate, mean number (minimum, maximum) and standard deviation of

variants penalized or not selected for the IVW method, and the frequency (%) all invalid instruments had their weights penalized by the penalized weights method or were not selected for the IVW method under Lasso selection with the heterogeneity stopping rule or LTS selection for wLT S,2 and the automated approach for Scenarios 2-4 with a null (θ = 0) or

positive (θ = 0.3) causal effect by the number of invalid instruments.

Null causal effect(θ = 0) Positive causal effect(θ = 0.3)

Mean Mean no. SD Freq. Mean Mean no. SD Freq. estimate (min, max) (%) estimate (min, max) (%)

1 invalid IV

Scenario 2. Balanced pleiotropy, InSIDE satisfied

Penalized weights -0.002 1.04 (0, 5) 0.542 86.2 0.289 1.01 (0, 5) 0.632 78.0 Lasso selection -0.002 0.91 (0, 8) 0.621 80.3 0.287 0.83 (0, 8) 0.730 69.8 LTS wLT S,2 variants -0.001 1.10 (0, 7) 0.739 85.3 0.288 1.01 (0, 7) 0.804 75.2

LTS automated -0.002 0.84 (0, 5) 0.475 79.3 0.287 0.74 (0, 5) 0.544 68.0 Scenario 3. Directional pleiotropy, InSIDE satisfied

Penalized weights 0.007 1.04 (0, 5) 0.540 86.5 0.303 1.00 (0, 5) 0.629 76.8 Lasso selection 0.006 0.91 (0, 10) 0.623 80.7 0.301 0.81 (0, 9) 0.681 69.7 LTS wLT S,2 variants 0.004 1.12 (0, 7) 0.737 85.7 0.299 1.02 (0, 7) 0.786 75.4

LTS automated 0.006 0.85 (0, 4) 0.467 79.9 0.301 0.74 (0, 5) 0.536 68.4 Scenario 4. Directional pleiotropy, InSIDE violated

Penalized weights 0.021 0.88 (0, 5) 0.614 70.1 0.301 0.77 (0, 4) 0.673 54.7 Lasso selection 0.024 0.69 (0, 9) 0.725 56.2 0.301 0.57 (0, 10) 0.764 42.3 LTS wLT S,2 variants 0.017 0.95 (0, 7) 0.835 67.7 0.316 0.85 (0, 7) 0.878 54.2

LTS automated 0.025 0.59 (0, 3) 0.558 54 0.324 0.46 (0, 3) 0.559 40.0

3 invalid IV

Scenario 2. Balanced pleiotropy, InSIDE satisfied

Penalized weights 0.000 2.73 (0, 6) 0.723 62.4 0.291 2.51 (0, 7) 0.853 45.0 Lasso selection 0.000 2.48 (0, 10) 0.928 52.5 0.288 2.16 (0, 9) 1.10 35.8 LTS wLT S,2 variants 0.001 2.21 (0, 7) 1.000 41.7 0.289 1.78 (0, 7) 1.07 24.6

LTS automated 0.000 2.35 (0, 6) 0.837 49.5 0.289 1.98 (0, 6) 0.96 31.8 Scenario 3. Directional pleiotropy, InSIDE satisfied

Penalized weights 0.033 2.67 (0, 6) 0.790 56.1 0.346 2.46 (0, 7) 0.942 38.6 Lasso selection 0.031 2.48 (0, 9) 1.06 52.4 0.340 2.14 (0, 10) 1.21 36.1 LTS wLT S,2 variants 0.041 2.15 (0, 7) 1.02 41.2 0.355 1.75 (0, 7) 1.13 26.1

LTS automated 0.030 2.29 (0, 5) 0.912 49.5 0.340 1.92 (0, 7) 1.04 32.9 Scenario 4. Directional pleiotropy, InSIDE violated

Penalized weights 0.083 1.97 (0, 6) 1.01 26.2 0.336 1.58 (0, 5) 1.04 12.8 Lasso selection 0.116 1.45 (0, 11) 1.68 16.4 0.348 1.08 (0, 11) 1.50 8.2 LTS wLT S,2 variants 0.090 1.41 (0, 7) 1.27 22.1 0.406 1.10 (0, 7) 1.19 12.4

LTS automated 0.105 1.10 (0, 5) 1.19 18.9 0.424 0.74 (0, 6) 1.03 9.1

6 invalid IV

Scenario 2. Balanced pleiotropy, InSIDE satisfied

Penalized weights 0.001 5.26 (1, 9) 0.979 37.4 0.295 4.73 (0, 9) 1.14 18.5 Lasso selection 0.001 4.72 (0, 12) 1.64 31.8 0.288 3.87 (0, 13) 1.86 15.5 LTS wLT S,2 variants 0.002 1.64 (0, 7) 1.8 5.3 0.287 1.12 (0, 7) 1.48 1.8

LTS automated 0.001 4.28 (0, 7) 1.53 24.6 0.289 3.30 (0, 7) 1.67 9.5 Scenario 3. Directional pleiotropy, InSIDE satisfied

Penalized weights 0.148 4.94 (0, 11) 1.36 15.6 0.511 4.23 (0, 10) 1.45 5.1 Lasso selection 0.164 4.63 (0, 13) 2.75 23.3 0.513 3.37 (0, 13) 2.65 10.5 LTS wLT S,2 variants 0.283 1.40 (0, 7) 1.82 5.7 0.605 0.93 (0, 7) 1.50 2.5

LTS automated 0.119 3.78 (0, 7) 2.02 26.2 0.490 2.60 (0, 7) 2.09 12.2 Scenario 4. Directional pleiotropy, InSIDE violated

Penalized weights 0.231 2.36 (0, 7) 1.48 1.6 0.399 1.81 (0, 7) 1.30 0.4 Lasso selection 0.286 0.97 (0, 13) 1.87 0.1 0.446 0.83 (0, 13) 1.58 0.0 LTS wLT S,2 variants 0.266 0.59 (0, 7) 1.29 2.2 0.448 0.52 (0, 7) 1.08 0.9

LTS automated 0.265 0.63 (0, 7) 1.44 3.8 0.570 0.44 (0, 7) 1.03 1.2

Abbreviations: no., number; min, minimum; max, maximum; SD, standard deviation; Freq., frequency; IV, instrumental variable; InSIDE, instrument strength independent of direct effect; LTS, least trimmed squares.

3.5 Simulation study 71

In document Robust methods in Mendelian randomization (Page 92-101)