4.5.1
Setup
In addition to the theoretical analysis of our method in Section 4.4, we also conduct a simulation study to investigate (i) the performance of our method and other com- parators and (ii) sensitivity of our method to violations of the regularity assumptions mentioned above, most notably (IN2) and (IN3). The data generating process for the simulation follows the models (4.2.2) and (4.2.3) in Section 4.2.2 with pz = 100 instruments and px = 150 covariates where Wi. is a multivariate normal with mean
zero and covariance Σ∗ij = 0.5|i−j| for 1≤ i, j ≤ 250. The parameters for the models are: β∗ = 1, φ∗ = (0.6,0.7,0.8,· · · ,1.5,0,0,· · · ,0) ∈ R150 so that s
(1.1,1.2,1.3,· · · ,2.0,0,0,· · · ,0)∈ R150 so that s
x2 = 10, and variance-covariance of the error terms are Var(i1) = Var(i2) = 1.5, and Cov(i1, i2) = 0.75. Instruments that satisfy Assumption (A1) are S∗ = {1, . . . ,7} and instruments that satisfy all
three IV assumptions (A1)-(A3) are V∗ = {1,2,3,4,5}; thus instruments 6 and 7
only satisfy (A1), but do not satisfy (A2) and (A3). We fix these values throughout the entire simulation study.
The parameters we vary in the simulation study are: the sample size n, the strength of IVs via γ∗, and violations of (A2) and (A3) via π∗. For sample size, we let n = (100,200,300,1000,3000). For IV strength, we set γV∗∗ =K(1,1,1,1, ρ1)
and γS∗∗\V∗ =K(1,1) and γ(S∗)C =0, where we vary K (to be discussed later) and
ρ1 = (0,0.1,0.2) across simulations. The value K controls the global strength of instruments, with higher |K| indicating strong instruments in a global sense. The valueρ1 controls the relative individual strength of instruments, specifically between the first four instruments inV∗and the fifth instrument. For example,ρ
1 = 0.2 implies that the fifth IV’s individual strength is only 20% of the other four valid instruments, i.e IVs 1 to 4. Also, varyingρ1 would simulate the adherence of regularity assumption (IN2).
To specify K across simulations, we introduce a quantity we call the oracle con- centration parameter (OCP) denoted as C(γ∗,V∗, n)
C(γ∗,V∗, n) =n γV∗|∗ Σ∗V∗V∗−Σ∗V∗(V∗)CΣ ∗−1 (V∗)C(V∗)CΣ ∗ (V∗)CV∗ γV∗∗ |V∗|Θ∗ 22 , (4.5.1)
whereΣ∗IJ denotes the submatrix containing Σ∗ij for i∈I and j ∈J andγV∗∗ denotes
the subvector containingγj∗ forj ∈ V∗. We define the OCP because the usual concen-
tration parameter can be misleading when there are unknown redundant and invalid instruments and the OCP serves as a proxy for the usual concentration parameter.
Having defined the OCP, we can specify K as a function of n and C(γ∗,V∗, n).
Specifically, if n is set at a baseline of 100 and the simulation parameters V∗, ρ
1, Σ∗ and Θ∗22 are specified as above, we can find K for a particular value of the expected oracle concentration parameter C(γ∗,V∗,100). Thus, by varying C(γ∗,V∗,100) =
(50,100,150,200,250,500,1000), we vary K.
Finally, we vary π∗, which controls the validity of the IVs by defining πj∗ = ρ2γj∗
for j = 6,7 and πj∗ = 0 for all other j so that ρ2 controls the magnitude of the violation of IV assumptions (A2) and (A3) from the 6th and 7th instruments. In the ideal case, we would have ρ2 = 0 so that S∗ = V∗ = {1,2,3,4,5,6,7}. But, ρ2 6= 0 implies that the last two instruments do not satisfy (A2) and (A3). As such, we vary π∗ by varying ρ2 = (0,1,2). Also, varying ρ2 would simulate the adherence of regularity assumption (IN3).
In summary, we varyn, the strength of IVs viaγ∗, and violations of (A2) and (A3) via π∗ in our simulation study, with ρ1 and ρ2 simulating the adherence to the new regularity assumptions in the paper, (IN2), and (IN3), respectively. For the setting n≤p, we compare our procedure toβbH, which assumes IVs are valid. For the setting
n ≥ p, we add two additional comparators, the two-stage least squares (TSLS) and OLS. TSLS is the most popular IV method where one regresses D onZ and X, and uses the predicted value of D in the regression of Y on X and D. Note that the way we implement TSLS mimics most practitioners’ use of TSLS by simply assuming all the instruments Z are valid. OLS is defined as where one regresses Y on D and
X. OLS will be biased because of confounding on D. Finally, for both low and high dimensional settings, we have the oracle TSLS where an oracle provides us with the true set of valid IVs, which will not occur in practice. Our simulations are repeated 500 times.
4.5.2
Results
We present the most representative results from our simulation study. First, Figure 4.1 considers the high dimensional setting with n = 200 and three comparators, our procedure βb that is robust to invalid IVs, our procedure βbH that assumes all valid
IVs, and the oracle TSLS. Columns “Weak” and “Strong” in the figure represent cases where ρ1 = 0.2 and ρ1 = 0, respectively. Columns “Valid” and “Invalid” represent cases where ρ2 = 0 and ρ2 = 2, respectively. The row “MAE” in the figure represents the median absolute error of the estimators, which measures the performance of the point estimators. The row “Coverage” represents the coverage performance of the confidence intervals. Finally, the row “Length” represents the average length of confidence intervals across simulations.
Both estimatorsβband βbH perform well in terms of estimation accuracy, coverage
and length of confidence intervals and have similar performance to the benchmark,
b
βoracle, when all the instruments are valid (i.e. first and second columns of Figure 4.1). For example, in the MAE and length plots, the solid lines, which represent our estimator, the dashed lines, which represent our estimator assuming all valid IVs after conditioning on covariates, and the dotted lines, which represent the oracle, overlap with each other. However, if the instruments are invalid (i.e. the third and fourth columns of Figure 4.1), βbH is not consistent and loses coverage, which makes sense
since βbH assumes all the IVs are valid after conditioning. However, our proposed
estimator βballows for possibly invalid instruments and performs as well as the oracle
in terms of estimation accuracy and coverage. The average length of our robust confidence interval is only slightly larger than that of the oracle.
Figure 4.2 represents the same setting as Figure 4.1 except we now consider a larger sample sizen = 1000. Even though n is larger than p, we still consider this to be in the many controls/high dimensional setting because the ratio ofp ton is away
from zero at 1/4. As expected, the estimators βb and βbH along with the traditional
TSLS estimator perform similarly to the oracle benchmark in terms of estimation accuracy, coverage and the length of confidence intervals when all the instruments are actually valid. For example, in the MAE plot of Figure 4.2, the solid, dashed, green and dotted lines, representingβb,βbH, TSLS and the oracle, respectively, overlap
with each other. Note that OLS cannot deal with confounding and hence, produces a biased estimate. However, when the instruments are invalid, the traditional TSLS estimator and βbH are biased and fail to have the correct coverage. In contrast, the
proposed estimator βbperforms as well as the oracle estimator in terms of estimation
accuracy and coverage, with the length of the proposed estimator being slightly longer than that for the oracle.
Finally, Figure 4.3 represents the setting where invalid instruments are present after conditioning on low dimensional covariates where pz = 9 and px = 10 so that
no coefficients for φ∗ and ψ∗ are zero and the sample size is n = 1000. If we use the estimator βbE defined in (4.3.15) and the confidence interval (4.3.16), the proposed
procedure performs almost the same as the oracle in terms of accuracy, coverage property and length, which supports the theory established in Theorem 20. Note that the performance of our procedure under the low dimensional setting with invalid IVs does not rely on assumptions (R1)-(R3) and, more importantly, (IN2)-(IN3).