14.74 Lecture 5: Health (2)
Esther Duflo
February 17, 2004
1
Possible Interventions
Last time we discussed possible interventions. Let’s take one: providing iron supplements to people, for example.
From the data, what effect do we expect from this intervention?
-But what doubts can we have about this interpretation of the data ?
-2
Finding out what works: the value of experiments
This conversation should have convinced you that, using only data from observations, we can form intelligent hypotheses, but not resolve them. Before spending all of our money on some-thing, how do wefind out whether or not it will work?
Why do we have problems teasing out causal relationship in real-life data?
-To address these, we need to compare comparable people, some of whom were exposed to a particular policy and some of whom were not.
Why not do it to test an intervention in this context?
To test the effect of a policy, we can use randomized evaluation, where a randomly selected
treatment group receives a treatment, while the other group does not (this is the comparison group). We will collect data on both the treatment and the comparison group, and compare the result. Because the treatment and the comparison group have been randomly selected, we can conclude that any statistically significant difference we observe between the treatment and the comparison group is due to the interventions.
Now, a more formal introduction to this problem: Let us call YT
i the earnings level of an individual i with a diet rich in iron i and YiN T the
earnings of the same individual iif his diet is poor in iron. Can we observeYiT and YiN T at the same time?
YiT and YiN T are calledpotential outcomes. We are interested in the difference:
YiT −YiN T
The effect of having textbooks for school i.
The problem: we don’t observe individual iboth with and without the diet rich in iron at the same time. What can we do? We will never know the effect of having iron on a particular individual. We may hope to learn theaverage effect of a diet rich in iron.
E[YiT −YiN T]
Imagine we have access to data on lots of individuals in the regions. Some individuals have a diet rich in iron and others do not. We may think of taking the average in both groups, and the difference between the two. Why does it make sense?
E[YiT/high iron diet]−[YiN T/low iron diet] =E[YiT/T]−E[YiN T/N T]
Subtract and addE[YiN T/T]
E[YiT/T]−E[YiN T/T]−E[YiN T/N T]+E[YiN T/T] =E[YiT−YiN T/T]+E[YiN T/T]−E[YiN T/N T] • The first termE[YiT −YiN T/T]is the treatment effect that we try to isolate: on average,
• What is: -E[YiN T/T]? -E[YN T
i /N T]?
- The differenceE[YiN T/T]−E[YiN T/N T]? - Which is likely to be bigger? Why?
The difference is the selection bias. It tells me that beside the effect of the textbooks, there may be systematic differences between those who have iron and those who do not.
2.1
What happens when we randomly allocate the treatment?
Suppose that we select the individual to whom we give the iron supplement randomly within a population of individuals. We observe the test scores in both the treatment schools, and the other schools, which will form ourcontrol (or comparison) group.
On average, what do we expect tofind if we compare the treated schools and the comparison schools before the intervention? If we compare other characteristics of these schools?
CompareE[YN T/N T]and E[YN T/T] → What isE[YT/T]−E[YN T/N T]equal to?
Example: Iron supplementation in Indonesia.
• Base level of anemia: figure 1
• STEP ONE: design. About 3,000 households. Households are randomly selected to be in the placebo or treatment group. Iron is distributed at home in blister packs.
• STEP TWO: Baseline comparison: table 3.
— In which column do we see the baseline comparison?
— What do we expect for the baseline comparison?
— Why is it important?
— What is the mean difference at baseline for men? for women?
— Are these differences significant?
• STEP THREE: Protecting the design. Compliance is strictly enforced (over 90%).
• What is the right comparison? Why?
— Those who took the pills versus all of those who did not?
— Those who took the pills versus the comparison group?
— All of those initially in the treatment group versus (supposed to take the pills) all of those initially in the comparison group (not supposed to take the pills)?
— This comparison is called the INTENTION TO TREAT estimate.
— How do we obtain the AVERAGE EFFECT on those who took the pills?
• Remark: Is it a program that could be scaled up? Why or why not? Why do we care about the results then?
• STEP FOUR: Attrition
— What could happen to the sample if the treatment people were much healthier because of the experiment and the comparison people saw no improvement?
— How could that affect the results?
— What do we need to do to avoid that?
— In this experiment:
∗ Attrition was 3%
∗ Attrition was no lower in treatment group
∗ Attrition is not related to baseline hb levels.
• STEP FIVE: Results
• Effect on hb level:
• Results: figure 2, table 3: effect on hb level in blood.
— What is column 5? What is the difference with column 3, and which is best to use?
— What is column 6? How does it differ? Do we expect it to be different from 5 ? What is best to use?
— What is column 7? How does it relate tofigure 2? Why do we see the pattern we see infigure 2?
— What is column 9?
• Do we observe what we expected?
• Tables 4 to 7: eesults on work, health, happiness. How do we read these results? What are the main conclusions we can draw?
-3
Experiments in Udaipur
What experiments do you propose to do in Udaipur?
The experiments we propose: -Public Health: chlorination of wells -Nutrition: iron fortification of flour
-Health care: second ANM in public centers -Health investment: incentives for vaccinations.
Design: we will work in 120 villages, randomly divided into enough groups to test interven-tions against each other, and in combination.
What are the advantages of doing several experiments in one place?
What will the results from these experiments tell us about policy in Rajasthan? About the broader policy debate about whether one should invest in health care?
