Temperature, Data Quality, and Interviewer Incentives

Given the strong evidence that temperature affects data production, I now turn to in-terviewer incentives to examine whether inin-terviewers respond to temperature in a predictable manner. As mentioned in Section 2.2, DHS supervisor and field editor guidelines suggest that the quantity of data produced, especially in terms of number of interviews completed, is more easily observable to supervision than quality. Given that temperature may increase the disutility of effort or decrease the marginal benefit of effort in terms of productivity,

we may expect temperature to produce a differential change in productivity for quality vs.

quantity of data production.²⁴

I first examine the impact of temperature on the measure of productivity most ob-servable to supervisors: the number of completed interviews. Figure 2.6 gives the results from Equation 2.1 where the outcome variable is number of interviews completed in a day.

Interviewers do not seem to perform worse on this measure on very hot days; the coefficient on 85 degree wet bulb days is actually positive, though statistically insignificant. Given that interviewers are not paid by the hour, this suggests that from the employer’s perspective, productivity (at least, in terms of quantity of data produced) per dollar of wages does not decline on hot days. However, productivity per hour of the interviewer’s time declines, as shown in Figure 2.5, suggesting that interviewer welfare may be negatively affected by high heat through a loss of leisure hours. This is explored further in Section 2.5.3.

To further probe into whether productivity declines on less observable tasks, I use a framework similar to Equation 2.1 to examine whether measures of data quality respond to high temperatures. These regressions are run at the level of the individual interview. Figure 2.7 shows the results of regressions using counts of data quality flags and missing responses, respectively, as outcome variables. For both measures, the coefficients on the hottest wet bulb bins are positive and statistically significant, indicating these types of mistakes are more common on hot days. These two pieces of evidence together suggest that performance suffers more on hot days on measures that are less easily scrutinized by the supervisor.

24In Appendix B.2, I show a simple model that produces this prediction in the case where interviewers’

probability of job retention depends more on quantity of production than quality and temperature increases the disutility of effort.

Table B3 shows the results for the missing responses and quality flags variables shown in Figure 2.7, but also for counts of valid skips and counts of inconsistent or “I don’t know”

responses. The results on valid skips, displayed in Column 2, suggest a positive effect of both very cold and very hot temperatures on number of blank questions, although the impacts of hot temperatures are not statistically significant. The counts of don’t know and inconsistent responses do not show a clear pattern according to temperature on the day of interview, suggesting that respondents’ ability to respond correctly may not be significantly affected by wet bulb temperature.²⁵

Another feature of the interviewers’ incentive scheme that may affect the results is the fact that their main incentive to perform well, due to the fixed wage contract, comes from any risk of losing the job and the continuation value associated with it.²⁶ Interviewer jobs are temporary, so the continuation value of the job (i.e. the quantity of wages that would be lost if the interviewer were fired) should decrease as the survey round progresses, which could amplify any negative effects of heat on productivity. On the other hand, experience may be protective or may lead workers to learn to take more adaptive actions. Figure 2.8 examines how experience interacts with temperature by interacting each wet bulb bin with a measure of how many days the interviewer has worked on the survey round. The outcome variable is number of interviews completed per hour worked. In fact, the effect of hot weather significantly increases with experience.²⁷ As shown in Figure B6, the impact of experience

25However, examining the impact of dry bulb temperature on this outcome variable yields a positive and significant effect. This could be driven either by interviewers or respondents, since interviewers may have an incentive to write that the respondent said “I don’t know” in response to a question in order to lighten their workloads.

26 The model in Appendix B.2 additionally predicts that an increase in the continuation value of the job should be associated with a reduction in the negative effect of temperature on interviewer effort.

27There may be selection into levels of experience, if, for example, less experienced workers are more likely

at high temperatures is driven by working hours: more experienced interviewers are in the field for more hours on hot days. This could be consistent with a learning effect: more experienced interviewing teams learn to cope with the slowing pace of worker activity by working longer hours. Or, it could be that workers with lower continuation value of the job differentially allow themselves to slow down on hot days due to weaker incentives to perform well. On the other hand, the impact of experience on reactions to cold temperature are driven by the numerator: experienced interviewers conduct fewer interviews on cold days.