Randomised Control Trials Nouveau Impact Assessment Studies

In the last three years some researchers have moved away from conventional impact assessments and have begun using randomised control trials (RCTs) to assess the impact of microfinance. While in conventional impact assessments the researcher has no control over who gets into the intervention or the control group, RCTs enable the researcher to randomly assign people into either the control or the intervention group. RCTs are therefore able to overcome the perennial problem of identifying a suitable control group for comparison. Findings of RCTs are considered more credible because the researcher is able to control the research process and avoid the pitfalls associated with conventional impact assessments. What do RCTs tell us about the implementation processes and the impacts of microfinance interventions? Dupas and Robinson (2008) conducted a randomised field study in Kenya to test whether savings constraints prevented poor micro-entrepreneurs from increasing the size of their businesses. The researchers gave interest-free savings accounts to randomly selected daily income earners and collected self-reported log-books that respondents updated daily. The study found that owning a savings account was associated with substantial and positive impact on productive investment levels and expenditures for women but no measurable impact on men. The findings suggested that savings accounts enabled women to cope with household shocks without having to draw down their working capital.

Most impact assessments usually assess the entire microfinance intervention on poverty. Dupas and Robinson‘s study is one of the few that have focused on one component (savings) of the intervention. The use of self-reported data is a plus because it likely to provide useful insights from respondents which could not have been accessed in any other way. Like the other impact assessments examined in this thesis, the study by Dupas and Robinson (2008) does not investigate how the savings facilities are implemented by the village bank or provide

31 much contextual information which may have been insightful in understanding the findings of the study. For example why did a substantial proportion of entrepreneurs (40%) not use the savings account? What was entrepreneurs‘ perception of the savings facilities?

Two studies employing randomised control trials are examined next. Karlan and Zinman (2008a) used RCT to examine the impact of consumer loans in South Africa. The same authors Karlan and Zinman (2009) measured the impact of expanded access to credit on business growth and well-being in Manila. In South Africa respondents who had access to consumer loans were more likely to have kept their jobs, have significantly higher incomes and have a more positive attitude of the future. In the study in Manila it was found that increased access to microloans resulted in less investment in the targeted business (fungibility?). The authors contend businesses shrunk as a result of clients shedding unproductive workers. Treatment effects were found to be stronger for male and higher income earners, groups which were not normally targeted by micro-lenders. Microfinance was found to mainly work through risk management and investment at the household level rather than directly through targeted businesses.

As stated previously, the main difference between findings obtained through normal impact assessments and randomised control trials is that findings of randomised control trials are more credible. However, like other impact assessments, the studies by Karlan and Zinman fail to incorporate the intervention processes and the context in which the interventions occurred in order to interpret and understand the findings of the studies. For instance, contextual data would have explained why people who took loans in the South African study were more likely to have kept their jobs. It is possible that people who took loans in South Africa kept their jobs because if they had taken the risk of changing jobs and had been

32 rendered unemployed even just for a few months, it would have increased their chances of loan default. The clients of the two interventions are much wealthier than those found in typical microfinance settings. For example clients in the South African study were employees who earned average salaries of $3600 per year and could access loans from other banking institutions. The consumer loan scheme in South Africa cannot be strictly classified as a microfinance scheme as per the operational definition of microfinance for this thesis (see section 2.2). In addition the two studies were conducted in urban communities and do very little to enlighten us about how the interventions have fared in rural areas.

To conclude, it is evident that models of impact assessment reviewed focus on ensuring that their findings are relatively credible and valid. Except for the study by Johnson and Copestake (2002) none of the other studies reviewed had significantly considered the issue of context as an explanatory factor in their analysis. In addition if these studies are somehow representative of impact assessments, then it can be concluded that impact assessments pay little attention to how microfinance interventions are implemented. It is argued that intervention designs (which impact studies focus on) can significantly vary from how they are implemented (Matin and Hulme, 2003). This implementation gap can be very wide in developing countries.