Discussion of the systematic review

The dominance of household multi-indicator surveys that focus on mortality, morbidity and nutrition seems relevant to the need for planning and assessing humanitarian interventions in Darfur. Most of the studies targeted IDPs and the

host communities and to a lesser extent the refugees. This could be explained on a methodological basis, given that PPS, which is used in these surveys, focuses on larger groups, and the IDPs and host communities are larger than the refugee groups.

The studies included in this review were mostly conducted between 2004 and 2007, which is concomitant with the influx of INGOs into the region. This is similar to the findings of Degomme’s work, which also showed an average of five surveys per month between 2003 and 2008, reaching as high as 11 surveys in a single month (Degomme and Guha-Sapir, 2010). CEDAT reports more than 800 mortality and nutrition surveys conducted in Darfur between 2003 and 2012 (CEDAT, 2013). CEDAT also reports that in 2005 alone, more than 150 nutritional surveys and 50 mortality surveys were undertaken in Darfur. This could be attributed to the international attention given to the crisis in Darfur. The 2010 peak in the CRED reports could be a reporting peak rather than a true peak. The CRED’s archive depends completely on the voluntary sharing of reports. CRED officials mentioned that INGOs might share the reports of the previous few years all at once. Practically, this means that a higher proportion of the reports of an NGO at any given point in time may reflect the timing of commitments to sharing their reports, rather than their being the most active organisation during that time.

As would be expected, most of the surveys were done by humanitarian agencies. For example, two-thirds of the included studies from the online search (51; 75%) and most of the CRED studies (119; 86.2%) were conducted by UN agencies and/or other INGOs. It is also understandable that the proportion of

INGO reports is higher in the CRED search than in the online search, because the CRED’s archive depends almost entirely on the INGOs’ contribution.

The contribution of NNGOs both in the online search (2; 2.9%) and in the CRED search (2; 1.4%) is too low to be explained by the general difficulty in accessing the reports of the included studies. This could reasonably be explained by the NNGOs’ limited research undertakings, which could in turn be an area that needs further research. It might reasonably be expected that there would be a greater contribution from the NNGOs, which are supposed to be working in these conflict-affected areas and should have done assessments like those of the INGOs. Given that the results reflect a significantly smaller contribution of NNGOs in the studies, it is important to study the reasons for this to work on improving this contribution.

The online studies that were published in peer-reviewed journals (26, 34%) were also concerned with the humanitarian impact of the conflict, as well as other specific conditions like hepatitis, malaria, and HIV/AIDS (Boccia et al., 2006; Guthmann et al., 2006; Yousif, Mansour and Ateem, 2009) and genetics (Nicand

et al., 2005). Unsurprisingly, as RCTs require stability and logistics that were not

available in Darfur, there was little RCT activity, with only one retrievable RCT conducted on Darfur refugees in Cairo (Meffert and Marmar, 2009). Moreover, RCTs are unlikely to be methodologically useful to answer the kinds of research questions raised in conflict settings.

7.5.2 Sampling, type of data collected and data collection methods

Multi-stage cluster population-proportional sampling (MSPPS) is considered the standard sampling technique for household surveys (Bostoen et al., 2007; Checchi et al., 2007; Galway et al., 2012) and is usually employed by the UN and INGOs to ensure that they have a sample as representative as possible of the whole population. The difficulty that this method encounters in conflict situations is the inaccessibility of some clusters due to insecurity (Hussein, 2006; Olivia Lomoro et al., 2007), which also explains the use of convenience or targeted sampling as an alternative.

MSPPS was the sampling method used most in CRED surveys (135; 97.8%) and online (50; 73.5%).

The use of FGDs was mentioned more in the online studies (23, 33.8%) than in the CRED studies (10; 7.2%). Similarly, taking biosamples (mainly blood (whole or serum) (11; 16.2%), and urine/stool (3; 4.4%)) was mentioned more in the online studies (15; 22.1%) than in the CRED studies (2; 1.4%) (Table 7-2). These differences probably reflect the nature of the studies and the target publication site. The collection of biosamples is usually used in studies that involve clinical aspects and not purely epidemiological studies. The CRED reports were all on epidemiological studies.

The most commonly used data collection tools in both the online and CRED searches were questionnaires and interviews (including verbal autopsy), which are the most methodologically relevant tools given the aims of the retrieved studies.

As most of the included reports were household studies that looked for epidemiological indicators (e.g. mortality) within a given recall period, questionnaires are the best tool to collect large amounts of data in a relatively short time. Focus group discussions were used in the studies that aimed at objectives beyond the calculation of a given epidemiological rate, such as mental health issues (Rasmussen et al., 2010) and vulnerability to sexual assault (Deribe

et al., 2011).

The use of anthropometric measures, including measurements of the height and weight of children under five years old, was mentioned in most of the studies retrieved from the CRED search (128; 92.8%). About one-fifth of the studies retrieved from the online search (15; 22.1%) stated that they collected biological body samples. Moreover, in three of the retrieved studies, the biosamples were sent to European laboratories, and nothing is mentioned about whether the patients consented to this exportation or the further use of their samples.

7.5.3 Mention of ethical review

As none of the CRED studies mentioned their ethical approval status, the following focuses only on the studies retrieved from the online search. Most of these studies (59; 86.6%) did not mention whether they had obtained ethical approval and none of the studies found in the CRED did. This does not necessarily mean they did not seek ethical review or that they were not ethically approved. There are a number of reasons why a study’s ethical review status would not be mentioned. These possibilities are discussed below and form the basis of the hypotheses that were studied later in the project. Though one of the

aims of contacting the authors of the retrieved reports was to explore why the studies’ ethical review statuses were not mentioned; the limited feedback received from the authors cannot be used as the basis for a generalizable conclusion.

7.5.4 Possibilities to explain the lack of mention of obtaining ethical approval

7.5.4.1 Possibility one: These studies were held to be exempt from ethical review

This possibility is supported by a statement that was found in one of the studies included in this review. The Crude Mortality Survey that was led by the WHO and jointly conducted by other UN agencies and the GoS states that “WHO guidelines do not require ethical review for retrospective surveys during humanitarian emergencies …” (World Health Organization and Federal Ministry of Health, 2005a, p. 7 of 7).

This statement clearly outlines that WHO surveys during emergencies do not require ethical approval, yet the justification for this exemption from ethical review can be questioned. One of the authors who completed the online form mentioned that ethical approval was not sought because

“[E]mergency survey[s] [...] done for primarily operational programming reasons – no time to seek review; WHO guidelines suggested that emergency surveys not involving the collection of biosamples could be exempt from IRB review” (Participant 1).

However, it is not possible to draw a firm conclusion from a statement mentioned in one study or by one participant. Other reasons for exemption from

ethical review were set by the MSF ethics committee for the MSF REB for retrospective analysis of previously collected clinical data (subject to various protections) (MSF Ethics Review Board, 2013b). This exemption is unlikely to have been applied in the studies included in this systematic review, whose main inclusion criterion was the prospective collection of personal data and/or biosamples.

7.5.4.2 Possibility two: Mentioning ethical review was not required

Most of the studies retrieved from the online search (39; 57.4%) and almost all of those retrieved from the CRED search (128; 92.7%) were conducted by humanitarian aid agencies. Given the difficulties faced by these agencies during conflicts, their focus is on sharing the findings of their surveys. Perhaps it is less relevant to share details about gaining ethical approval or seeking participants’ consent. However, even if this is true regarding the studies published in disaster- specific databases, where mentioning ethical review or consent is not a requirement for publishing a study report, it should not apply to the one-third of studies (26, 34%) that were published in peer-reviewed journals. In the latter, “authors should indicate whether the procedures followed were in accordance with the ethical standards of the responsible [ethics review] committees” (International Committee of Medical Journal Editors, 2013). The discrepancy between the 34% of studies that were published in peer-reviewed journals and the 13.2% that mentioned being ethically reviewed could be interpreted in three ways. It could either suggest that studies were published in peer-reviewed journals without stating their ethical approval status, or the published study did not require ethical approval as judged by the editors of the relevant journal. Lastly, it is possible that

the submission process required a statement but that this statement was not then published.

7.5.4.3 Possibility three: Ethical review was considered as if granted

This possibility assumes that the studies done by the organisation were assumed to have been ethically approved, and so the researchers did not have to mention this approval in their published reports. This possibility could be supported by other findings in the present study. For example, the only studies that were ethically approved by an NGO committee were those reviewed and approved by the MSF ethics committee. However, MSF has criteria for exempting some types of (retrospective) research from ethical review (MSF Ethics Review Board, 2013a). Thus, those who were exempted from ethical review would not mention obtaining ethical approval, and only those who required ethical approval would mention it. This variation in the initial requirement for ethical approval could explain why some of the MSF studies reported their ethical review while others did not.

7.5.4.4 Possibility four: Pre-approved proposals

This is an alternative approach to ethical review that is based on ethically reviewing and approving ready-made generic study protocols of emergency research when the research needs to be conducted in an urgent and timely manner, i.e. where it cannot wait for a full ethical review. This approach has been suggested by a WHO Technical Group (2009) for research to be conducted during pandemics (Group, 2009) and has been adopted by the MSF ethics committee (MSF Ethics Review Board, 2013a). In the context of this systematic review, this could mean that one or more of the included studies was viewed as ‘emergency

research’ that was part of wider research whose protocol had previously received ethical approval. There is no finding in this review to support this possibility.

7.5.4.5 Possibility five: The ethical review was not part of the template used At least for the studies retrieved from the CRED search, the patterns and formatting used for reporting were very similar, as though they used a common template. These similarities applied to the methodologies and the reporting of the results. For example, multi-stage cluster sampling was used by almost all of the CRED studies reviewed (137; 99.3%) and more than half of those found online (36; 52.9%). Moreover, many of the reports used exactly the same wording to describe the sampling procedure.

This possibility can also be supported by the finding that the studies conducted by one INGO mentioned ‘Ethical Considerations’ using exactly the same words and structured under exactly the same bullet numbering (Relief International, State Ministry of Health and UNICEF, 2011; State Ministry of Health, Relief International and UNICEF, 2011). This is particularly significant if other INGOs also use a template. Theoretically, changing the template that such organisations use to report their studies may change the extent of inclusion of ethical considerations in future studies. For example, if a template included a section on ‘informed consent’ or ‘ethical approval’, then those using it would be likely to include more details about these aspects.

7.5.5 Mention of consent

In the online search, more studies mentioned that they obtained consent (29; 43%) than mentioned that they had been ethically approved (19; 13.2%). More

studies in the online search than in the CRED search (17; 12.3%) mentioned obtaining consent. The former finding could be partially explained by the fact that most of the included studies were household-based studies which used similar methodologies that were described in detail and made available to humanitarian aid workers to use (Rose et al., 2006; World Relief and State Ministry of Health, 2010). These methodologies are described in common guides used by the researchers in these agencies. These guides usually mention a section on ‘informed consent’ under the ‘methodology’ section, so those who use these templates consider obtaining consent a part of the methodology. This assumption could be supported by the finding that some commonly used template guides mention obtaining consent from participants without mentioning other issues related to ethical review (Rose et al., 2006; Humanitarian Accountability Parternership, 2010). Therefore, those who follow these guides would only mention what these guides contain, which is consent and not ethical approval.

As might be expected, most participants’ consent was obtained verbally (18; 27%), which is more feasible than obtaining written consent, given the culture of Darfur where people do not like to or cannot sign papers.

The finding that consent was mentioned in more of the online studies than the CRED studies has two possible explanations. First, there is more variation in the studies found in the online search, which included publications in peer- reviewed journals in addition to epidemiological field reports. It is more likely to find consent mentioned in an article published in a peer-reviewed journal than in household surveys that are mainly shared for their epidemiological findings.

Second, most of the CRED studies were produced by a relatively limited number of organisations whose main interest is the field-related details, namely the results and survey methods. In contrast, the online studies included studies done for non- humanitarian purposes by non-humanitarian researchers who may follow different reporting formats. Also, the NGOs might have used template guides of survey methodologies that did not include or did not emphasise the mention of consent. For example, consent is built into the first part of the standard survey template and is considered a routine that it is not considered worth mentioning.

7.5.6 Characteristics of the studies that mentioned being ethically approved

The nine studies that mentioned receiving ethical approval do not have much in common with one another and reflect the general trend of the other studies, with the exception that eight of them were published in peer-reviewed journals (Grandesso et al., 2005; Kim, Torbay and Lawry, 2007; Souza, Yasuda and Cristofani, 2009; Elfatih et al., 2010; Hagan, Brooks and Haugh, 2010; Deribe et

al., 2011; Badri, Crutzen and Van den Borne, 2012; Tsai et al., 2012; Trani and

Cannings, 2013). However, there are three points worth noting.

First, MSF’s procedure for the ethical review of its field surveys (ERB) is the only INGO ethics-related oversight mechanism mentioned in the studies found in this review. Other NGOs might have their own ethics committees and procedures, but they were not mentioned in the included studies.

Secondly, it appears that only two studies were ethically approved in Sudan. One was reviewed by a Sudanese university’s ethics committee (Badri, Crutzen

and Van den Borne, 2012), while the other was the only study that was reviewed and approved by the NREC (Elfatih et al., 2010). Both committees are in Khartoum, not Darfur. Sudanese national guidelines require all studies that involve humans that are “linked to external bodies and that take place in more than one state (Interstate)” (National Ministry of Health, 2008, p. 23) to be reviewed by the NREC. This single study fulfils this condition, but most of the other studies were also done in more than one state and should have been submitted to the national committee, if the national guidelines had been followed.

The findings on the mention of consent and ethical review were key elements of the interviews and the discussions with the representatives of the NGOs and the governance bodies.