R E S E A R C H A R T I C L E
Open Access
Systematic reviews on selected nutrition
interventions: descriptive assessment of conduct
and methodological challenges
Rehana A Salam
1, Vivian Welch
2and Zulfiqar A Bhutta
1,3*Abstract
Background:Rigorous and transparent systematic reviews are recognized internationally as a credible source for evidence of effectiveness. However, in the field of nutrition, despite attempts at developing consensus on actions and interventions to reduce undernutrition and micronutrient deficiencies, there is lack of coordination among various groups.
Methods:The aim of this overview of systematic review is to assess the process and conduct of systematic reviews published in the past 5 years to make recommendations on improving process and methodology of systematic reviews in the field of nutrition. We identified nine interventions from four areas of nutrition through a consultative process and conducted a comprehensive search to identify systematic reviews on the selected interventions published in the last 5 years.
Results:We identified 90 systematic reviews across these nine intervention areas. The median overall Assessment of Multiple Systematic Reviews (AMSTAR) score was 8 (range 2–11) with methodological quality of Cochrane reviews being fairly consistent with a median AMSTAR score of 10 (range 8–11), while for the non-Cochrane reviews, it ranged from 2 to 11 with a median of 7.5. From the 11-point AMSTAR criteria, 91% of the reviews followed an a priori design, 81% did duplicate screening and data extraction, 88% conducted a comprehensive search, 64% used status of publication as an inclusion criteria, 44% provided the list of included and excluded studies, 60% assessed and documented the scientific quality of the included studies and used it in interpreting the results, 61% used appropriate methods to combine the results, 40% assessed publication bias and 82% stated the conflict of interest. We found considerable variation in methodological quality, lack of standardization of outcomes, lack of
standardized assessment of risk of bias of included studies, variation in study designs included and variation in how heterogeneity was handled.
Conclusions:Each of these methodological choices influences the findings of the reviews, and lack of standardization across these domains increases the complexity for users of systematic reviews in interpreting results. There is a need to develop a consensus on methodologies for nutrition reviews, criteria for assessing the evidence and possibly facilitating development and collation of the evidence in the subject area.
Keywords:Systematic reviews, Nutrition, Methodological assessment
* Correspondence:[email protected] 3
Robert Harding Chair in Global Child Health and Policy, Centre for Global Child Health, The Hospital for Sick Children, Toronto, ON, Canada 1
Center of Excellence in Women and Child Health, Division of Woman and Child Health, Aga Khan University, Karachi 74800, Pakistan
Full list of author information is available at the end of the article
© 2015 Salam et al.; licensee BioMed Central. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Salamet al. BMC Nutrition (2015) 1:9
Background
To reach the targets set for Millennium Development Goal (MDG) 1, which is to reduce extreme poverty and hunger by half, many countries are on-track in reducing the income poverty while less than a quarter of the de-veloping countries are on-track for achieving the goal of halving undernutrition [1]. The global burden of under-nutrition remains high with little evidence of change in many countries despite economic growth. Millions of people are faced with starvation and malnutrition; the State of Food Insecurity (SOFI) in the World 2012 esti-mated that about 870 million people have been under-nourished (in terms of dietary energy supply) in the
period 2010–2012, mostly from low-middle-income
countries (LMIC) [2]. About one third of deaths in chil-dren under 5 years of age are due to underlying under-nutrition, which includes stunting, severe wasting, deficiencies of vitamin A and zinc and suboptimum breastfeeding [3]. The majority of an estimated 178 million children under five suffering from stunting are in sub-Saharan Africa and South-central Asia [4]. Around 55 million children are wasted, 19 million of these are severely affected and at high risk of premature death [4]. Altogether, more than 3.5 million mothers and children under five die unnecessarily each year in LMICs due to the underlying cause of undernutrition [5]. With the existing burden of undernutrition, the problem of overweight and obesity among children is also on the rise. In 2011, globally, an estimated 43 million (7%) chil-dren younger than 5 years were overweight, an increase from an estimated 28 million in 1990. This trend is expected to continue and reach a prevalence of 9.9% (64 million) in 2025 [6]. Many of the LMICs are said to have
the double burden of malnutrition—continued stunting
of growth and deficiencies of essential nutrients along with the emerging issue of obesity.
Although there have been several attempts at develop-ing consensus on actions and interventions for nutrition, there is lack of coordination between various academic groups, United Nations (UN) and development agencies, and these are working in parallel silos. The Lancet Nutrition series in 2008 [7-11] and more recently in 2013 [6,12-14] are important attempts at bringing the field together; however, progress by countries has been slow and also hampered by the global increase in food and oil prices; climate change; unprecedented draughts and increased number of countries affected by fragility, conflict and emergencies. As per the World Bank in 2011, 33 countries fall in the fragile situations category as per the World Bank definition [15]. In addition, con-flict and fragility also occur at the sub-national level within some strongly performing countries.
Systematic reviews and meta-analysis have become increasingly popular in evidence-based healthcare over
the past two decades [16], although these have been evaluated for reliability since quite a long time [17-20]. Rigorous and transparent systematic reviews are recog-nized internationally as a credible source for evidence of effects and as the basis for evidence-informed policy and decisions [16,21,22]. Within the field of nutrition, sys-tematic reviews have been used to answer many complex questions. A description of systematic review method-ology with particular regard to the steps for performing systematic reviews and areas unique to the discipline of nutrition is already available, published by the US Agency for Healthcare Research and Quality (AHRQ) [23]. Furthermore, guidance on assessing equity for users and authors of systematic reviews of interventions has also been highlighted recently to improve the relevance of systematic reviews for both clinical practice and pub-lic popub-licy-making [24]. However, several gaps pertaining to the conduct and methodological approaches still exists particularly with reference to nutrition interven-tions implemented through sectors other than health, such as agriculture, social welfare and trade.
Across nutrition-related systematic reviews, there is multiplicity of reviews in the same subject area with variability in methods such as intervention, target popu-lation comparison, outcomes and consideration of con-textual variables related to nutritional interventions and outcomes. Additionally, with the increasing attention to social determinants, there is a need to bring in add-itional qualitative aspects to such reviews. The World Health Organization (WHO) has led the field with a series of activities geared towards development of nutri-tion policy guidelines based on the best evidence. In 2011, a new online resource, the electronic Library of Evidence on Nutrition Actions (eLENA), was launched that provides access to available sources of evidence and to WHO guidelines, as well as to ongoing clinical trials in the field of nutrition [25].
This overview of systematic reviews from the selected areas in nutrition was produced as a background docu-ment for a WHO-convened meeting on improving the quality of systematic reviews of nutrition [26]. The aim of this paper is to assess the process and conduct of systematic reviews (including systematic reviews and systematic reviews + meta-analysis) published in the past 5 years to make recommendations on improving process and methodology of systematic reviews in the field of nutrition.
Methods
Children’s Emergency Fund (UNICEF), World Food Pro-gram (WFP) and Food and Agriculture Organization (FAO); International Food Policy Research Institute (IFPRI); and interested bilateral agencies including the Department for International Development (DFID), United States Agency for International Development (USAID), Norwegian agency for International Develop-ment (NORAD) and representatives from the Bill & Melinda Gates Foundation to identify nine focused inter-ventions under four umbrella areas for our review. These areas were selected based on consultations and WHO interests and included preventive (interventions to pre-vent undernutrition and micronutrient deficiencies), therapeutic (interventions to treat undernutrition and micronutrient deficiencies), food fortification strategies and delivery platforms (Table 1).
We defined systematic review according to the
‘Cochrane Handbook of Systematic Reviews’ [27] as ‘A systematic review attempts to collate all empirical evi-dence that fits prespecified eligibility criteria in order to answer a specific research question’. Meta-analysis was defined as ‘the use of statistical methods to summarize the results of independent studies’. An overview of sys-tematic reviews was defined as ‘Systematic and explicit methods to identify select and critically appraise rele-vant findings from systematic reviews and/ or meta-analysis of effectiveness of interventions’. We included systematic reviews of trials as well as observational studies. We considered all available systematic reviews and overviews of systematic reviews on the predefined interventions. A separate search strategy was developed for each intervention using preidentified Medical Sub-ject Headings (MeSH) and key search terms. A search was conducted in the Cochrane Library, Campbell Col-laboration, PubMed and Google Scholar to identify sys-tematic reviews in each of the nine intervention areas. Additional reviews were identified by hand searching
references from included reviews. A priori inclusion cri-teria were outlined stated as below:
Systematic reviews pertaining to effectiveness or efficacy of nutrition-related interventions, assessed by prospective studies (including both trials and observational studies), published between 2005 and 2011 were included.
Overviews of systematic reviews were also included if they were found to be relevant.
Reviews of all languages with English translations were considered eligible for inclusion.
Inclusion was not restricted to any particular reported outcome.
Reviews that met the inclusion criteria were selected and double data abstracted on a standardized abstraction sheet for source journal, funding, authors, year, title, tar-get population, intervention, study setting (high-income countries or low-middle-income countries) and com-parison, outcomes, types of included studies (random-ized trials, quasi trials, before-after, etc.), gaps in knowledge base (as identified by the author), and con-clusions. Any disagreements on selection and abstraction of reviews between the two primary abstractors were resolved by the third reviewer. We also assessed each systematic review according to the components from the Cochrane Public Health Review Group guidance includ-ing study design; searchinclud-ing non-health databases; asses-sing risk of bias, ethics and inequality; context; sustainability and applicability [28]. For each systematic review, we assessed the quality of the review by the Assessment of Multiple Systematic Reviews (AMSTAR) rating consisting of 11 items which has been demon-strated to have good face and content validity for meas-uring the methodological quality of systematic reviews [29]. We then compared the reviews within the same
Table 1 Topic areas of landscape study
Topic area Interventions
Preventive nutrition interventions (interventions to prevent undernutrition and micronutrient deficiencies)
Preventive zinc supplementation Breast feeding
Complementary feeding
Preventive multiple micronutrient (MMN) supplementation Therapeutic nutrition interventions (interventions to treat undernutrition and
micronutrient deficiencies)
Undernutrition/malnutrition [ready-to-use therapeutic food for community management of severe acute malnutrition] Therapeutic zinc supplementation
Therapeutic MMN supplementation Fortification strategies (deliberately increasing the content of an essential
micronutrient in food irrespective of whether the nutrients were originally in the food before processing or not, so as to improve the nutritional quality of the food supply)
Single nutrient fortifications (Folic Acid)
Delivery platforms (specific modes and channels of delivering interventions) Conditional cash transfers
domain with respect to their journals, funding, research question, included study designs and meta-analysis, reporting tools, and definition and assessment of outcomes.
Results
Our search identified 10,926 titles, of which 217 full texts were reviewed and 90 systematic reviews across the nine content areas were included (Figure 1). All eligible texts were available in English, and there was no exclu-sions based on language. We did not include any unpub-lished or grey literature review. A wide range of groups were responsible for commissioning and funding the reviews including governments, non-governmental orga-nizations (NGOs), various institutions, UN agencies and private foundations (Table 2). Sixty-eight percent (53/78) of the systematic reviews from the selected nutrition domains were produced from authors based in the high-income countries (HIC) with major contributions from USA, UK, Canada and Australia; thirty-two per-cent (25/78) of the reviews had authors from the LMICs with Pakistan (13/25) being the major contributing site (Figure 2). From the illustrative areas reviewed, the journal publishing the most systematic reviews in the last 5 years in these nine intervention areas was the Cochrane Database of Systematic Reviews (35/90), followed by BioMed Central (7/90), ‘Journal of Nutrition’ (4/90),
‘The Lancet’(4/90),‘American Journal of Clinical Nutrition’ (4/90),‘Food and Nutrition Bulletin’(3/90) and‘Journal of
American Medical Association’ (3/90) (Figure 3). The
included reviews, and overviews of reviews are summa-rized using the Cochrane public health guideline grid (Table 3). The assessment of the quality of systematic re-views using the AMSTAR rating, and further description of the attributes of the systematic reviews and overviews, are provided below.
Methodological quality-AMSTAR rating
The median overall AMSTAR score was 8 (range 2–11)
with methodological quality of Cochrane reviews being fairly consistent with a median AMSTAR score of 10
(range 8–11), while for the non-Cochrane reviews, it
ranged from 2 to 11 with a median of 7.5 (Figure 4). From the 11-point AMSTAR criteria, 91% (82/90) of the reviews followed an a priori design, 81% (73/90) did du-plicate screening and data extraction, 88% (79/90) con-ducted comprehensive literature search, 64% (58/90) used status of publication as an inclusion criteria, 44% (40/90) provided the list of included and excluded stud-ies, 34% (31/90) provided the list of included studies only, 74% (67/90) provided characteristics of the included studies, 60% (54/90) assessed and documented the scientific quality of the included studies and used it in interpreting the results, 61% (55/90) used appropri-ate methods to combine the results, 40% (36/90) assessed publication bias and 82% (74/90) stated the conflict of interest.
Research question
The research question guides the systematic review and is probably the most important step that defines the population, the intervention, the comparison and the outcomes of interest. We compared research questions for reviews identified on the same intervention to assess for overlap of questions. Although we found several re-views for the same intervention, each review was differ-ent because of differences in aspects of the population, intervention, outcome or comparison. For preventive zinc supplementation, two of the included reviews tar-geted pregnant women, seven tartar-geted children under 5 years of age, one targeted adults while one targeted general population. Besides differences in the target population, reviews evaluated various outcomes; five
reviews evaluated childhood infections [30-34], of which three focused on pneumonia alone [31-33]; one on re-spiratory illness and diarrhoea [30] and one on diar-rhoea, pneumonia and malaria altogether [34]. Similarly reviews on multiple micronutrient (MMN) supplementa-tion targeted a variety of age groups including pregnant and women of reproductive age, children, healthy adults, and HIV-infected women and children. Table 4 reports the population, topics and outcomes reported in the in-cluded reviews.
Included study designs
Good quality randomized controlled trials (RCTs) are considered as the highest quality evidence because they minimize the influence of bias in their results; however,
Category Countries ( number of reviews)
Lower Income Countries Ethiopia (1)
Lower Middle Income Countries India (4), Nigeria (2), Pakistan (13)
Upper middle Income Countries Serbia (1), South Africa (1), Argenna (1), Thailand (2), Brazil (1)
High Income Countries New Zealand (1), Italy (1), Denmark (3), Switzerland (1), Poland (2), Norway (2), Canada (8), Australia (7), UK (13), USA (14)
Figure 2Countries of the lead authors producing reviews. Table 2 Commissioning and funding of the reviews
Organizations
Government HTA program-HCA; the National Institutes of Health for the prevention of childhood disease; Canadian Institutes of Health Research; United States Agency for International Development Office of Foreign Disaster Assistance; Research Council of Norway; Department of Health and Ageing-Australia; USAID; The Department for International Development (UK); UK Department for International Development (DFID) for developing countries; Concern Worldwide; Ministry of Science and Technological Development, Serbia; Food Standards Australia New Zealand; Norwegian Agency for Development Cooperation (NORAD)
UN agencies WHO, World Bank, UNICEF, World Food Program, United Nations System Standing Committee on Nutrition (SCN) Private
foundations
Bill & Melinda Gates Foundation, The Richard and Rhoda, Goldman Fund
NGOs ICDDRB (Bangladesh); The Nutricia Research Foundation; International Initiative for Impact Evaluation; German Technical Co-operation (GTZ), Germany; Aubrey Sheiham Public Health and Primary Care Scholarship-UK; International Nutrition Council Institutions The Alberta Heritage; Medical University of Warsaw; The Monsanto Fund; Institute of Child Health, London; Royal College of
Paediatrics and Child Health-UK; The Copenhagen Trial Unit; Harvard Medical School
RCTs cannot always be relied upon for conducting sys-tematic reviews as such robust study designs account for only 10% of the healthcare literature [35,36]. Within the selected nutrition interventions, we found that 53% (48/90) of the reviews included RCTs while the others included quasi and observational studies along with RCTs. Most of the Cochrane reviews (73%) included RCTs and quasi ran-domized studies; however, for most of the non-Cochrane reviews and some specific interventions like breastfeeding, fortification and management of malnutrition, data from pre-post designs were also included.
Quality assessment of the included studies
Table 3 specifies the quality assessment tools used by the reviews in each domain. All of the Cochrane reviews used the Cochrane risk of bias tool to assess the methodological quality of included studies, while some non-Cochrane re-views used various quality assessment scales including the CONSORT checklist, Jadad scale, GRADE quality assess-ment, US Preventive Task Force Quality Rating and Criteria by Glasziou et al. For fortification reviews, methodological quality was assessed using The Food Standards Australia New Zealand (FSAN) framework for assessing evidence when substantiating nutrition, health and related claims on foods. Some reviews mentioned conducting a quality assessment; however, the methodology used was not clear, while information on methodological quality assessment was found missing or unclear in a few reviews. The over-view of the reover-views used an informal method of quality as-sessment that is the Oxman et al. 1991 criteria.
Outcomes standardization and assessment
Standardized, valid outcome measures are pivotal in interpreting findings of systematic reviews, as intervention
effectiveness may vary depending on the outcome meas-urement tools. For example, the specificity of case defini-tions for acute respiratory infecdefini-tions (ARI) substantially influenced inferences for the effectiveness of preventive and therapeutic interventions [33,37]. Various definitions have been used for defining ARI; however, in some re-views, we found that standardization measures were taken by restricting the inclusion criteria to studies in which ARI case definition aimed to distinguish lower from upper tract respiratory disease; various definitions were also scored for standardization purposes in one of the reviews [33]. Furthermore, despite showing an overall effectiveness of preventive zinc supplementation on pneumonia pre-vention, the impact changed from statistically significant to non-significant when the definition of pneumonia
changed from ‘pneumonia defined by specific clinical
criteria’ to‘lower specificity pneumonia case definition (i.e. age specific fast breathing with or without lower chest in-drawing)’ [31,33]. Few other reviews reported considerable variation in outcome definition across studies, for example definition of exclusive breastfeeding and mixed breast feeding [38], definitions of diarrhoea [30,34] and the definition of health care workers delivering the interventions [39]. In areas where WHO standard defi-nitions exist like undernutrition, reviews have used stand-ard WHO definitions [40-42], while for some reviews, reviewers have self-defined the outcome for standardization [43,44]. Forty three of the included reviews considered contextual factors and conducted sensitivity and subgroup analysis to adjust the outcomes according to these factors.
Meta-analysis and heterogeneity
Table 3 Cochrane public health guideline grid
Methods Data from studies Public health guidance
Preventive and promotive interventions (n= 50) Systematic reviews: 12Systematic review + meta: 36Overview: 2
Therapeutic nutrition interventions (n= 34) Systematic reviews: 10Systematic review + meta: 24Overview: 0
Fortification (folic acid) (n= 2) Systematic reviews: 0Systematic review + meta: 2Overview: 0
Delivery platforms (CCT) (n= 4) Systematic reviews: 3Systematic review + meta: 0Overview: 1
Study designs (primary studies)
RCTs: 25 reviewsMixed (RCTs, quasi and other designs): 25 reviews
RCTs: 23 reviewsMixed (RCTs, quasi and other designs): 11 reviews
RCTs: 0 reviewsMixed (RCTs, quasi and other designs): 2 reviewsOverview: 0 reviews
RCTs: 0 reviewsMixed (RCTs, quasi and other designs): 4 reviews
RCTs may have limited
generalizability or not be feasible or ethical. Non-randomized trials or cluster RCTs may be more appropriate for the question
Searching non-health databases, number seen out of total (range)
6 (1–5) 11 (1–2) 0 3 (3–5) Public health may require range of databases and detailed text words because terminology is imprecise
Assessing risk of bias Tools used:Cochrane risk of bias: 17GRADE: 5Jadad: 1CONSORT: 1Other tools: 18Not stated: 8
Tools used:Cochrane risk of bias: 14GRADE: 0Jadad: 0CONSORT: 0Other tools: 19Not stated: 1
Tools used:Cochrane risk of bias: 0GRADE: 1Jadad: 0CONSORT: 0Other tools: 1
Tools used:Cochrane risk of bias: 2GRADE: 0Jadad: 0CONSORT: 0Other tools: 2
Appraisal criteria depend on the type of study in the review. Authors should use Cochrane risk of bias tool for RCTs
Ethics and inequalities
HIC: 9LMIC: 21Both: 20 HIC: 8LMIC: 15Both: 11 HIC: 1LMIC: 0Both: 1 HIC: 0LMIC: 2Both: 2 Public health interventions may unintentionally increase health inequalities if the more advantaged benefit more than the worst off. Authors should consider whether populations are likely to be disadvantaged across PROGRESS+ acronym factors (place of residence, race or ethnicity, occupation, gender, religion, education, socioeconomic position (SES) and social capital
Context (explore context as an effect modifier)
29 13 1 0 Public health interventions depend on social, economic and political
context in which they are developed and implemented. Context should be explored as an effect modifier (e.g. host organization, target population)
Sustainability (sustainability was assessed or described)
11 6 0 4 Consider the extent to which the intervention’s effects on outcomes are
sustained over time, e.g. by assessing economic and political variables, strength of implementing institutions, integration of services into existing services, training component, community involvement
Applicability (conducted subgroup analysis to assess applicability to different settings)
29 13 1 0 Authors are well-positioned to collect data needed to make
judgments about applicability to different settings, including political environment, social norms, cultural preferences, resources required, educational level of target population, organizational capacity to deliver the intervention, structural barriers to intervention, health care providers have capacity to deliver this intervention
HIChigh-income countries,LMIClow-middle-income countries,RCTrandomized controlled trials.
Salam
et
al.
BMC
Nutrition
(2015) 1:9
Page
7
of
hence, it is challenging to pool data for meta-analysis which leads to a lack of consensus on effectiveness and contradicting findings. Almost all the reviews reported heterogeneity in the data. Out of 90 included reviews, 62 were systematic reviews with a meta-analysis, 25 were sys-tematic reviews without meta-analysis, while almost 11% (10/90) of the systematic reviews reported heterogeneity as a reason for not conducting meta-analysis. The hetero-geneity reported was both clinical as well as statistical due to diverse interventions, target population and outcome definitions. None of the reviews conducted meta-analysis using individual patient data (IPD).
Discussion
Our overview findings pertain to the process and con-duct of systematic reviews and not individual studies since we did not assess the primary studies. Our findings suggest that from the 90 nutrition-related reviews, we found considerable variation in methodological quality as assessed by the AMSTAR (ranging from 2 to 11), lack of standardization of outcomes, lack of standardized as-sessment of risk of bias of included studies, variation in study designs included and variation in how heterogen-eity was handled. Each of these methodological choices influence the findings of the reviews, and lack of standardization across these domains increases the com-plexity for users of systematic reviews in interpreting results—sometimes of conflicting reviews. None of the included meta-analyses used IPD, which provides more detailed analyses and a broader methodological scope due to the presence of raw data. Our analysis shows that these nutrition reviews are comparable to a random sample of systematic reviews across all topics. However, the sample of nutrition reviews had considerably more reviews led by an LMIC author, 32% compared to the 2004 sample where <20% of all reviews had the contact author outside of USA, UK, Canada, New Zealand and Germany [45]. Systematic reviews and meta-analysis have become increasingly popular in evidence-based healthcare over the past two decades [16-20]. The
selected systematic reviews on nutrition came from vary-ing sources as the nutrition-related interventions are not solely implemented by health sector. There is an evident gap of quantitative data in some major areas concerning nutrition like undernutrition that direly needs policy guidance. We found considerable overlap between the topics and questions addressed in multiple reviews; how-ever, the outcomes reported varied.
Overall, the methodological quality of the Cochrane reviews was found to be better, which is not surprising since they are conducted according to standardized guidelines published in the‘Cochrane Handbook’. How-ever, within the Cochrane Collaboration, several editorial groups have undertaken nutrition-related reviews, and while the broad methods are consistent, there is variabil-ity in nutrition-specific decisions about the methods such as eligibility criteria for population, intervention, comparison and outcomes and consideration of context-ual variables. Furthermore, in some areas like condi-tional cash transfers (CCTs), the scope is very broad and thus fails to focus on nutrition-specific indicators. There is a need for improved justification of methodological choices in nutrition systematic reviews. We propose the need to follow standards for conducting systematic reviews such as the ‘Cochrane Handbook’ [46], as well as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting systematic reviews and meta-analyses [47]. With respect to public health nutrition, there is a need for standardization in defining of the population, interven-tion, comparison and outcomes, as well as a need for at-tention to aspects of context, appropriate study designs, sustainability and applicability. For outcomes, there is a need to standardize the measurement and definition of outcomes. The Core Outcome Measures in Effectiveness Trials (COMET) initiative is a major opportunity to explore this for nutrition. The differences in methods used are consistently observed in both Cochrane and non-Cochrane reviews, suggesting a need for improved guidance for nutrition reviews.
0 2 4 6 8 10 12
Cochrane Non-Cochrane
AMST
AR RAT
ING
Table 4 Population, topics and outcomes reported in the included reviews
Intervention N (Cochrane/ non-Cochrane)
Population Topics Outcomes Setting
Preventive zinc 14 (7/7) Pregnancy (2)Children <5 (7) Adult (1)General population (1)
Supplementation during pregnancy and outcome assessment: 2
Supplementation to children under five and impact on growth: 2 Supplementation to children and impact on infections: 4
•Anthropometric Indices •APGAR•All-cause mortality •Age-specific mortality •Cause-specific mortality (diarrhoea, pneumonia) •Neonatal (birth weight, linear growth)
•Morbidity (incidence and duration of ARI, diabetes, otitis media)
•Haemoglobin level
LMIC (n= 8) HIC (n= 1)Both (n= 2)
Breast feeding 16 (11/05) Pregnant + reproductive age woman (7)Children (5)General population (2)
Interventions to promote breast feeding: 6
•Initiation, duration and exclusive
breast feeding •Neonatal and infant morbidities
•Anthropometric indices •Maternal satisfaction •Cognitive development •Neonatal mortality •Micronutrient status •Maternal weight loss •Breast and ovarian cancer •Osteoporosis
•Pain
•Maternal morbidities
LMIC (n= 1) HIC (n= 6) Both (n= 4) Not clear (2)
Complementary feeding
06 (1/5) Children (6) Education and counselling interventions to promote complementary feeding: 5
•Anthropometric indices •Micronutrient intake and status
•Morbidity
LMIC (n= 6)
Preventive multiple micronutrient supplementation
14 (4/10) Pregnant + reproductive age woman (6)Children (2)Women and children (4)Adult men (1)
Supplementation during and prior to pregnancy and its impact on pregnancy
outcome: 9 Supplementation to HIV-infected women and children: 2
•Birth weight •Kwashiorkor •Prematurity, SGA, LBW •Stillbirth, neonatal and perinatal mortality
•Premature rupture of membranes •Preeclampsia
•Miscarriage •Congenital anomalies •Nutritional status •Micronutrient level •Maternal anaemia •Multiple pregnancies •Psychological effects •HIV transmission and viral load •Prostate cancer occurrence, metastasis, mortality
LMIC (n= 4) HIC (n= 2) Both (n= 4) Not clear (3)
Therapeutic zinc 12 (4/8) Children (6)General population (5)Adult (1)
Adult supplementation and its impact on cold symptoms: 2
Supplementation among children and impact on diarrhoea and pneumonia: 6
•Ulcer healing (time, rate) •Incidence, severity and duration of cold
•Pneumonia: incidence, duration and recurrence of pneumonia. Recovery time and hospitalization •Adverse effects
•Lipid profile
•Incidence and duration of malaria and TB
•Hospital stay, cost and mortality of critically ill patients
LMIC (n= 3) HIC (n= 4) Both (n= 3) Not clear (2)
Interventions for undernutrition
11 (4/7) Children (11) Management of under
nutrition through supplementary feeding: 9
•Kwashiorkor
•Cognitive development •Hospitalization
LMIC (n= 11)
Existing evidence related to systematic review con-duct in the field of nutrition highlight the fact that sys-tematic reviews in nutrition are unique and challenging as it encompasses various nutrition-related consider-ations including baseline nutrient exposure, nutrient status, bioequivalence of bioactive compounds, bio-availability, multiple and interrelated biological func-tions, undefined nature of some interventions and uncertainties in assessment [23]. Developing capacity-building programmes, searching the primary literature for research gaps and extending reporting tools such as the PRISMA Statement to the field of nutrition are recommended to improve the conduct of systematic reviews in nutrition [48,49].
One strength of this paper is the assessment of methods used in systematic reviews of nutrition using
accepted tools including the AMSTAR and the
‘Cochrane Handbook of Public Health Guidance’. One
limitation is that we used a non-validated search for nutrition topics and thus may have missed some sys-tematic reviews. However, we think this limitation is unlikely to affect the findings of this study since we intended to identify a purposive sample, rather than an exhaustive sample of systematic reviews in nine
intervention areas. Furthermore, our assessment of quality of the included systematic reviews is limited to how they were reported.
Given the growth in the field, there is a clear need to develop a consensus on methodologies for nutrition reviews, criteria for assessing the evidence and possibly facilitating development and collation of the evidence in the subject area. There also needs to be clarity on the purpose of the reviews and on the questions that they are addressing, for them to be useful in the con-text of programme and policy guidance. The diverse methods involved in such systematic evidence synthe-ses and conclusions thereof have a clear bearing on impact on policy and practice. Given the differing needs of information syntheses by practitioners and public sector policymakers, a consensus on the quality indicators for such reviews and methods is an import-ant prerequisite. While the Cochrane and Campbell collaborations have a due process with published guide-lines within the ‘Cochrane Handbook’, other journals and peer review processes may not follow a standard set of criteria. Cochrane and Campbell are well-positioned to take the lead in improving quality as our analysis also suggests that the quality ratings for the Table 4 Population, topics and outcomes reported in the included reviews(Continued)
•Anthropometric indices •Recurrent illnesses •Mortality
•HIV prevalence and mortality •Morbidity: diarrhoea, malaria and ARI
•Maternal knowledge, anxiety Therapeutic
multiple micronutrient supplementation
11 (3/8) Children (1)Adults (4)Adult Men (1)Population with HIV (4)
HIV-infected populations: 5 •Infant mortality •Anthropometric indices •Micronutrient levels •Anaemia
•Mental and motor development •Pregnancy outcomes
•HIV progression, transmission and viral load
•HIV hospitalization, morbidity and mortality
•Fasting glucose and insulin resistance
•Treatment and length of hospitalization of TB •Age-related disorders like age-related macular degeneration and cataract
•Occurrence, metastasis and mortality due to prostate cancer
LMIC (n= 1) HIC (n= 2) Both (n= 4) Not clear (4)
Fortification (folic acid)
02 (0/2) Pregnant and women of reproductive age (1)General population (1)
•Neural tube defect •Neonatal mortality for NTD •Twining
Both (n= 2)
Conditional cash transfers (CCT)
4 (1/3) General population (4) All 4 reviews assessed the impact CCT on health outcomes
•Coverage of health service •Access to health care •Household health expenditure •Nutritional outcomes
LMIC (n= 2) Both (n= 2)
Cochrane reviews were found to be fairly consistent as compared to the non-Cochrane reviews. The Campbell International Development Nutrition group is actively working on these priority areas of methodological quality, outcome standardization and avoiding duplica-tion. Engagement with a range of stakeholders could lead to wider uptake and endorsement of such stan-dards and priorities. This could then expand to include guidance to other journals and agencies evaluating and publishing nutrition-related systematic reviews.
Conclusions
To conclude, there is a need to develop a consensus on methodologies for nutrition reviews, criteria for assessing the evidence and possibly facilitating devel-opment and collation of the evidence in the subject area. We therefore propose the development of a po-tential nutrition review editorial process and a group inclusive of major stakeholders in the field to help bring the field together.
Abbreviations
AMSTAR:Assessment of Multiple Systematic Reviews; COMET: Core Outcome Measures in Effectiveness Trials; DFID: Department of International Development; EPPI: Evidence for Policy, Practice Information and Coordinating Centre; FAO: Food and Agriculture Organization; MeSH: Medical Subject Headings; MMN: Multiple micronutrient; OMERACT: Outcome Measures in Rheumatoid Arthritis Clinical Trials; OR: Odds ratio; PICO: Participants, Intervention, Comparison and Outcome; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses; RCT: Randomized controlled trial; RR: Relative risk; RUTF: Ready-to-use therapeutic food; SAM: Severe acute malnutrition; SMD: Standard mean difference; SOFI: State of Food Security; UN: United Nations; UNICEF: United Nations International Children’s Emergency Fund; USAID: United States Agency for International Development; WFP: World Food Program; WHO: World Health Organization.
Competing interests
The authors declare that they have no competing interests.
Authors’contributions
ZAB was responsible for designing and coordinating the review. VW was responsible for the methodological and quality assessment of the reviews. RAS was responsible for data collection, screening the search results, screening retrieved papers against inclusion criteria, appraising quality of papers and abstracting data from papers. All the authors contributed towards writing the manuscript. All authors read and approved the final manuscript.
Acknowledgements
We would like to acknowledge the academic groups, editors of the Cochrane and Campbell collaborations, 3ie, UN agencies (WHO, UNICEF, WFP, FAO), IFPRI and other bilateral agencies (notably DFID, USAID, NORAD) and representatives from the Bill & Melinda Gates Foundation for funding the work and consultation to guide this study. We would also like to acknowledge Omer Bangash, who helped us with the double data extractions.
Author details
1Center of Excellence in Women and Child Health, Division of Woman and Child Health, Aga Khan University, Karachi 74800, Pakistan.2Centre for Global Health, University of Ottawa, Ottawa, Canada.3Robert Harding Chair in Global Child Health and Policy, Centre for Global Child Health, The Hospital for Sick Children, Toronto, ON, Canada.
Received: 19 September 2014 Accepted: 5 February 2015
References
1. Global Alliance for Improved Nutrition 2014. Highlighting recent advances in understanding the bioavailability of micronutrients. Available at http:// www.gainhealth.org/knowledge-centre/highlighting-recent-advances-understanding-bioavailability-micronutrients/.
2. Shah PS, Aliwalas LL, Shah V. Breastfeeding or breast milk for procedural pain in neonates. Cochrane Database Syst Rev. 2006;12(12):CD004950. 3. Bhutta ZA, Salam RA. Global nutrition epidemiology and trends. Ann Nutr
Metab. 2012;61(1):19-27.
4. Bhutta ZA. Micronutrient needs of malnourished children. Curr Opin Clin Nutr Metab Care. 2008;11(3):309–14.
5. World Health O. Severe malnutrition: report of a consultation to review current literature. Geneva: World Health Organization; 2005. 6. Black RE, Victora CG, Walker SP, Bhutta ZA, Christian P, de Onis M, et al.
Maternal and child undernutrition and overweight in low-income and middle-income countries. Lancet. 2013;382(9890):427–51.
7. Black RE, Allen LH, Bhutta ZA, Caulfield LE, De Onis M, Ezzati M, et al. Maternal and child undernutrition: global and regional exposures and health consequences. Lancet. 2008;371(9608):243–60.
8. Victora CG, Adair L, Fall C, Hallal PC, Martorell R, Richter L, et al. Maternal and child undernutrition: consequences for adult health and human capital. Lancet. 2008;371(9609):340.
9. Bhutta ZA, Ahmed T, Black RE, Cousens S, Dewey K, Giugliani E, et al. Maternal and child undernutrition 3: what works? Interventions for maternal and child undernutrition and survival. Child-Care Health Dev. 2008;34(3):404–404.
10. Bloomberg JH. Maternal and child undernutrition 4. Maternal and child undernutrition: effective action at national level. Lancet. 2008;371:510–26. 11. Morris SS, Cogill B, Uauy R. Effective international action against undernutrition:
why has it proven so difficult and what can be done to accelerate progress? Lancet. 2008;371(9612):608–21.
12. Bhutta ZA, Das JK, Rizvi A, Gaffey MF, Walker N, Horton S, et al. Evidence-based interventions for improvement of maternal and child nutrition: what can be done and at what cost? Lancet. 2013;382(9890):452–77.
13. Gillespie S, Haddad L, Mannar V, Menon P, Nisbett N. The politics of reducing malnutrition: building commitment and accelerating progress. Lancet. 2013;382(9891):552–69.
14. Ruel MT, Alderman H. Nutrition-sensitive interventions and programmes: how can they help to accelerate progress in improving maternal and child nutrition? Lancet. 2013;382(9891):536–51.
15. The World Bank. Fragility and Conflict. Accessed from http://wbi. worldbank.org/wbi/about/topics/fragile-states.
16. Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. J Health Serv Res Policy. 2005;10 Suppl 1:35–48.
17. Furukawa TA, Streiner DL, Hori S. Discrepancies among megatrials. J Clin Epidemiol. 2000;53(12):1193–9.
18. LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. N Engl J Med. 1997;337(8):536–42.
19. Trikalinos TA, Moorthy D, Chung M, Yu WW, Lee J, Lichtenstein AH, Lau J: Concordance of randomized and nonrandomized studies was unrelated to translational patterns of two nutrient-disease associations. J Clin Epidemiol. 2012;65(1):16–29.
20. Villar J, Carroli G, Belizan JM. Predictive ability of meta-analyses of randomised controlled trials. Lancet. 1995;345(8952):772–6. 21. Cook DJ, Greengold NL, Ellrodt AG, Weingarten SR. The relation between
systematic reviews and practice guidelines. Ann Intern Med. 1997;127(3):210–6. 22. Lavis JN, Posada FB, Haines A, Osei E. Use of research to inform public
policymaking. Lancet. 2004;364(9445):1615–21.
23. Lichtenstein AH, Yetley EA, Lau J. Application of systematic review methodology to the field of nutrition. (Prepared by the Tufts Evidence-based Practice Center under Contract No. 290–020022). AHRQ publication No. 09–0025. Rockville, MD: Agency for Healthcare Research and Quality; 2009. Available at http://www.ncbi.nlm.nih.gov/books/NBK44076/pdf/TOC.pdf. 24. Tugwell P, Petticrew M, Kristjansson E, Welch V, Ueffing E, Waters E, et al.
Assessing equity in systematic reviews: realising the recommendations
of the Commission on Social Determinants of Health. BMJ. 2010; 341:c4739.
25. WHO. e-Library of Evidence for Nutrition Actions (eLENA). Access from http://www.who.int/elena/en/.
26. WHO. Use of systematic reviews of the evidence in public health nutrition. Proceedings of an informal consultation held 18 October 2011 in Geneva, Switzerland. Geneva: World Health Organization; 2013. Available at http:// apps.who.int/iris/bitstream/10665/85792/1/9789241506038_eng.pdf. 27. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews
of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org 28. Systematic Reviews of Health Promotion and Public Health Interventions.
http://ph.cochrane.org/sites/ph.cochrane.org/files/uploads/Guidelines% 20HP_PH%20reviews.pdf.
29. Shea B, Grimshaw J, Wells G, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):10. 30. Aggarwal R, Sentz J, Miller MA. Role of zinc administration in prevention of
childhood diarrhea and respiratory illnesses: a meta-analysis. Pediatrics. 2007;119(6):1120.
31. Lassi ZS, Haider BA, Bhutta ZA. Zinc supplementation for the prevention of pneumonia in children aged 2 months to 59 months. Cochrane Database Syst Rev. 2010;12(12):CD005978.
32. Mathew JL. Zinc supplementation for prevention or treatment of childhood pneumonia: a systematic review of randomized controlled trials. Indian Pediatr. 2010;47(1):61–6.
33. Roth DE, Richard SA, Black RE. Zinc supplementation for the prevention of acute lower respiratory infection in children in developing countries: meta-analysis and meta-regression of randomized trials. Int J Epidemiol. 2010;39(3):795.
34. Yakoob MY, Theodoratou E, Jabeen A, Imdad A, Eisele T, Ferguson J, et al. Preventive zinc supplementation in developing countries: impact on mortality and morbidity due to diarrhea, pneumonia and malaria. BMC Public Health. 2011;11 Suppl 3:S23.
35. Funai EF, Rosenbush EJ, Lee MJ, Del Priore G. Distribution of study designs in four major US journals of obstetrics and gynecology. Gynecol Obstet Invest. 2001;51(1):8–11.
36. Scales Jr CD, Norris RD, Peterson BL, Preminger GM, Dahm P. Clinical research and statistical methods in the urology literature. J Urol. 2005;174(4 Pt 1):1374–9. 37. Rudan I, Tomaskovic L, Boschi-Pinto C, Campbell H. Global estimate of the incidence of clinical pneumonia among children under five years of age. Bull World Health Organ. 2004;82(12):895–903.
38. Kramer MS, Kakuma R. Optimal duration of exclusive breastfeeding. Cochrane Database Syst Rev. 2002;1(1):CD003517.
39. Lewin S, Munabi Babigumira S, Glenton C, Daniels K, Bosch Capblanch X, Van Wyk BE, et al. Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases. Cochrane Database Syst Rev. 2010;3(3):CD004015. 40. Dewey KG, Adu Afarwuah S. Systematic review of the efficacy and
effectiveness of complementary feeding interventions in developing countries. Maternal Child Nutr. 2008;4:24–85.
41. Imdad A, Sadiq K, Bhutta ZA. Evidence-based prevention of childhood malnutrition. Curr Opin Clin Nutr Metab Care. 2011;14(3):276. 42. Shi L, Zhang J. Recent evidence of the effectiveness of educational
interventions for improving complementary feeding practices in developing countries. J Trop Pediatr. 2010;57(2):91.
43. Lagarde M, Haines A, Palmer N. Conditional cash transfers for improving uptake of health interventions in low-and middle-income countries. JAMA. 2007;298(16):1900.
44. Lagarde M, Haines A, Palmer N. The impact of conditional cash transfers on health outcomes and use of health services in low and middle income countries. Cochrane Database Syst Rev. 2009;4(4):CD008137.
45. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78. 46. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews
of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
47. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
48. Moher D, Tricco AC. Issues related to the conduct of systematic reviews: a focus on the nutrition field. Am J Clin Nutr. 2008;88(5):1191–9.
49. Lichtenstein AH, Elizabeth AY, Joseph L. Application of systematic review methodology to the field of nutrition. J Nutr. 2008;12(138):2297–306.
Submit your next manuscript to BioMed Central and take full advantage of:
• Convenient online submission
• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution
