The application of knowledge synthesis methods in agri-food public health: Recent advancements, challenges and opportunities

(1)

ContentslistsavailableatScienceDirect

Preventive

Veterinary

Medicine

jo u r n al ho me p ag e :w w w . e l s e v i e r . c o m / l o c a t e / p r e v e t m e d

The

application

of

knowledge

synthesis

methods

in

agri-food

public

health:

Recent

advancements,

challenges

and

opportunities

Ian

Young

a,b,∗

,

Lisa

Waddell

a,b

,

Javier

Sanchez

c

,

Barbara

Wilhelm

b

,

Scott

A.

McEwen

b

,

Andrijana

Raji ´c

b,d

a_Laboratory_for_Foodborne_Zoonoses,_Public_Health_Agency_of_Canada,₁₆₀_Research_Lane,_Guelph,_Ontario,_Canada_N1G_5B2 b_Department_of_Population_Medicine,_University_of_Guelph,₅₀_Stone_Road,_Guelph,_Ontario,_Canada_N1G_2W1

c_Department_of_Health_Management,_University_of_Prince_Edward_Island,₅₅₀_University_Avenue,_{Charlottetown,}_PE,_Canada_C1A_4P3 d_Nutrition_and_Consumer_Protection_Division,_Food_and_Agriculture_{Organization,}_Viale_delle_Terme_di_Caracalla,_Roma_00153,_Italy

a

r

t

i

c

l

e

i

n

f

o

Articlehistory:

Received14August2013 Receivedinrevisedform 13November2013 Accepted15November2013 Keywords: Knowledgesynthesis Systematicreviews Meta-analysis Agri-foodpublichealth

a

b

s

t

r

a

c

t

Knowledgesynthesisreferstotheintegrationoffindingsfromindividualresearchstudies onagiventopicorquestionintotheglobalknowledgebase.Theapplicationof knowl-edgesynthesismethods,particularlysystematicreviewsandmeta-analysis,hasincreased considerablyintheagri-foodpublichealthsectoroverthepastdecadeandthistrendis expectedtocontinue.Theobjectivesofourreviewwere:(1)todescribethemost promis-ingknowledgesynthesismethodsandtheirapplicabilityinagri-foodpublichealth,and(2) tosummarizetherecentadvancements,challenges,andopportunitiesintheuseof system-aticreviewandmeta-analysismethodsinthissector.Weperformedastructuredreviewof knowledgesynthesisliteraturefromvariousdisciplinestoaddressthefirstobjective,and usedcomprehensiveinsightsandexperiencesinapplyingthesemethodsintheagri-food publichealthsectortoinformthesecondobjective.Wedescribefiveknowledgesynthesis methodsthatcanbeusedtoaddressvariousagri-foodpublichealthquestionsortopics underdifferentconditionsandcontexts.Scopingreviewsdescribethemaincharacteristics andknowledgegapsinabroadresearchfieldandcanbeusedtoevaluateopportunities forprioritizingfocusedquestionsforrelatedsystematicreviews.Structuredrapidreviews arestreamlinedsystematicreviewsconductedwithinashorttimeframetoinformurgent decision-making.Mixed-methodandqualitativereviews synthesizediverse sourcesof contextualknowledge(e.g.socio-cognitive,economic,andfeasibilityconsiderations). Sys-tematicreviewsareastructuredandtransparentmethodusedtosummarizeandsynthesize literatureonaclearly-definedquestion,andmeta-analysisisthestatisticalcombinationof datafrommultipleindividualstudies.Webrieflydescribeanddiscusskeyadvancements intheuseofsystematicreviewsandmeta-analysis,including:risk-of-biasassessments;an overallquality-of-evidenceapproach;engagementofstakeholders;Bayesian, multivari-ate,andnetworkmeta-analysis;andsynthesisofdiagnostictestaccuracystudies.Wealso highlightseveralchallengesandopportunitiesintheconductofsystematicreviews(e.g. inclusionofgreyliterature,minimizinglanguagebias,andoptimizingsearchstrategies) andmeta-analysis(e.g.inclusionofobservationalstudiesandapproachestoaddressthe

∗ Correspondingauthorat:LaboratoryforFoodborneZoonoses,PublicHealthAgencyofCanada,160ResearchLane,Unit206,Guelph,Ontario,Canada N1G5B2.Tel.:+15198262098;fax:+15198262255.

E-mailaddress:[email protected](I.Young).

http://dx.doi.org/10.1016/j.prevetmed.2013.11.009

(2)

340 I.Youngetal./PreventiveVeterinaryMedicine113(2014)339–355

insufﬁcientreportingofdataandsigniﬁcantheterogeneity).Manyofthesedevelopments haveyettobecomprehensivelyappliedandevaluatedinanagri-foodpublichealthcontext, andmoreresearchisneededinthisarea.Thereisaneedtostrengthenknowledgesynthesis capacityandinfrastructureattheregional,national,andinternationallevelsinthissector toensurethatthebestavailableknowledgeisusedtoinformfuturedecision-makingabout agri-foodpublichealthissues.

1. Introduction

Researchend-users(e.g.policy-makers,practitioners, and other decision-makers) should be informed with the best available knowledge in order to demonstrate accountableandevidence-informeddecision-making for complexissueswithimportanthealthandsocio-economic implications.Theprocessofmovingresearchknowledge into policy and practice and enhancing its utilization amongend-usersisreferredtoasknowledgetransferand exchange(Lavisetal.,2003;Mittonetal.,2007;Raji ćetal., 2013).Knowledgesynthesisisakeyfoundationof knowl-edgetransferandexchangebecauseitintegratesfindings frommultipleindividualstudiesandothersourcesona giventopic or questionintothe globalknowledge base (Grimshaw,2010;Triccoetal.,2011).Knowledge synthe-sisprovidesamoreaccurateandreliableassessmentofthe stateofknowledgeaboutatopicthanindividualstudies (Lavisetal.,2005), anditfollowsamorestructuredand transparentmethodologythantraditionalnarrative litera-turereviews(Sargeantetal.,2006a,b;Waddelletal.,2009). Twospecificknowledgesynthesismethods,systematic reviewsandmeta-analysis,havebeenwidelyadoptedin multiplesectorsoverthepastseveraldecadesinanattempt toimprovethe generalutilizationof knowledgeamong end-usersandtoinformpolicy-makingwiththebest avail-able knowledge (Tricco et al., 2011; Raji ć et al., 2013). Forexample,intheagri-foodpublichealthsector, meta-analysiswas usedby an expert panelin 1998–1999 to informHealthCanada’spotentialapprovalofrecombinant bovinesomatotropin(rbST)foruseindairycattle produc-tion(Health Canada,1998;Dohooetal., 2003a,2003b). Basedonthesefindings,thepanelconcludedthatthere wereseveral animal health and welfare concerns asso-ciatedwithrbSTandthis contributedtothesubsequent decisionnottoapproverbSTinCanada(HealthCanada, 1998;Dohooetal.,2003a,2003b).Despitethesuccessful useofmeta-analysistosupportthispolicydecision,the for-maladoptionofknowledgesynthesismethodstoaddress agri-foodpublichealthissuesdidnotwidelyoccuruntil thepublicationofinitialsystematicreviewguidelinesin thissectorin2005(Sargeantetal.,2005,2006a,b).

Sincethattime,systematicreviewsandmeta-analysis havebeenincreasinglyconductedinthissectorto inves-tigatequestions aboutintervention efﬁcacy,risk factors forinfectionordisease,prevalenceandconcentrationof outcomes,and diagnostic test accuracy in a wide vari-etyof topic areas (Wilkins et al., 2010; Wilhelmet al., 2011a;Bucheretal.,2012a;Snedekeretal.,2012;Tuˇsevljak et al., 2012; Kerr et al., 2013). Additional knowledge synthesismethods,includingscopingreviews,structured rapidreviews,andmixed-methodreviews,haverecently

developedinthehealthandsocialsciencesectors(Arksey andO’Malley,2005;Maysetal.,2005;Ganannetal.,2010), andthesehavealsobeguntobeadaptedandimplemented toaddressagri-foodpublichealthissues(Ilicetal.,2012; Tuˇsevljaketal.,2012;Raji ćetal.,2013).Integrated find-ingsbasedononeormoreofthesemethodscanbeused toinform policyandprogramme development,to iden-tify knowledge gaps and prioritize future research, and to inform risk and decision analysis (Fazil et al., 2008; EuropeanFoodSafetyAuthority[EFSA],2010;Raji ćetal., 2013).

Manydevelopmentsintheconductofknowledge syn-thesis,particularlysystematicreviewsandmeta-analysis, havebeenpublishedduringthepastseveralyears(Sheldon, 2005;SuttonandHiggins,2008;HigginsandGreen,2011), butthesehavenotbeencomprehensivelydescribedand discussed in previous introductory guides in the agri-foodpublichealthcontext(Sargeantetal.,2005,2006a,b; EFSA,2010;Gonzales-BarronandButler,2011).In addi-tion,throughourconductofseveralknowledgesynthesis projectsoverthepastdecadewehaveencounteredmany uniquechallengesandconsiderationsintheapplicationof thesemethodstoagri-foodpublichealthissues.Webelieve that theseexperiences andinsightswould beneﬁtother researchersinthisarea.Theobjectivesofthisrevieware: (1)todescribeand discussthekeyknowledgesynthesis methodsandtheircontextualapplicabilitytotheagri-food publichealthsector,and(2)todiscusstherecent advance-ments,challenges,andopportunitiesrelatedtotheuseof systematicreviewandmeta-analysismethodsinthis sec-tor.Throughoutthis reviewwe usetheterm“agri-food publichealth”torefer tothecross-cuttingand overlap-pingareasofveterinarypublichealth,foodsafety,and“One Health”(Sargeantetal.,2006a,b;Raji ´cetal.,2013).Wealso referto“knowledge”asencompassingresearchaswellas othersourcesofinformation(e.g.governmentpolicies)that couldbesynthesizedandusedtoinformdecision-making.

2. Reviewapproach

Astructuredreviewwasconductedaspartofalarger projectaboutknowledgetransferandexchangeto iden-tify, classify,and summarize key information aboutthe most promising and recommended knowledge synthe-sis methods as reportedin various sectors (Raji ćet al., 2013).Briefly,acomprehensiveandpre-testedsearchwas implementedonJuly25,2011,infiveonlinebibliographic databases(Medline,Scopus,CommonwealthAgricultural Bureau[CAB]Direct,Current ContentsConnect,andthe CumulativeIndextoNursingandAlliedHealthLiterature) toidentifyreviews,reports, commentaries,casestudies, andothercomprehensiveliteratureaboutthesubject(Raji ć Open access under CC BY-NC-ND license.

(3)

et al., 2013). We also conducted a Scopus web search, limited to theﬁrst 100hits as sorted by relevance, to identifygreyliterature,andincludedtwobooksaboutthe subject (Strauset al.,2009; Bennett and Jessani, 2011). Additional articles about knowledge synthesis methods wereidentiﬁedfromthereferencelistsofrelevantarticles andprocuredduringarticlecharacterization.

Relevancescreeningofallidentifiedcitationswas con-ductedbytwoindependentreviewersusingapre-tested form with two questions (Raji ć et al., 2013). Citations wereconsideredrelevantiftheywerepublishedinEnglish, Spanish,orFrenchandiftheydescribedoneormore knowl-edgesynthesismethodstosupportorfacilitateknowledge transferandexchangeforpolicy-ordecision-making.Two reviewers(I.Y.andA.R.) independentlycharacterizedall relevantarticlesusinganiterativelydevelopedform(Raji ć etal.,2013).I.Y.extractedkeyinformationfromthearticles aboutthemethoddescriptions,contextofuse,advantages anddisadvantages,andanynotedchallengesor opportuni-ties.Anarrativesynthesisoftheextractedinformationwas conductedforeachsynthesismethod(Maysetal.,2005; Raji ćet al.,2013).Thereview wasconductedusingthe onlinemanagement softwareDistillerSR(Evidence Part-nersIncorporated,Ottawa,ON).Additionaldetailsabout thereviewmethods,includingacopyofthespecificsearch algorithmandallformsused,arereportedinRaji ćetal. (2013).

To obtain more detailed information about specific systematic reviewand meta-analysisadvancementsand challenges, four authors (I.Y., L.W., J.S. and A.R.) devel-opedalistofkeyissuesthataroseduringourcollaborative conduct of various knowledge synthesis projects since publication of a previousguide in this sector(Sargeant et al., 2005, 2006a,b).These issues wereidentified and informed by reviewing published knowledge synthesis articlesconductedbythereview authorsand otherkey collaboratorsandthroughinformalandadhocgroup dis-cussions and consultations during this time. The final list of issues included: risk-of-bias assessments; over-allquality-of-evidenceapproaches;summary-of-findings tables;stakeholderengagement;inclusionofgrey litera-tureandobservationalstudies;languagebias;optimizing search strategies;updatingreviews; resourceand logis-tical requirements;Bayesian,multivariate,network, and individual participant data meta-analysis; synthesis of diagnostictestaccuracy studies;insufficientdata repor-ting;andmeta-analysisofasmallnumberofstudiesand in the presence of significantheterogeneity. Thislist is notmeanttobeexhaustiveofallpossibleadvancements andchallengesinthisareabutreflectsthoseconsidered bytheauthorstohaveimportantimplicationsfor knowl-edgesynthesisinagri-foodpublichealth.Relevantarticles to support these sections of the review were obtained fromreferencespreviouslyknowntotheauthors,through searching theirreference lists,and viaadhocliterature searchesinonlinebibliographicdatabases.

3. Overviewofknowledgesynthesismethods

We identiﬁed seven key knowledge synthesis meth-ods from the initial screening of 827 unique abstracts

and characterizationof 168relevant articlesduring the structuredreviewoftheknowledgetransferandexchange literature(Raji ´cetal.,2013).Systematicreviewsand meta-analysiswereoriginallycategorizedasonecomprehensive methoddue totheircomplementarynaturebutare dis-cussed in this reviewas distinct methods.Anoverview ofﬁveoftheseknowledgesynthesis methods,including theirbriefdescription,contextual applicability,and key advantagesanddisadvantagesareshowninTable1and discussed below. We primarilyfocus thediscussion on systematicreviewsandmeta-analysisdue totheirmore extensivedevelopmentand wider adoptionin this sec-tor. Two additional methods, knowledge mapping and synthesisofpublicpolicies(Ebeneretal.,2006;National CollaboratingCentreforHealthyPublicPolicy,2010),are notdiscussedherebecausetheyprimarilyfocuson synthe-sizingsourcesofknowledgeotherthanresearchandthey wereconsideredbeyondthescopeofthisreview.

3.1. Scopingreviews

Scopingreviewsareusedtomapoutthedistribution and characteristics of a broad knowledge area or issue (ArkseyandO’Malley,2005;Andersonetal.,2008).They canbeconductedtosummarizethestateofknowledge on a particular issue, to identify research gaps, and to prioritizequestionsfor asystematicreview (Arkseyand O’Malley,2005;Andersonetal.,2008).Incontrasttoa sys-tematicreview,scopingreviewsusuallyfocusonabroader researchquestionthatisoftenpolicy-drivenandthey typ-icallydonotincludearisk-of-biasassessmentstep(Arksey andO’Malley,2005;Andersonetal.,2008).Theycanbe conductedincombinationwithasystematicreviewtohelp focustherisk-of-biasassessment,detaileddataextraction, and analysissteps toareas where sufﬁcient knowledge isavailable.Forexample,theresultsofarecent scoping reviewinvestigating theprevalenceofzoonotic bacteria andantimicrobialresistancein farmedandwildaquatic speciesandseafoodwereusedtoprioritizespeciﬁcareas (i.e.bacteria,aquaticandseafoodspecies,andpointinfood chaincombinations)for targetedsystematic reviewand meta-analysis(Tuˇsevljaketal.,2012).

The initial scoping review framework proposed by ArkseyandO’Malley(2005)consistsofthefollowingsix steps:(1)identifytheresearchquestion;(2)identify rele-vantstudies(i.e.searchstrategy);(3)studyselection(i.e. relevancescreeningandextractionofkeycharacteristics from relevant articles); (4) data charting; (5) collating, summarizing,andreportingtheresults;and(6)anoptional stakeholderengagementstep.Manyofthesesteps corre-spondtosimilarstepsinthesystematic reviewprocess (Fig. 1), withthe major differences betweenthese two methods largelyrelated thenature ofthe review ques-tion (broadand policy-driven vs. focused and specific). Based onourexperienceconducting scopingreviewsto addresscomplexetiologicalquestions(e.g.source attribu-tionofinfectiousdiseases),thefindingsandconclusions canbedependentonthespecificvariablesusedto catego-rizeandsummarizethepublishedresearch.Forexample, inanongoingreview abouttheroleofswineand other animalspeciesinthetransmissionofemergingvirusesto

(4)

342 I. Young et al. / Preventive Veterinary Medicine 113 (2014) 339–355 Table1

Anoverviewofthekeycharacteristicsofﬁveknowledgesynthesismethods.

Synthesismethod Briefmethoddescription Agri-foodpublichealthexamplequestions Keyadvantages Keydisadvantages Approximate timelineto completiona Structuredrapid

review

•Streamlinedsystematicreview conductedwithinashorttimeframeor withlimitedresourcesandthatfeeds directlyintodecision-making(Ganann etal.,2010)

Whatarethepublicattitudestowards emergingfoodtechnologies?(Four-month rapidreviewconductedfortheUKFood StandardsAgency)(Lyndhurst,2009)

•Lessresource-intensivethana fullsystematicorscopingreview

•Rapidtimeframeusedtoprovide urgentadviceforpolicy-and decision-making

•Someﬂexibilityintheprocedures

•Lackofstandardizedand validatedprocedures

•Exclusionormodiﬁcationof somestepscanintroducebias

•≤3months

Scopingreview •Reviewofabroadresearchquestion tomapoutthekeycharacteristicsofa knowledgeareaandthemainsources andtypesofinformationavailable (ArkseyandO’Malley,2005)

Whatisthecharacterizationand distributionofpublishedprimaryresearch aboutmicrobialhazardsinleafygreen vegetables?(Ilicetal.,2012)

•Canidentifyknowledgegapsand informsystematicreviewsand decision-making

•Someﬂexibilityintheprocedures

•Involvementofstakeholders increasesrelevanceofresults

•Lackofstandardizedand validatedprocedures

•Usuallydoesnotincludea risk-of-biasassessment

•Involvementofstakeholders couldintroducebias

•6–12months

Mixed-methodand qualitative reviews

•Modiﬁedsystematicreviewthat includesadiverserangeofqualitative andquantitativesourcesofknowledge (Maysetal.,2005)

•Manyvariationsexist,including realistreview(Pawsonetal.,2005), integrativereview(Whittemoreand Knaﬂ,2005),andmeta-ethnography review(Atkinsetal.,2008)

Whatarethekeyprinciplesofknowledge transferandexchangeandtheirpotential applicabilitytotheagri-foodpublichealth sector?(Raji ´cetal.,2013)

•Broadrangeofknowledge considered

•Flexibilityinchoiceofmethod andprocedures

•Usefultoinformpolicy-and decision-makingduetorangeof contextualknowledgeconsidered

•Difﬁcultiesinidentifyingand evaluatingqualitativeresearch

•Canbesubjectiveandlack transparency

Lackofstandardizedprocedures

•3–18months

Systematicreview •Astructuredreviewofaclearly deﬁnedquestion(Sargeantetal.,2006;

HigginsandGreen,2011)

•Usessystematicandexplicit procedurestoidentify,select,critically appraise,extract,andanalyzedata fromprimaryresearch(Sargeantetal., 2006;HigginsandGreen,2011)

Intervention:

Whatistheefﬁcacyofchilling interventionstoreduceSalmonella contaminationofchickencarcassesduring processing?(Bucheretal.,2012a) Riskfactor:

Whatistheroleofswine,porkandpork productsasapotentialsourceofzoonotic hepatitisEvirusinhumans?(Wilhelm etal.,2011a)

Diagnostictestaccuracy:

Whatisthediagnosticaccuracyofculture andPCRtodetectSalmonellainswine? (Wilkinsetal.,2010)

•Isarigorous,transparent,and reliablemethod

•Providesacomprehensiveand crediblesummaryofthestateof knowledgeonaspeciﬁctopic

•Frameworkandproceduresare well-deﬁnedandestablished

•Isresource-andtime-intensive

•Somestudydesignstypically excluded(e.g.observational)

•Questionmightbetoonarrowly focusedforuseinpolicy-making contexts

•3–18months

Meta-analysis •Thestatisticalcombinationofdata frommultipleindividualstudies (Borensteinetal.,2009; Gonzales-BarronandButler,2011)

•RefertoTable2foranoverviewof traditionalandadvancedapproaches

Fixed-effectmeta-analysis:

WhatisthebestestimateofrbSTtoaffect theriskofclinicalmastitisindairycattle? (Dohooetal.,2003a)

Random-effectsmeta-analysis:

Whatistheaverageestimateofefﬁcacyof TypeIIIproteinvaccinestoreducefaecal sheddingofE.coli0157incattlefaeces? (Snedekeretal.,2012)

•Canincreaseprecisionandpower ofeffectestimates

•Estimatescanbeusedascredible inputsforriskand

decision-analysismodels

•Manyproceduresexisttopool dataandexploreheterogeneity underdifferentscenarios

•Isonlyreliableifmodel assumptionsareadheredto,with well-conductedstudies,andifdata aresufﬁcientlyreportedand comparable

•Someadvancedmethodsrequire specializedexpertiseandarestill underdevelopment

•1–3months(in additiontothe timerequiredfor completionofthe systematicreview)

a_Timelines_depend_on_various_factors,_including_available_resources_and_expertise,_complexity_of_the_question_and_topic,_and_if_the_review_is_conducted_as_part_of_a_larger,_experienced_team_in_settings_that

(5)

Fig.1.Overviewofthesystematicreviewprocessandrecommendedsteps.

humans,themostimportantsourceofhumanexposure dif-fereddependingonthefollowingvariables:animalspecies mostcommonlyinvestigatedasapotentialsource,species mostcommonlyconﬁrmedastheexposuresource,species withthegreatestreportedprevalence,andwhetherarticles reportedvirusdetectionintheputativeexposuresource. Therefore,werecommendthatsensitivityanalysisbe con-ductedinscopingreviewsduringthecategorizationand analysisstagetoensurethatdifferentpossible interpreta-tionsareidentiﬁedandconsidered.

A comprehensive review was conducted in 2012 to identify and characterize all scoping reviews published from 1999 to2012 and guidelines for their conduct in allsectors(Phametal.,2013b).Resultsindicatethatthe majorityofscopingreviews(>70%)werepublishedinthe healthsectorsince2010(Phametal.,2013b).Theresultsof thisreviewwillbeusedtodevelopascopingreview frame-workthatisspeciﬁctotheagri-foodpublichealthcontext (Phametal.,2013b).

3.2. Structuredrapidreviews

Decision-makers oftenrequire knowledgetobe pre-sented in a short timeframe, which conﬂicts with the many monthsoryears it couldtaketoconducta scop-ingreview followedbyone ormorefocused systematic reviews(Lavisetal.,2005;Ganannetal.,2010).For exam-ple, under a research setting, approximately two years wererequiredtocompletealargescopingreviewfollowed bycomplementarysystematicreviewsandmeta-analyses evaluating interventionstocontrolSalmonella in broiler

chickensatfarmandprocessing(Farrar,2009).Structured rapidreviewsarestreamlinedandacceleratedsystematic reviewsdesigned toprovidemore timelyknowledge to informdecision-makingfor policyand practice(Ganann etal.,2010).Theyaretypicallydrivenbyanurgentdemand fromend-usersforinformationaboutatopic(Ganannetal., 2010).Thiscouldoccurintheagri-foodpublichealth con-text,forexample,ifthereisaneedtoprovideknowledgeon potentialgovernmentpolicyoptionsfollowingafoodborne orzoonoticdiseaseoutbreak,ifnopreviouslypublished systematicreviewexists,andifthereisashortwindowof opportunitytorespondwithaparticularcourseofaction.

There is no consistent and standardized approach toconducting rapid reviews, althoughvarious methods have been proposed to shorten the systematic review timeframe,includingsearchstrategylimitations(e.g. pub-licationyears,language,andnumberofdatabases)anduse ofonlyonereviewerforrelevancescreening,risk-of-bias assessment,ordataextraction(Ganannetal.,2010;Harker andKleijnen,2012).Arecentreviewofrapidreviews pub-lishedintheareaofhealthtechnologyassessmentsfound asignificantpositivecorrelationbetweenthenumberof recommendedsystematicreviewproceduresreportedand the length of time taken (in months) to complete the review(HarkerandKleijnen,2012).Theuseof method-ologicalrestrictionsinrapidreviewscouldimpacttherisk ofbias,strengthofevidence,andcredibilityoftheir find-ings(Buscemietal.,2006;Ganannetal.,2010).Therefore, rapidreviewsshouldincludedetaileddescriptionsoftheir modifiedmethodsandexplicitlyhighlighttheirpotential limitations.

(6)

3.3. Mixed-methodandqualitativereviews

End-usersoftenmustaddresscomplexproblemsthat require an analysis of contextual information, such as stakeholder attitudes, values, and opinions and other underlying socio-behavioural mechanisms potentially affectingthesuccessorfailureofinterventions,andthis information is often found only in qualitative research studies (Mays et al., 2005; Dixon-Woods et al., 2006). Mixed-methodandqualitativereviewsweredevelopedas anextensionofsystematicreviewstoincludeand synthe-sizebothquantitativeandqualitativeresearchstudiesas wellasothersourcesofknowledge(e.g.reviewarticles, reports,andpolicydocuments)thatmightcontainrelevant contextualinformationabouta giventopic(Maysetal., 2005;Dixon-Woodsetal.,2006).Thereisnostandardized framework for conducting mixed-methodor qualitative reviews,althoughgeneralguidelinesareavailable(Mays etal.,2005;Dixon-Woodsetal.,2006).Speciﬁcvariations of mixed-method and qualitative reviews include real-istreviews,whichinvestigatehowcomplexinterventions workorwhytheyfailinparticularsettings(Pawsonetal., 2005),andmeta-ethnographyreviews,whichareusedto develop higher-order theories about humanbehaviours andexperiences(Atkinsetal.,2008).Thesereviewmethods couldhavepotentialapplicationsintheagri-foodpublic healthsectorgiventhemorefrequent useofqualitative researchmethodstoinvestigateissuessuchasstakeholder constraintstowardsfoodsafetypolicydevelopmentand factorsrelatedtoproducers’implementationofgood agri-culturalpractices (Sargeantet al.,2007a,b;Youngetal., 2011).

3.4. Systematicreviews

Anoverviewofthesystematicreviewprocessisshown inFig.1.Theprocessshouldbeginwiththeestablishment ofareviewteamthatconsistsofcollaboratorswithtopic, methodological,andinformationscienceexpertise(Fig.1). Thenextandmostimportantstepistoformulateaclear andconcisereviewquestiontoaddressthereview objec-tives(Sargeantetal.,2006a,b).Questiondevelopmentcan befacilitated bydetermining thereview question’sﬁve PICOScomponents:Population,Intervention(orExposure), Comparison,Outcome,andStudydesign.Aprotocol, devel-opedatthebeginningofthereviewtoguidetheprocess, shouldincludeadetaileddescriptionofallmethodsused ateachstageofthereview,includinganyeligibility crite-ria (e.g. study designs, publication types,or languages) (Sargeantetal.,2005,2006a,b).

Acomprehensivesearchstrategyshouldbedeveloped inconsultationwithaninformationspecialistorlibrarian (Fig. 1). It should include searching of multiple biblio-graphicdatabases(e.g.PubMed,Scopus,andCABDirect) supplementedwithothersources,suchaswebsearches forgreyliteratureandhand-searchingthereferencelistsof relevantarticles(HigginsandGreen,2011;Horsleyetal., 2011;Grindlayetal.,2012).Identiﬁedcitationsshouldbe screenedforrelevanceattheabstractlevel,andrelevant articlesmayundergoasecondaryscreening(i.e. conﬁrma-tionofrelevance).Arisk-of-biasassessmentisconducted

concurrentlywithdataextractiontoobtainapriori-deﬁned dataonthereportedstudymethods,outcomes,and inde-pendentandconfoundingvariablesofinterest.Eachstep should be conducted by two independent reviewersto minimizeerrors.

Atminimum,allsystematicreviewsshouldinclude a descriptiveanalysisandnarrativesummaryoftheresults, datacharacteristics,andriskofbiasfortheincluded stud-ies(Moheretal.,2009).Typically,thegenerationofpooled oraverageeffectestimates(i.e.meta-analysis)willbeone ofmaingoalsofasystematicreview.Authorswillneedto determinewhethermeta-analysisispossibleand appro-priategiventheamountand natureof thedataandthe objectivesofthereview.Forexample,meta-analysismight notbesuitableandcouldbemisleadingwhentheincluded studieshaveahighriskofbias,whenthereareserious pub-licationorreportingbiases,orwhenthereissigniﬁcantand unexplainedheterogeneity(i.e.thestudiesaretoodiverse tobemeaningfullycombined)(HigginsandGreen,2011). Evenwhenmeta-analysisisnotpossible,authorsshould considerthepossibilityofconductingmeta-regressionto explorereasonsforheterogeneity.

Reporting of systematic reviews should follow the PRISMA(PreferredReportingItemsforSystematicreviews andMeta-Analysis)guidelines(Moheretal.,2009),and evi-dencethattheseguidelineshavebeenadheredtoisnow requiredaspartofthesubmissionprocessformany peer-reviewedjournalsbeforeconsiderationforpublication.In particular,PRISMAindicatesthatauthorsshoulddiscuss theimplications of theirresultsin thecontextof other evidence and future research (Moheret al., 2009).One ofthecommonmisconceptionswithsystematicreviews is thattheywillleadtoa definitivetruth regardingthe researchquestionandanymeasuresofeffect,butinmany casesthemostimportantimplicationsrelatetothecareful evaluationandinterpretationofmajorsourcesof hetero-geneity,knowledgegaps,andprioritiesforfutureresearch. Anotherimportantguidelineindicatesthatauthorsshould considertherelevanceoftheirfindingstovariousgroups ofend-users(e.g.researchers,practitioners,producers,and decision-makers).Therefore,authorsshouldhavea knowl-edgetransferandexchangeplantofacilitatetheuptakeand utilizationoftheirresultsbytargetedend-usersthrough multiple audience-specificformats (e.g.journal articles, conference presentations, and user-friendly summaries) (Lavisetal.,2003;Raji ćetal.,2013).

3.4.1. Recentadvancementsinsystematicreviews

3.4.1.1. Risk-of-bias assessment. An important develop-ment inthesystematic reviewprocessis inrisk-of-bias assessment.Theterm“riskofbias”ispreferredoverother termssuchas“qualityassessment”becauseitavoidsthe ambiguityofqualityofreportingvs.qualityoftheresearch. Itrecognizesthatstudiesofrelativelyhigherqualitymight stillhaveahighriskofbiasandthatsomequality crite-riamightnotbetrueindicatorsoftheriskofbias(Higgins and Green, 2011).TheCochraneCollaboration now rec-ommendsa domain-basedevaluationof therisk ofbias of includedstudies,and speciﬁc toolshave been devel-oped for reviews of interventions and diagnostic test accuracy(HigginsandGreen,2011;Whitingetal.,2011).

(7)

For interventionreviews,therecommendedapproach is forreviewerstoassessandmake judgementsaboutthe risk of bias (low, high, or unclear) for each outcome acrosssevendomains:sequencegeneration(i.e. random-ization);allocationconcealment;blindingofparticipants andpersonnel; blindingofoutcomeassessment; incom-pleteoutcomedata;selectiveoutcomereporting;andother issues(HigginsandGreen,2011).Reviewersthen summa-rizetheoverallriskofbiasforeachoutcomeineachstudy (HigginsandGreen,2011).However,recenttestingofthis toolshowedlowrevieweragreement,indicatinganeedfor moredetailedreviewerguidance(Hartlingetal.,2013).

Therisk-of-biastooldevelopedtoassessdiagnostictest accuracystudiesisreferredtoasQUADAS(Whitingetal., 2003,2011).Wilkinsetal.(2010)adaptedQUADAStothe agri-foodpublichealthcontextinasystematicreviewand meta-analysisoftheaccuracyofbacterialcultureandPCR todetectSalmonellaspp.inswine.Adomain-based risk-of-biastoolhasnotbeenformallydevelopedforreviewsthat includeobservationalstudies.However,manyothertools andchecklistsareavailabletoassistreviewersinassessing theriskofbiasofthesestudydesigns(Sargeantetal.,2005; Sandersonetal.,2007;vonElmetal.,2007;Farrar,2009).

3.4.1.2. Overallquality-of-evidenceapproachand summary-of-findings tables. The Cochrane Collaboration’s risk-of-biasassessmentispartofanoverallquality-of-evidence approachcalledGradesofRecommendation,Assessment, Development,andEvaluation(GRADE)(Guyattetal.,2011). The GRADE approach combines the study-level risk-of-bias assessment with an assessment of the directness of evidence,heterogeneity,precisionof results,and risk of publication bias to determine the overall quality of evidenceforeachoutcomeatoneoffourlevels:high, mod-erate,low,orverylow (Guyattetal.,2011;Higgins and Green,2011).Wilhelmetal.(2011a,2012)applieda modi-fiedGRADEapproachtotwosystematicreviewsinthearea ofagri-foodpublichealth:oneinvestigatingthepotentialof swineandporkproductstobeasourceofzoonotichepatitis Evirusinfectioninhumansandtheotherinvestigatingthe efficacyoffiveon-farminterventionstoreduceSalmonella

sheddingandsero-prevalenceinswine.Inbothreviews, overallevidencewasclassifiedaseither‘verylow’or‘low’, primarilyduetoinsufficientreportingofkey methodologi-calcriteriasuchasjustificationofsamplesizeandreported useofconvenienceinsteadofrandomorsystematic samp-ling. A sensitivityanalysis conducted byWilhelm et al. (2012)showedthattheGRADEratingwasincreasedfrom ‘low’to‘moderate’foralloutcomeswhenonlyevidence publishedafterrelease ofinternationalreporting guide-lines wasconsidered,indicatingthe potentialimpactof theseguidelinesonimprovingfuturereportingofprimary researchinthisarea.

ThequalityofevidencefromGRADEcanbecombined withtheoveralleffectestimates(ifapplicable)andother keyresultsinasystematicreviewandmeta-analysisfor presentationinasummary-of-ﬁndingstable(Guyattetal., 2011). The purpose of these tables is to enhance the interpretationanduptakeofsystematicreviewand meta-analysisﬁndings among end-usersbyproviding a more conciseanduser-friendlysummaryformat(Guyattetal.,

2011).However,evidencefromthehealthsectoris incon-sistent on the utility of these tables to inform various end-usersaboutknowledgesynthesisresultscomparedto thefullarticlealoneandotheruser-friendlysummary for-mats(Rosenbaumetal.,2010;Opiyoetal.,2013).Similarly, a 2012surveyof policy-makersand thosewho support them(e.g.policyanalystsandadvisors) intheCanadian agri-food public health sector found that summary-of-findingstablesweretheleastpreferredformattoinform policy-making (13% of respondents) compared to the fulljournalarticle(15%),one-pagesummaries(23%),and three-pagesummaries (49%) with additional contextual information(e.g.aboutintervention costsand practical-ity)(Phametal.,2013a).Therefore,summary-of-findings tablesmightbemoresuitableasasupplementaryresource tosystematicreviewsratherthanastand-alonesummary. Anexampleofasummary-of-findingstableadaptedtothe agri-foodpublichealthcontextcanbefoundinWilhelm etal.(2012).For assistancedevelopingthesetables,the CochraneCollaboration hascreateda softwareprogram calledGRADEpro(Brozeketal.,2008).

3.4.1.3. Stakeholder engagement.End-users are increas-ingly being involved in systematic reviews, as well as scoping reviews, as stakeholders or team members to increasetherelevance,practicality,andutilizationofthe results(Lavisetal.,2005;Keownetal.,2008).End-users are engaged through a variety of processes, including: inputmeetings and consultations(e.g. toprovide input onthereviewtopic,scope,searchstrategy,orresults dis-semination);interactivesteeringcommittees;orasafull teammemberthroughouteachstepofthereview(Keown etal.,2008).Anessentialfirststepinthesereviewsisto clearlyidentifyandengagethemostapplicablestakeholder groupsthat mightbeaffectedby orhave aninterestin theissueunderinvestigationandtoensureappropriate representationfromthesegroups.Benefitsofstakeholder engagement include a review scope and question for-mulationthataremorerelevanttoend-users,additional feedbacktoimprove theclarity and applicabilityof the methodsand results,and increasedinterest inthe find-ings and enhanced appreciation for and knowledge of SRsamongend-users(Keownetal.,2008).Potential chal-lengesincludetheneedforadditionaltimeandresources, requiredflexibilityinthereviewframework,and poten-tialforintroductionofbiasintotheprocess(Keownetal., 2008).Thestakeholder engagementprocessis currently beingusedin aninteractivescopingreviewontherole ofwildlifeinthetransmissionofpathogenicbacteriaand antimicrobial resistancein the food chain (Greiget al., 2012). The review incorporates a stakeholder advisory group of 11 industry experts and end-users to obtain feedbackandinsightsonthereviewquestion,scope,and knowledgetransfer and exchange strategy (Greig et al., 2012).

3.4.2. Keysystematicreviewchallengesandpotential solutions

3.4.2.1. Grey literature. Grey literature is deﬁned as lit-erature that is not formally published in sources such asjournalsandbooksandthat isgenerallynot indexed

(8)

in online bibliographic databases (Higgins and Green, 2011). Examples of grey literature include government andindustryreports,conferenceproceedings,andtheses anddissertations.ACochranereviewofrandomized con-trolledtrialsofhealthcareinterventionsfoundthatgrey literaturetrialstendedtobesmallerandshowedalower overalltreatmenteffectcomparedtotrialsinthepublished literature(Hopewelletal.,2007b).Inaddition,ina system-aticreviewoftheeffectofhazardanalysiscriticalcontrol pointprogrammestoreducemicrobialcontaminationof food-animalcarcassesatabattoirs,onlyalimitedamount of relevant literature was identified, and three of the largeststudieswerenon-peer-reviewedarticlesobtained fromanInternet search (Wilhelm etal., 2011b). There-fore,exclusionofgreyliteraturecouldpotentiallyimpact systematicreviewconclusions.However,manygrey liter-aturedocumentsarenotpeer-reviewed,theyaredifficult andtime-consumingtoaccess,andtheymightnotreport sufficient information to allow risk-of-bias assessment, dataextraction,andmeta-analysis(Eysenbachetal.,2001; Hopewelletal.,2007b;Doshietal.,2012).

Grey literature can be identified through internet searches,butcomprehensivesearchingcanbeachallenge becausesearchenginessuchasGooglearenotdesigned forcomplexqueriesthatareusedinsystematic reviews andthereisnoindicationofwhatthelimitsofanInternet searchshouldbe(Eysenbachetal.,2001).Some special-izeddatabasescanbeusedtosearch forgreyliterature (e.g.aScopuswebsearchfunctionthatconductsfiltered searches for science-specific information), and engage-ment of stakeholders in reviews can also be useful to identifythesesources.Giventheaboveconsiderations,we recommendthat authorsconsiderincluding grey litera-turewhenunpublishedorproprietaryinformationmight beexpectedbasedonthenatureofthetopic(Wilhelmetal., 2011b).Inaddition,inclusionofgreyliteraturemightbe necessarywheninvestigatingcomplexandpolicy-relevant questionsthatrequireanalysisofcontextualinformation, whichmightonlybefoundinthesesources(Greenhalgh andPeacock,2005).Inbothcases,authorsshouldinclude acarefulevaluationanddiscussionoftheriskofbiasand potentialimpactsofthesesourcesonthereviewfindings.

3.4.2.2. Language bias.Language biasis a potential con-cernifnon-Englishstudiesareexcludedfromasystematic review.Forexample,Eggeretal.(1997b)foundthatauthors ofrandomizedcontrolledtrialsweremorelikelyto pub-lishstatisticallysignificantfindingsinEnglish-comparedto German-languagejournals.However,otherauthorsinthe healthsectorhavefoundthatexcludingnon-English stud-iesmightnothaveanydiscernibleeffectsontheoutcome forsomeinterventions(Jünietal.,2002;Moheretal.,2003; Phametal.,2005).Thepotentialimpactoflanguagebiasis difficulttopredict,andthedecisiontoincludenon-English articlesshouldbemadebasedonthespecificquestion, con-text,andscopeofthereviewandavailabilityofresources fortranslation(Jünietal.,2002;Moheretal.,2003).

Inourexperience,mostsystematicreviewsinagri-food publichealth identifya relativelysmallnumberof non-Englishlanguagearticlesthatappeartoberelevantbased onrelevance screening of the abstract (Wilhelm et al.,

2011b;Mederosetal.,2012;Tuˇsevljaketal.,2012).Inthese situations,exclusionofnon-Englisharticlesisnotlikelyto havealargeimpactonthereviewﬁndingsduetothe gen-erallysmallproportionofthesestudies.However,inother situations,thenatureofthereview topicand scopecan resultinalargerproportionofpotentiallyrelevant stud-iesbeingexcludedduetolanguage(Wilhelmetal.,2009, 2011a;Ilic etal.,2012).Iftheauthorshavea reasonto believethatexclusionofnon-Englisharticlesmightimpact theconclusionsandresourcescannotbereadilyobtained fortranslation, authorsshouldacknowledgeand discuss thepotentialsigniﬁcanceofthisbiasandattemptto inves-tigateandevaluatethecharacteristicsofarticlesthatare excludedbasedontheirtitlesandabstracts(e.g.potential under-representationofageographicalarea).

3.4.3. Optimizingthesearchstrategy

It is criticaltothevalidity ofa systematic reviewto ensurethatallrelevantandavailableknowledgeis iden-tified.However,giventhetimeandresourcerequirements involvedincomprehensiveliteraturesearches,anoptimal balanceisneededintheselectionofsearchtermsand elec-tronicdatabases(RoyleandWaugh,2003;Lovarinietal., 2006; Waddelletal., 2008).Thecomplexity ofa search dependslargelyonthebreadthandscopeofthereview question.Waddelletal.(2008)evaluatedtheeffectiveness of brief search strategies for three completed system-aticreviewsinagri-foodpublichealthand foundthat a combinationofthreebroadandsubject-specificdatabases for 2/3reviewscaptured>90%ofrelevantcitations, and nearlyallrelevantcitationswereidentifiedaftera thor-oughsearchverificationstrategywasused.Basedonour experiencesandpreviousresearch(Waddelletal.,2008; Grindlayetal.,2012),werecommendaselectionof3–5 general(e.g.PubMed,Scopus,andCurrentContents Con-nect)andsubject-specific(e.g.CABDirectandFoodScience andTechnologyAbstracts)databasestoprovidesensitive resultswithout usingexcessive resources.Thedatabase searchesshouldbesupplementedwithsearchesforgrey literature (where appropriate)and a search verification strategy(RoyleandWaugh,2003;Hopewelletal.,2007a). Todevelop anappropriate searchalgorithm,we recom-mend that reviewers select a range of 20–30 relevant articles,extractkeytermsfromtheirtitlesandabstracts, andthroughadocumentedtrial-and-errorprocess deter-mine thecombination of key terms that results in the highestpercentageofrecoveryofthesearticlesinoneor moretargeteddatabases.Thisprocessshouldbeconducted inconsultationwithalibrarianorinformationspecialist.

3.4.4. Othersystematicreviewchallenges

Currentlythereisnoconsensusaboutthebestmethods todeterminewhenandhowtoupdatesystematicreviews (Moheretal.,2008;HigginsandGreen,2011;Tsertsvadze et al.,2011).TheCochraneCollaboration’spolicyis that asystematicreviewshouldbeupdatedeverytwoyears, but a review ofCochranereview updates from1998to 2002foundthat only9%ofupdatedreviewsresultedin changestotheconclusions,indicatingthatthedecisionto updateareviewshouldbebasedmoreonprioritythanon timealone(Frenchetal.,2005).Indeterminingwhether

(9)

toupdate a systematic review, authorsshouldconsider thepotential fornew researchtobepublishedand the natureoftheresearchquestionorissue(Frenchetal.,2005; Tsertsvadzeetal.,2011).Anotherchallengeinsystematic reviewsisthepotentialinclusionofobservationalstudies, which havetraditionallybeenexcludedfromreviewsof healthcareinterventionsbecausetheyaremoreproneto biasandprovideweakerevidencecomparedto random-izedcontrolledtrials(Eggeretal.,1998;HigginsandGreen, 2011).However,observationalstudiesareoftentheonly feasibledesign wheninvestigating riskfactor questions, andtheytendtohavelargersamplesizeswhicharemore representativeofbroaderpopulations(Eggeretal.,1998; Shrieretal.,2007).Therefore,itmaybebeneﬁcialtoinclude observational studies in somesituations to ensure that allavailableknowledgeisconsideredfordecision-making (Shrieretal.,2007).

Otherlogisticalchallenges includesecuringsufﬁcient resources and establishing a diverse, multidisciplinary teamwiththenecessaryexpertisetoappropriately con-duct systematic reviews within reasonable timeframes. In addition, many systematic reviews are conducted underresearchandacademicsettings,whichlimitstheir potential timeliness and utility to inform policy- and decision-making contexts (Lavis et al., 2005). Planning, investment,andintegrationofknowledgesynthesis meth-odswithinregional,national,andinternationalagri-food publichealthagenciesisnecessarytosupportandmaintain sufﬁcientcapacityandinfrastructureinthisarea.

3.5. Meta-analysis

Theobjectiveofatraditionalmeta-analysisisto com-binetheresultsofhomogenous studiesusingtheeffect estimateand uncertainty fromeach studytoproducea weightedmeanor overallmeasureof effect(Borenstein et al., 2009; Higgins and Green, 2011). Meta-analysis shouldalwaysbeprecededbya systematicreview, and authorsmustevaluatetheirdatasetforsuitabilityto con-ductameta-analysis.Meta-analysisshouldbeconsidered if there are groups of studies that are evaluating the sameeffectinsimilarsettingsandpopulations.Any poten-tialsourcesofvariationorheterogeneityintheoutcome measures should be predicted and deﬁned before con-ducting meta-analysis tojustify the biological basis for combiningstudies.Thereporteddataandoutcomesshould then be evaluated to determine the most appropriate measure of effect.For dichotomousoutcomes, itis rec-ommendedtouserelative(e.g.oddsratiosorriskratio) ratherthanabsolute(e.g.riskdifference)measuresofeffect becausetheyaregenerallymoreconsistentacrossstudies (Deeks,2002;Borensteinetal.,2009;HigginsandGreen, 2011).

Oncethedesiredmeasureofeffectisspeciﬁed,authors mustdeterminewhethertouseaﬁxed-orrandom-effects meta-analysismodel(Table2).Random-effectsmodelsare thepreferredoptionwhenauthorsexpectthatthestudies willvaryduetoreasonsotherthanrandomerrorandwhen theywanttoestimateanoverall measureof effectthat canbegeneralizedtoarangeofpopulations(Borenstein etal.,2009;Higginsetal.,2009;HigginsandGreen,2011).

Whenheterogeneityis detected(i.e.thebetween-study variance,2_,_is_>0),_{consideration}_of_the_ﬁxed-_vs. random-effectsmodeliswarrantedbecausethelattermodeltends toadjust theweightoflargerstudiesdownandsmaller studiesuptogiveamorebalancedsummaryeffectestimate (Borensteinetal.,2009).Thisisofparticularinterestwhen thereareextremeoutcomemeasuresinsmallorlarge stud-ies(Borensteinetal.,2009).Meta-analysisresultsshould bedisplayedintheformofaforestplot,whichareusedto visualizevariationintheeffectestimatesacrossstudiesand theirweightedcontributiontotheoverallestimate(Lewis andClarke,2001).Anexampleofaforestplotfroma meta-analysisoftheeffectofcompetitiveexclusionproductsto reducecolonizationofSalmonellaspp.inbroilerchickens onfarmsisshowninFig.2(Kerretal.,2013).

Authorsshouldthenquantifyheterogeneityintheeffect estimates(Table2).Achi-squaredstatistic(Cochran’sQ) canbeusedtotestforthepresenceofheterogeneity,but it haslow power,soa liberal Pvalueof ≤0.10 is often usedtoindicatestatisticalsignificance(Higginsetal.,2003; Ioannidis,2008).Amoreinformativeapproachisto quan-tifyheterogeneityusingI2_,_which_measures_the_percentage (0–100%)oftotal variation acrossstudiesthat isdue to heterogeneity rather than chance (Higgins et al., 2003; Ioannidis,2008).However,thistestalsosuffersfromlow poweranditisrecommendedthattheI2_confidence inter-valalsobecalculated(Ioannidis,2008).Ifheterogeneityis identifiedandisconsideredimportantforexploration(e.g.

I2_>_25%),_sub-group_analysis_or_{meta-regression}_should_be used(ThompsonandHiggins,2002;Higginsetal.,2003; Higgins and Green, 2011). Meta-regression investigates whetherstudy-levelcovariatesexplainanyofthe hetero-geneityintheeffectestimatesbetweenstudies(Thompson andHiggins,2002).However,toavoidtheidentification ofspurious relationships, theseanalysesshouldonly be conductedusingalimitednumberofpre-specified vari-ablesthatareidentifiedwhenconsideringthesuitability ofundertakinga meta-analysis(ThompsonandHiggins, 2002;HigginsandGreen,2011).

Thenextstepinmeta-analysisistotestforthepresence ofsystematicbiases(e.g.publicationbias)andtoconduct sensitivityanalysis(Table2).Funnelplotscanbeusedto visualizetherelationshipbetweenthemeasureofeffectfor eachstudycomparedtoitsprecision(i.e.standarderror), andstatisticaltests(e.g.Egger’sregressiontest)are avail-abletotestthisrelationship.AnexampleisshowninFig.3. Asymmetryin thefunnel plot mayindicate publication biasorothersmall-studyeffectssuchasselectiveoutcome reporting,differencesintheriskofbias,ortruedifferences intheinterventioneffectduetostudysize(Eggeretal., 1997a;Higginsetal.,2009;Sterneetal.,2011).However, these tests are not reliable when there are <10 stud-ies,whenthereissubstantialheterogeneity(e.g.I2_>_50%), whennoneofthestudiesaresigniﬁcant,orwhenthe vari-ance ratiobetweenthesmallestand largeststudyis >4 (Eggeretal.,1997a;IoannidisandTrikalinos,2007;Sterne etal.,2011).Sensitivityanalysisshouldbeconductedto testtherobustnessofthemeta-analysiseffectestimates againstarbitraryoruncertaindecisions(e.g.exclusionof studiesbasedoncharacteristicssuchasstudytype, popu-lation,intervention,oroutcome).

(10)

Table2

Anoverviewofmajormeta-analysismethodsandselectedkeyresources.

Meta-analysismethoda _Brief_method_description _Key_resources

Traditionalmethods

Fixed-effectmodel •Assumesthateachstudyestimatesthesameinterventioneffect

•Severalweightingoptionsareavailable,includinginversevariance(oftenthe default),Mantel-Haenszel(usedfor2×2data),andPeto(usedforspecialcasesof sparse2×2data)

SuttonandHiggins (2008)

Random-effectsmodel (DerSimonianand Lairdweighting)

•Assumestheinterventioneffectsfollowadistributionacrossstudies

•Incorporatesthebetween-studyvariationusingamoment-basedestimateand assumesanormaldistribution

DerSimonianandLaird (1986),DerSimonian andKacker(2007),

Higginsetal.(2009) Heterogeneitytests •Forestplotscanbeusedtovisuallyevaluatetheconsistencyofstudyresults,butthis

methodissubjective

•TheChi-squaretest(Qstatistic)canbeusedbutsuffersfromlowpowerwhenthere arefewstudiesandwhenstudieshavesmallsamplesizes

•I2_can_be_used_to_quantify_the_proportion_of_total_variation_between_studies_that_is

attributabletoheterogeneity

Ioannidis(2008)

Evaluationofsourcesofheterogeneity

Sub-groupanalysis Anevaluationofcategoricalstudy-levelcovariatesbyconductingseparate meta-analysesoneachcategoryandtestingforhomogeneityacrosscategories

SuttonandHiggins (2008),Borenstein etal.(2009) Meta-regression •Conductedtoinvestigatewhetherstudy-levelcovariatesexplainanyofthevariation

intheinterventioneffectsbetweenstudies

•Shouldonlybeconductedusingpre-speciﬁedcovariatesandwhen≥10studiesare availableforthemodelandeachadditionalcovariate

•Ispronetotheecologicalfallacy

ThompsonandHiggins (2002),Suttonand Higgins(2008)

Systematicbias assessment

•Systematicbiases(e.g.publicationbias)canbeinvestigatedbyexploringthe presenceofasymmetryinfunnelplots

•Avarietyofstatisticaltests(e.g.Begg’stestandEgger’stest)andsensitivityanalyses (e.g.thetrimandﬁllmethod)canalsobeused

•Thesemethodsshouldonlybeusedwith≥10studies,whenthereislittleorno heterogeneity,whensomestudieshavestatisticallysigniﬁcantresults,andwhen thereisvariationinstudysize

Ioannidisand Trikalinos(2007),

HigginsandGreen (2011),Sterneetal. (2011)

Advancedmethods Individualparticipant

datameta-analysis

•Regardedasthe“goldstandard”approachtometa-analysis

•Hasmanyadvantages,includingabilitytohandletime-to-eventdata

•Notabledisadvantageistheincreasedtimeandcostsrequiredtoobtain,format,and analyzedataandpotentiallyobtainadditionalethicsapproval

Rileyetal.(2010)

Random-effects meta-analysiswith complexdata

•Correlationstructurescanbeusedformeta-analysisofcomplexdatawhen dependant,multi-groupcomparisonsarereportedandwhenmultipleoutcomesor time-pointsarereportedineachstudyandaremeasuredonthesameparticipants

Borensteinetal.(2009)

Bayesianmeta-analysis •Canspecifyﬂexibleﬁxed-orrandom-effectsmodelsbasedontheBayesian framework

•Manybeneﬁts,includingabilitytoaccountforfulluncertaintyinallparameters

•Requirescarefulconsiderationandjustiﬁcationofpriordistributionsselectedand sensitivityanalysistoexploretheireffectontheﬁndings

SuttonandAbrams (2001),Higginsetal. (2009)

Multivariateand network meta-analysis

•Developedtomakeinferencesaboutstudiesthatreportmultiplecorrelated outcomes,whereeachstudyprovidesthewithin-studycovariancematrix

•Someapplicationsincludetheevaluationofdiagnosticteststudies(seebelow), networkmeta-analysis,andtoexamineassociationsingeneticstudies

•Network(ormultiple-treatmentscomparison)meta-analysisisonecommon applicationthatcanbeusedtoextendthesimplepairwisecomparisonofatraditional meta-analysistoincludemultiplecomparisonsacrossanumberofinterventiongroups

•Within-studycorrelationsarerequiredbutareoftenunknownandthereisno consensusaboutthebestapproachtoaddressthisissue

Hoaglinetal.(2011), Jacksonetal.(2011), Jansenetal.(2011) Meta-analysisof diagnostictest accuracy

•Aspecialcaseofmultivariatemeta-analysisfordiagnostictestaccuracystudies

•Severalmethodsavailable,includingsimplepoolingofsensitivityand/orspeciﬁcity withoutaccountingforthecorrelationbetweenthem,summaryROCcurves (Littenberg-Mosesmethod),andthebivariaterandom-effectsmodelandhierarchical summaryROCmodel

Harbordetal.(2008),

Macaskilletal.(2010)

a_All_{meta-analysis}_methods_except_advanced_Bayesian_approaches_can_be_conducted_in_standard_statistical_software_(e.g._SAS,_STATA,_and_R)._The

stand-alonesoftwareprogramComprehensiveMeta-Analysis(CMAversion2,http://www.meta-analysis.com/)canalsoconducteachofthetraditionalmethods, buttheirmeta-regressionfunctionalityiscurrentlylimitedtounivariateanalysiswithcontinuouscovariates.Anotherstand-aloneprogramcalled Meta-AnalysiswithInteractiveeXplanations(MIX2.0,anExceladd-in,http://www.meta-analysis-made-easy.com/)cansupportalltraditionalmeta-analysis methodsexceptmeta-regression.BayesianapproachescanbeconductedusingthespecializedsoftwareWinBUGSandOpenBUGS.

3.5.1. Recentadvancementsinmeta-analysis

3.5.1.1. Bayesianmeta-analysis. Meta-analysiscanalsobe conductedwithintheBayesianstatisticalframework,and although this approach has been long established and

appliedinthehealthandothersectors(SuttonandAbrams, 2001),ithasnotbeenwidelyadoptedinsystematicreviews of agri-food public health issues. Some advantages of Bayesianmeta-analysisincludetheabilitytomakedirect

(11)

Fig.2. Exampleofaforestplotfromarandom-effectsmeta-analysisofchallengetrialsreportingtheeffectofPreemptTM_(CF-3)_to_reduce_the_odds_of Salmonellaspp.colonizationinbroilerchickens(reproducedfromKerretal.,2013).Trialsarestratifiedbyageofbirdsatfinalsampling.Theestimateof interventionefficacy(oddsratio)and95%confidenceintervalfromeachindividualtrialwithineachstudyarerepresentedasuniquerowsandthesizeof theboxsurroundingeachestimaterepresentstherelativeweightofthattrialinproducingtheaverageeffectestimate.Inthiscase,theaverageestimate showsasignificantbeneficialeffect(OR=0.04,95%CI=0.03–0.06)andalowamountofheterogeneity(I2₌_6.4%,_Q_test_P_value₌_0.362).

probabilityandpredictivestatements(conditionalonthe currentknowledge)andtheabilitytoincorporate exter-naljudgementsandadditionalknowledge(e.g.costsand utility)intopriordistributions(SuttonandAbrams,2001; Higginsetal.,2009;HigginsandGreen,2011).TheBayesian frameworkcanalsobeusedtoperformcomplexanalyses such as multivariate and network meta-analysis, hier-archical models, and generalized synthesis models that incorporate multiplestudy designs (Higgins and Green, 2011;Jansenetal.,2011).However,therearesome poten-tialdisadvantages of a Bayesianapproach:prior beliefs can besubjective(to avoid this, anuninformative prior canbeused);therearemultiplepriordistributions that canlead todifferentresults;there isnodirect measure ofstatisticalsigniﬁcanceanalogoustoaPvalue,although credibility intervalscan be calculated;and theycan be

computationallydemandingandrequirespecialized sta-tisticalexpertise(SuttonandAbrams,2001;Higginsand Green,2011).Sensitivityanalysesshouldalwaysbe con-ductedinBayesianmeta-analysisduetothesubjectivity involved in selecting a prior distribution (Sutton and Abrams,2001).

3.5.1.2. Multivariate and network meta-analysis. Multi-variatemeta-analysisreferstothesynthesisofstudiesthat reportseveralinterrelatedoutcomeparameters(Jackson etal.,2011).Oneofthemostcommonapplicationsof mul-tivariatemeta-analysisisthesynthesisofdiagnostictest accuracystudies(describedbelow).Anotherspecial appli-cationisnetworkmeta-analysis,alsoreferredtoas mixed-ormultiple-treatmentsmeta-analysis(Glennyetal.,2005; Hoaglin et al.,2011; Jansen et al., 2011; Salanti,2012).

(12)

Fig.3. Exampleofafunnelplotfromarandom-effectsmeta-analysisof challengetrialsreportingtheeffectofPreemptTM_(CF-3)_to_reduce_the_odds

ofSalmonellaspp.colonizationinbroilerchickens(Kerretal.,2013).The graphplotstheestimateofeffect(oddsratio)fromeachtrialagainstits standarderror.Smallerstudiestendtoscattermorewidelyatthebottom oftheplotduetotheirlowerprecisionandlargerstudiesclustertowards thetop.Ifpublicationbiasweresuspected,thiscouldbevisualizedbya lackofstudiesinthelowerrightcorner(suggestingthatsmallerstudies showinganon-signiﬁcanttreatmenteffectarelesslikelytobepublished). Inthisﬁgure,noevidenceofpublicationbiasissuggestedbasedonvisual examinationoftheplotsymmetry.Begg’srankcorrelationtest(P=0.230) andEgger’sregressiontest(P=0.742)alsodidnotindicateevidenceof asymmetry.

Networkmeta-analysiscansimultaneouslyevaluatethree or more interventions using direct comparisons within atrialand indirectcomparisonsacrosstrialsbasedona commoncontrolgroup(Glennyetal.,2005;Hoaglinetal., 2011;Jansenetal.,2011).Resultsfromtrialsthatdirectly comparethetwo(ormore)interventionscanbecombined withtheindirectresultsasa weightedaverage(Hoaglin etal.,2011;Jansenetal.,2011).Beneﬁtsofthisapproach includeimprovedprecisionintheestimatedeffectsizes and theability tocompare interventions that have not beendirectlycomparedinagivenstudy(Salanti,2012).

Multivariateand network meta-analysis canbe con-ducted usingeither a frequentistor Bayesianapproach, thelatterhastheadditional benefitofranking all com-paredinterventionsformoreintuitiveinterpretationofthe results(Hoaglinetal.,2011;Jansenetal.,2011).O’Connor et al.(2013) recentlyapplied network meta-analysis to investigatethe efficacy of various antibiotictreatments forbovinerespiratorydiseaseinbeefcattle.Thistypeof meta-analysishasincreasedrapidlyinthelastfewyearsin thehealthsector,andanumberofmethodologicalissues havebeenidentified (Hoaglin et al.,2011;Jansen etal., 2011;Salanti,2012).Forexample,theappropriatenessof thenetworkmeta-analysismodeldependsonaccurately specifyingthedirect andindirectrelationshipsbetween theinterventionscompared,aswellasassumptionsthat theincludedstudiesareclinically and methodologically similarandthatindirectevidenceisconsistentwithdirect evidence(Hoaglinetal.,2011;Jansenetal.,2011;Salanti, 2012).In responsetotheseconcerns, guidelinesfor the conduct and reporting of network meta-analysis have recentlybeenpublished(Hoaglinetal.,2011;Jansenetal., 2011).Additionalresourcesaboutthesemethodologiesare

availablefrom thefollowing Multiple-Treatments Meta-Analysiswebsite:http://www.mtm.uoi.gr/.

3.5.1.3. Meta-analysis of diagnostic test accuracy studies.

Meta-analysisofdiagnostictestaccuracystudiespresents somechallenges becausemultiple outcomes(e.g. sensi-tivity, specificity,and likelihood ratios)are reportedfor eachstudy.Thefirstrecommendedstepistocreatea sum-maryreceiveroperatingcharacteristic(ROC)graphofthe resultsofeachstudy(Leeflangetal.,2008;Macaskilletal., 2010).Forestplotscanalsobeproducedtoshowstudy esti-matesofsensitivityandspecificity,buttheydonotreflect thecorrelationandthresholdrelationshipsbetweenthese twovalues (Leeflangetal.,2008;Macaskilletal.,2010). These relationshipscanbeincorporated in two special-ized models:the hierarchicalsummary ROC model and thebivariaterandom-effectsmodel(Harbordetal.,2008; Leeflang et al., 2008; Macaskill et al., 2010). The mod-elshavedifferentparameterizations,buttheyareclosely relatedandestimatesfromeithermodelcanbeusedto gen-erateasummaryROCcurve,summaryoperatingpoint,and confidenceand predictionregions(Harbordetal.,2008; Leeflangetal.,2008;Macaskilletal.,2010).Inaddition, bothmodelscanincludeoneormorecovariatestoexplore heterogeneity(Harbordetal.,2008;Leeflangetal.,2008; Macaskilletal.,2010).Wilkinsetal.(2010)useda hier-archical summary ROC model toinvestigate sources of heterogeneityinthesensitivityofcultureandsensitivity andspecificityofPCRtodetectSalmonellaspp.inswine, andtheyfoundthattheformer wasinfluencedmoreby differencesinindividualtestprotocolswhilethelatterwas influencedmorebydifferencesbetweenstudies(sample typeandsamplesize).Theyalsofoundahighriskofbias amongincludedstudies,indicatinganeedformoreformal andstandardizedconductandreportingoffutureprimary researchinthisarea(Wilkinsetal.,2010).Anexampleofa summaryROCcurvefromWilkinsetal.(2010)isshownin Fig.4.

3.5.1.4. Individualparticipantdatameta-analysis.Analysis of individualparticipantdataisanalternative approach tometa-analysisthatisgainingpopularityinthehealth sector (Riley et al., 2010; Higgins and Green, 2011). It requiresthattheauthorsobtaintherawdatafromeach studyinsteadofrelyingonaggregatedataobtainedfrom publicationsandreports.Someimportantbeneﬁtsinclude theabilitytoanalyzedatainonestepwhileaccountingfor clusteringbystudy;toexplorebothstudy-and individual-levelcharacteristics;toanalyzetime-dependantstudies; to re-analyze and standardize themethod of statistical analysisacrossstudies; toverifydataand assumptions; and topotentially include studies that did not provide sufﬁcient orappropriatedataintheoriginalpublication (Rileyetal.,2010).Somenotabledisadvantagesincludethe considerableincreasedtimeandcostsrequiredtoobtain individualparticipantdatafromstudyauthors,formatand analyzedata,andpotentiallyobtainethicsapproval;the potentialbiasintroducedduetomissingdatasets;and chal-lengesin combining theindividualwith aggregatedata toincrease thenumberofstudiesincluded(Riley etal., 2010).

(13)

Fig.4.ExampleofasummaryROCcurvefromahierarchicalsummary ROCmeta-analysisofthediagnosticaccuracyofPCRcomparedto cul-turetodetectSalmonellaspp.inswine(reproducedfromWilkinsetal., 2010).Thegraphillustratesthetrade-offbetweentestsensitivity(yaxis) andspecificity(xaxis).Thesolidsquareindicatesthesummarypoint esti-mateofsensitivityandspecificity,withthe95%confidenceregionaround thisestimateshownasadashedline.Thegraphalsoshowsthe95% pre-dictionregion(dottedline),whichindicatestherangeofvalueswewould expect,with95%confidence,thetruesensitivityandspecificityofafuture studytolie(Macaskilletal.,2010).Inthisfigure,twopredictionregions areshown,onewithall21testevaluations(A),andanotherwithone influentialobservationremoved(B)(Wilkinsetal.,2010).

3.5.2. Keymeta-analysischallengesandpotential solutions

3.5.2.1. Meta-analysisof observationalstudies.If observa-tional studies are includedin a meta-analysis, it might bepreferabletoextractanduseadjustedeffectestimates ratherthanrawdataduetotheriskofconfoundingbias in these studies (Higgins and Green, 2011). If multiple adjustedestimatesarereportedforsomestudies,authors couldusetheestimatesfromtheﬁnalmodelorfromthe modelthatcontrolledforthemostimportantconfounding variables (Higginsand Green, 2011).It is generally rec-ommendedthatauthorsdonot combinemultiplestudy designsinthesamemeta-analysisbecauseresultsfrom dif-ferentdesignscandiffersystematically(Borensteinetal., 2009; Higgins and Green, 2011).Instead, theimpact of studydesign ontheoverallmeasureof effectshouldbe explored in meta-regression and other advanced mod-els.TheBayesianframeworkoffersanaturalextensionto accountforthevariation betweenstudydesigns(Sutton and Abrams, 2001;Higgins et al.,2009).Evidence from observationalstudiescanbeincorporatedintoprior dis-tributionsor canbedirectlymodelledalong withother study designs in a generalized synthesis model (Sutton

and Abrams,2001; Higgins et al.,2009).While the lat-terapproachcanassesstheinﬂuenceofobservationaland otherstudydesignsonthemeta-analysiseffectestimates, itdoesnotexplicitlyaddressthepotentialbiasesof obser-vationalstudies(SuttonandAbrams,2001;Higginsetal., 2009).

3.5.2.2. Insufficient data reporting.It is often difficult to obtain required data to conduct meta-analysis due to poorandinconsistentreportingofprimaryresearch stud-ies(Higginsand Green,2011; Bucheret al.,2012a).For example,inasystematicreviewandmeta-analysisof inter-ventions toreduceSalmonella contaminationofchicken carcassesatprocessing,nearly30%ofeligiblestudieswere excludedduetoinsufficientreportingofdata(Bucheretal., 2012a).Formostcontinuousanddichotomousoutcomes, formulasareavailabletoconvertoutcomesandvariance parameterswhentheyarenotreportedinthedesired for-mat(e.g.estimatingstandarddeviationsfromreportedP

values)(Higgins and Green, 2011;Bucher etal., 2012a; Mederoset al.,2012).Ifraw data arenot reportedand conversionscannotbeconducted,authorscanattemptto contactthestudyauthorstoobtainthenecessary informa-tion(YoungandHopewell,2009).However,thispractice is resource-intensive and authors are often not ableor willingtoprovidetherequesteddatafor manyreasons, includingconﬁdentialityconcerns(Sargeantetal.,2007a,b; Rileyetal.,2010).Recentreportingguidelinesforprimary research(e.g.REFLECTforcontrolledtrialsconductedin livestockpopulationsandSTROBEforobservational stud-ies)shouldhelptoimprovedatareportingoffuturestudies inagri-foodpublichealth(vonElmetal.,2007;Sargeant etal.,2010).

3.5.2.3. Small number of studies and significant hetero-geneity. Insufficientreporting and availability of datais oftenaconcerninsystematicreviews,butmeta-analysis is still technically feasible with as few as two studies (Ioannidisetal.,2008;Borensteinetal.,2009;Valentine etal.,2010).Whileameta-analysisofasmallnumberof studies will generate uncertain estimates, it might still be preferable to narrative or semi-quantitative synthe-ses (e.g. vote counting), which can be less transparent andmightmisinterprettheresults(Ioannidisetal.,2008; Borenstein et al., 2009; Valentine et al., 2010). Signif-icant heterogeneity is another common reason for not conducting meta-analysis, but there is inconsistency in the recommendations for how best to account for this and whetheroverall measures of effect should be cal-culatedin itspresence(Ioannidiset al.,2008).Different methodologicalapproachestometa-analysisareavailable dependingonthesourceofheterogeneityidentified(e.g. differencesinstudydesign,populations,interventions,or outcomes),andtheiruseshouldbeexploredbysystematic reviewandmeta-analysisauthors(Ioannidisetal.,2008). Someoftheseapproaches arehighlightedabove andin Table2. Furthermethodological developmentand eval-uationofthesemethodsisnecessaryinagri-foodpublic health.

(14)

4. Futureopportunitiesforknowledgesynthesis

methodsinagri-foodpublichealth

Weidentifiedanddiscussedfivekeyknowledge syn-thesismethodsthatcanbeusedindifferentsituationsto supportevidence-informed decision-makinginthe agri-foodpublichealthsector.Althoughwerefertoeachofthese approaches as unique methods, many couldbe consid-eredasmodifiedversionsofasystematicreviewadapted to specific contexts (e.g. scoping reviews for broader, policy-drivenquestions;rapidreviewsforurgenttimeand resourcerestrictions;andqualitativeandmixed-method reviewswhen contextual dataneedsto beconsidered). Scopingreviewsarevaluabletoresearchersasafirststep beforeconductingasystematicreviewinaspecifictopic areaandtheyarealsousefultogovernmentsandfunding agenciestodeterminepriorityareasforfuture research. Governmentagenciesin particularcouldbenefit froma wider adoption of structured rapid reviews toenhance thetransparencyandaccountabilityofevidence-informed decision-makinginemergencysituationssuchasduring outbreakresponse.Mixed-methodandqualitativereviews willbecomeincreasinglyrelevantasthepublicationof pri-maryqualitativeresearchgrowsinthissector.Systematic reviewsand meta-analysisarethemostwidelyadopted andappliedknowledgesynthesismethodsinthis sector andtheirutilityincludesabroadrangeofsituations,from informingend-usersaboutthemostefficacious interven-tionstogeneratingpreciseandcredibleinputstoinform risk assessments (Sargeant et al., 2006a,b; EFSA, 2010; HigginsandGreen,2011).

Riskanalysis,which consists ofthe key components ofriskassessment,riskmanagement,andrisk communi-cation,hasa longhistory ofapplicationintheagri-food public health sector to support food safety decision-making (Codex Alimentarius Commission, 1999). There is increasing momentum to formally link the use of knowledgesynthesismethodswithriskassessmentsand multi-criteriadecisionanalysisinthissectortoenhance thecredibilityandtransparencyoftheriskanalysis pro-cess(EFSA, 2010; Bucher et al., 2012b; Wilhelm et al., 2012;SmadiandSargeant,2013).Forexample, system-aticreviewsandmeta-analysishavebeenconductedand resultsusedasinputs toinformaquantitativeexposure assessmentoffarmandprocessinginterventionsto con-trolSalmonellainbroilerchickens(Bucheretal.,2012b); a quantitative risk assessment of human salmonellosis due to consumption of Canadian broiler chicken meat (Smadi and Sargeant, 2013); and a multi-criteria deci-sionanalysistoprioritizeselectedon-farminterventions to control Salmonella in swine (Wilhelm et al., 2012; personal communication,Dr. SarahParker). In addition, EFSAhasformallyadoptedsystematicreviewsand meta-analysis to inform their routine food and feed safety assessments(EFSA,2010),and theFoodand Agriculture Organization of the United Nations has begun to use thesemethods tosupporttheirscientiﬁc advicefor the CodexAlimentariusCommission.Thereisaneedforother agri-foodpublichealthorganizationsand agencies glob-allyto enhance theintegration of knowledge synthesis methods within the risk analysis paradigm to support

morerobustandtransparentdecision-makinginthis sec-tor.

Many systematic review advancements (e.g. risk-of-bias assessmentsand GRADE)were developedwiththe goal of improving the interpretation and utilization of synthesized knowledgeamong end-users inthe health-caresector(Guyattetal.,2008;HigginsandGreen,2011). Asaresult,therearestillseveralunresolvedissueswith the application of these advancementsin theagri-food publichealth sector, and futureresearch in this areais needed.Forexample,interventionresearchinthissector often usesstudy designs suchasbefore-and-after trials and challengetrials(Sargeantetal.,2006a,b, 2010), and thereisnoconsensusabouthowthesedesignsshouldbe assessedandratedunderthedomain-basedriskofbiasand GRADEframeworks. Stakeholder engagementin knowl-edgesynthesisisanotherkeydevelopmentthathasbeen shown toincrease therelevanceand uptake of system-aticreviewsandmeta-analysisﬁndingsamongend-users (Keownetal.,2008),andtheuseofthisapproachshouldbe consideredinfutureknowledgesynthesisresearchof agri-foodpublichealthissues. Logisticaladvancements,such astheuseofspecializedreviewsoftware(e.g.DistillerSR or RevMan), should also beconsidered tofacilitate the conductandmanagementoffutureknowledgesynthesis research.Inourexperience,theseprogrammesare cost-effective becausetheydecrease timespentonactivities suchascitationprocessingand theydecreasethe likeli-hoodthathumanerrorcouldoccurduetotheautomation ofvariousfunctions.

Many meta-analysis advancements covered in this reviewhavethepotentialtoimprovetheoverall synthe-sisofagri-foodpublichealth datain complexsituations (e.g.whenthereisaneedtocomparemultiple interven-tions, outcomes,orstudydesigns),and furtherresearch isnecessaryontheirapplicationinthissector.However, theincreasinguseofthesemethodscanmakeitdifﬁcult tomaintaincoherenceandconsistencyofknowledge syn-thesisresultsand interpretation.In addition,theuseof thesemethodshaslimitedbeneﬁtandutilityiftheanalysis is inappropriately conducted, too complex, or not eas-ilyunderstandableamongend-users.Thus,weencourage reviewauthorsandmeta-analyststocarefullyconsiderthe objectiveoftheirsynthesis,knownandexpectedsourcesof heterogeneity,andthequalityoftheobtainabledataprior toembarkingontheiranalysis.

There is a need to build stronger knowledge syn-thesis capacity in the agri-food public food sector to supportawideradoptionandapplicationofthesemethods in research and decision-making. For example, special-izedknowledgesynthesistrainingprogrammes,dedicated fundingopportunities,androutinesynthesisunitsin dif-ferent agencies could all contribute to improving the applicationanduptake ofsynthesizedknowledge.There is also a need for a consolidated infrastructure to sup-portknowledgesynthesisresearchinthissector,similarto theCochraneCollaborationinhealthcare,toimprovethe standardizationofconductandreporting,tosupportthe exchangeofknowledgesynthesisresourcesandexpertise, andtomaintainaformalrepositoryofsystematicreviews, meta-analysesandothersynthesisprojects.Finally,future

(15)

research is necessary to evaluatethe effectiveness and utility of different approaches and formats to summa-rizeknowledge synthesisﬁndings for variousend-users toimprovetheiruptake inpolicyand practice decision-making.

5. Conclusion

Knowledgesynthesisusesrobust,systematic,and trans-parent methods to identify, evaluate, and integrate all available knowledge about a topic. Knowledge synthe-sisresultscanbeusedtoidentifyknowledgestrengths, gaps, and opportunitiesand toinform decision-making forpolicyandpractice.Scopingreviews,structuredrapid reviews, mixed-method and qualitative reviews, sys-tematic reviews, and meta-analysis can all be used to synthesizecomprehensiveknowledgeinvarioussituations andcontexts.Additionalresearchisneededtoevaluateand applyspeciﬁcadvancementsandtoovercomevarious chal-lengesrelatedtotheapplicationofthesemethodsinthe agri-foodpublichealthsector.Awideradoptionand inte-grationofknowledgesynthesismethodsinthissectoris importanttoensurethatthebestavailableknowledgeis usedtosupportfuturedecision-making.

Conﬂictofintereststatement

Nonetodeclare. Acknowledgements

We gratefullyacknowledgealloftheindividuals and groups that have contributed to the development and applicationofknowledgesynthesismethodsinthe agri-foodpublichealthsector.Thisreviewwasfundedbythe UniversityofGuelphandOntarioMinistryofAgriculture andFood’sAgri-FoodandRuralLink“Knowledge Transla-tionandTransferFundingProgram”andtheLaboratoryfor FoodborneZoonoses,PublicHealthAgencyofCanada. References

Anderson,S.,Allen,P.,Peckham,S.,Goodwin,N.,2008.Askingtheright questions:scopingstudiesinthecommissioningofresearchonthe organisationanddeliveryofhealthservices.HealthRes.PolicySyst. 6,7.

Arksey,H.,O’Malley,L.,2005.Scopingstudies:towardsamethodological framework.Int.J.Soc.Res.Methodol.8,19–32.

Atkins,S.,Lewin,S.,Smith,H.,Engel,M.,Fretheim,A.,Volmink,J.,2008.

Conducting ameta-ethnography ofqualitative literature:lessons learnt.BMCMed.Res.Methodol.8,21.

Bennett,G.,Jessani,N.(Eds.),2011.TheKnowledgeTranslationToolkit. SAGEPublicationsIndiaPvt.Ltd.,NewDelhi,India.

Borenstein,M.,Hedges,L.V.,Higgins,J.P.T.,Rothstein,H.R.,2009. Introduc-tiontoMeta-Analysis.JohnWiley&Sons,Ltd.,Chichester,UK.

Brozek,J.,Oxman,A.,Schünemann,H.,2008.GRADEpro.Version3.2.

http://ims.cochrane.org/revman/gradepro/

Bucher,O.,Farrar,A.M.,Totton,S.C.,Wilkins,W.,Waddell,L.A.,Wilhelm, B.J.,McEwen,S.A.,Fazil,A.,Raji ´c,A.,2012a.Asystematic review-meta-analysisofchillinginterventionsandameta-regressionofvarious processinginterventionsforSalmonellacontaminationofchicken. Prev.Vet.Med.103,1–15.

Bucher,O.,Fazil,A.,Raji ´c,A.,Farrar,A.,Wills,R.,McEwen,S.A.,2012b. Eval-uatinginterventionsagainstSalmonellainbroilerchickens:applying synthesisresearchinsupportofquantitativeexposureassessment. Epidemiol.Infect.140,946–950.

Buscemi,N.,Hartling,L.,Vandermeer,B.,Tjosvold,L.,Klassen,T.P.,2006.

Singledataextractiongeneratedmoreerrorsthandoubledata extrac-tioninsystematicreviews.J.Clin.Epidemiol.59,697–703.

CodexAlimentariusCommission,1999.Principlesandguidelinesforthe conductofmicrobialriskassessment,www.codexalimentarius.net/ download/standards/357/CXG030e.pdf

Deeks, J.J., 2002. Issuesin the selection ofa summary statistic for meta-analysisofclinicaltrialswithbinaryoutcomes.Stat.Med.21, 1575–1600.

DerSimonian, R., Kacker,R., 2007. Random-effects mod