Increasing Impact: Evaluation in Global
Child Health Education, Clinical
Practice, and Research
Sophia P. Gladding, PhD,aParminder S. Suchdev, MD, MPH,bSarah Kiguli, MBChB, MMED, MHPE,c
Elizabeth D. Lowenthal, MD, MSCE,dON BEHALF OF THE GLOBAL HEALTH TASK FORCE OF THE AMERICAN BOARD OF PEDIATRICS
Opportunities in global child health (GCH), the study and practice of improving child health globally, are expanding and increasingly formalized. In education, more postgraduate pediatric training programs offer specialized GCH training, preparing graduates to provide health care to children worldwide.1Training in clinical areas such as neonatal
resuscitation for health care workers in low- and middle-income countries (LMIC) are increasing,2as are short courses for pediatricians from high-income countries (HICs) planning to practice in LMIC.3In clinical practice, there are growing numbers of LMIC-HIC clinical partnerships, which provide clinical care and support mutual professional development.4There are also expanding opportunities for HIC clinicians to participate in short-term medical trips.5In research, there are increasing partnerships and networks supporting collaborative transnational GCH research.4 As GCH programs and partnerships expand and formalize, robust evaluation is needed to inform best practices and program
improvement, provide accountability to communities served through GCH initiatives, and influence decision-making related to GCH priorities and funding. Our aim in this article is to (1)
highlight current evaluation strengths; (2) identify evaluation needs and key evaluation questions (Table 1); and (3) suggest possible approaches to addressing evaluation needs in GCH education, clinical practice, research and partnerships.
EVALUATION IN GCH EDUCATION PROGRAMS
Current Evaluation Strengths
Evaluations of the impact of GCH residency training on participants’ knowledge, skills, and
attitudes12,13are a strength of GCH education. Evaluations of the impact of short skills-based courses on participants’clinical practices2are an additional strength. These evaluations are important in determining the effectiveness of GCH education programs in preparing participants for GCH clinical practice.
Evaluation Needs and Key Evaluation Questions
Evidence To Inform Best Practices
Because increasing numbers of residency programs offer GCH training,1evaluations are needed to inform the best practices guiding their development. Key
questions regarding the optimal duration of GCH electives, the role of preceptors, and the benefits of concentrating or dispersing
aDepartments of Medicine and Pediatrics, University of Minnesota, Minneapolis, Minnesota;bDepartment of Pediatrics and Emory Global Health Institute, Emory University, Atlanta, Georgia;cDepartment of Paediatrics and Child Health, School of Medicine, College of Health Sciences, Makerere University, Kampala, Uganda; anddDepartment of Pediatrics, Perelman School of Medicine, University of Pennsylvania and Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania
This article is one of a series of articles conceptualized and produced by the Global Health Task Force of the American Board of Pediatrics. The content is solely the responsibility of the authors and does not necessarily represent the official views of the American Board of Pediatrics or the American Board of Pediatrics Foundation.
Drs Gladding and Lowenthal developed the content outline for the manuscript, recruited the author team, and drafted the manuscript; Drs Suchdev and Kiguli contributed to the conception and design of the manuscript, provided content expertise, and critically reviewed the manuscript; and all authors approved thefinal manuscript as submitted and agree to be accountable for all aspects of the work.
DOI:https://doi.org/10.1542/peds.2018-3716
Accepted for publication Jun 26, 2019
Address correspondence to Sophia P. Gladding, PhD, Departments of Medicine and Pediatrics, University of Minnesota, 141 VCRC, 410 E River Parkway, Minneapolis, MN 55455. E-mail: [email protected]
PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).
Copyright © 2019 by the American Academy of Pediatrics
TABLE 1Evaluation Needs, Key Evaluation Questions, and Existing Resources To Guide Evaluation Development for GCH Education and Clinical and Research Programs
GCH Domain Evaluation Need Key Questions Existing Resources To Guide Evaluation Development
Education Evidence to inform programmatic best practices
Programmatic best practices Guidelines and best practices for GCH training:
GCH electives:
•Federation of Pediatric Organizations Global Health Working Group guidance for resident programs6
•What is the minimal, maximal, and optimal duration of GCH electives?6
•Ethics and Best Practices Guidelines for Training Experiences in Global Health7
•What are the needs and roles of accompanying faculty preceptors?6
•Global Health in Pediatric Education: An Implementation Guide for Program Directors (available online)
•What are the benefits and possible negative consequences of concentrating or dispersing residents across partner sites?6
GCH tracks:
•To what extent do educational components (eg, curricula, mentorship, electives, orientation, and debriefing) contribute to effectively preparing residents in GCH competencies and for GCH experiences?1
•What degree of participation by residents in the GCH track is needed to effectively prepare residents for GCH electives and in GCH competencies?
GCH curriculum for all residents6
•What approaches are effective in providing basic GCH training to all pediatric residents? Effectiveness of innovative approaches •To what extent are online curricula and practice
resources effective in preparing trainees for GCH experiences?
Previous evaluations:
•How can predeparture simulations be most effectively used?
•Use of simulation in predeparture preparation, Simulation Use for Global Away Rotations8
Impact of GCH education on increasing health equity
•Does involvement in GCH training improve the quantity and quality of care provided to underserved populations?
Previous evaluations:
•Dimensions of quality of care could include health outcomes, communication skills and/or cultural humility, patient experiences, and cost-conscious care.9
A study of global health elective outcomes of pediatric residency graduates10
Clinical practice Health outcomes •What programs and strategies result in improved health outcomes?
Best practices GCH clinical programs:
•Best practices for clinical partnerships4
•Best practices for preparation for GCH clinical practice5
•Best practices for short-term clinical programs11
Metrics for global health outcomes9 Research Extent to which research agenda
matches data about worldwide causes of pediatric morbidity and mortality
•What causes of pediatric morbidity and mortality are underrepresented in research?
Comparison of global burden of diseases studies with clinical trials registries and research funding
•Online clinical trials registries (eg, https://www.clinicaltrials.gov)
•Global burden of disease studies
•Online resources for global health
financing (eg, http://www.
healthdataorgfinancing global health reports)
Closing the gap between efficacy studies and successful implementation
•What are the best strategies for closing the gap between efficacy studies and successful program implementation in high-need contexts?
Online resources for best practices for implementation science:
•Addressed through problem- and site-specific implementation-science questions
•National Implementation Research Network Guide for Assessing Best Practices
residents across elective sites proposed by the Federation of Pediatric Organizations in 20086have not yet been answered in the literature. There are similar key questions for GCH tracks, such as how effective the educational components of tracks are in preparing trainees for GCH experiences1and what degree of participation by residents
effectively prepares them for GCH experiences.
Effectiveness of Innovative Approaches
GCH education programs are increasingly incorporating
innovations such as online curricula3,14and simulation.8 Evaluations of the effectiveness of these approaches in preparing participants for GCH experiences, particularly compared with traditional approaches, can inform program development.
Impact of GCH Education on Health Equity
Increasing health equity is a goal of GCH.15Evaluations addressing the question of whether GCH-trained practitioners remain involved in GCH and service to underserved populations could help determine if
GCH training contributes to achieving health equity.
Possible Approaches To Address Evaluation Needs
More robust evaluation will require careful planning for data collection needed to address key questions before program implementation. Additionally, evaluation in GCH education could be strengthened through more multiinstitutional evaluations. Collaborations with initiatives such as the Association of Pediatric Program Directors
Longitudinal Educational Assessment Research Network, which supports
TABLE 1 Continued
GCH Domain Evaluation Need Key Questions Existing Resources To Guide Evaluation Development
Education, clinical practice, and research
Evidence to inform partnership best practices
Partnership best practices4 Proposed framework for evaluation of GCH partnerships4
Maximizing mutual benefit and reciprocity:
•What approaches are effective in leading to mutual benefit in partnerships?
Bidirectionality:
•What are the short- and longer-term outcomes of bidirectional opportunities for partners, including both intended (eg, professional development) and unintended (eg, brain drain) outcomes? Equitable allocation of resources:
•What are the actual costs of hosting visiting trainees, providers, and/or researchers? Alignment with local priorities:
•What are the short- and longer-term outcomes of aligning partnership initiatives with local priorities?
•What strategies are effective in ensuring that partnership projects are aligned with local priorities?
Professional development and capacity building
•In what ways is capacity built and professional development of both partners supported through GCH partnerships?
•What strategies and opportunities support the professional development of both partners?
•To what degree and in what ways do health professionals mutually benefit from involvement in GCH partnerships?
Cultural and contextual awareness:
•What strategies are effective in preparing trainees, practitioners, and researchers for the cultural and clinical contexts in which they will be working?
•What are the outcomes of a lack of adequate cultural and contextual preparation of trainees, practitioners, and researchers?
multisite educational studies, could bring resources, data, and expertise together to support larger-scale evaluations.
EVALUATION IN GCH CLINICAL PRACTICE PROGRAMS
Current Evaluation Strengths
A current strength of evaluation in GCH clinical practice is the evaluation of how LMIC-HIC clinical
partnerships are formed, how clinical needs are identified, how care is delivered, and how professional development is supported.4These evaluations allow other programs to replicate successful approaches.
Evaluation Needs and Key Evaluation Questions
Our understanding of these programs’impact on the health of communities they serve is more limited. Previous literature has highlighted problems occurring when HIC-based practitioners provide care that cannot be sustained after their departure from resource-limited settings and when under-prepared practitioners provide care inconsistent with local guidelines, public health initiatives, or cultural norms.5It is, therefore, important that health outcomes be evaluated and documented for accountability to communities served. Yet, a recent review reported that 95% of articles describing short-term medical trips (from multiple specialties) lacked rigorous data collection including for health outcomes.11In addition to providing accountability, evaluations of health outcomes can address the key question of what programs and strategies lead to improved health.
Possible Approaches To Address Evaluation Needs
GCH clinical programs should plan for the collection of health outcomes data during the design phase then collect and share data with partners to
identify areas for improvement. High-quality care metrics developed for global health could be used to help guide data collection.9Although outcomes data may not be easy to collect, particularly for shorter-term programs, longer-term partnerships have demonstrated it is possible to collect needed data, including on intermediate outcomes, such as enrollments in treatments or services provided through partnerships,16and on patient outcomes.17
EVALUATION IN GCH RESEARCH
Current Evaluation Strengths
Efficacy studies evaluating the performance of an intervention under ideal and controlled
conditions18and effectiveness studies evaluating the performance of an intervention under real-world conditions18are strengths of GCH research. These evaluations provide important information to help inform public health initiatives.
Evaluation Needs and Key Evaluation Questions
Implementation of Interventions
There are often differences between the performance of interventions under controlled and real-world conditions.19Evaluating the implementation of interventions could help our understanding of this performance gap.
GCH Research Agenda
There is currently no unified GCH research agenda prioritizing the greatest needs in GCH research. Evaluating the research agenda itself could identify areas of high pediatric morbidity and mortality currently underrepresented in GCH research, such as prevention of preterm births.20The evaluation results could be used to inform and influence funders and policy makers to address the most pressing needs in GCH.
Possible Approaches To Address Evaluation Needs
Implementation of Interventions
Implementation research can help our understanding of the gap in performance between efficacy studies and real-world implementation. Implementation studies may evaluate the integration and adoption of interventions into routine health care practice and identify barriers to and enablers of the uptake of proven interventions, helping accelerate the implementation of established best practices.19
GCH Research Agenda
Previous evaluations addressing the GCH research agenda have examined the relationship between the pediatric global burden of disease and studies published in high-impact pediatric journals21and clinical trial registries.22A comprehensive evaluation synthesizing comparisons of the pediatric global burden of disease with published studies, clinical trials registries, and research funding could inform the creation of a unified research agenda.
EVALUATION OF GCH PARTNERSHIPS
Current Evaluation Strengths
Our understanding of partnerships is strengthened by recent evaluations examining host perceptions of visiting faculty and trainees,23,24reporting outcomes of bidirectional
exchanges,25and describing the implementation of partnerships.16,17
Evaluation Needs and Key Evaluation Questions
More comprehensive evaluations of partnerships could help determine if partnerships are aligned with partnership best practices.4
Evaluations should consider the key questions of to what extent and how partnerships are mutually beneficial, equitable, aligned with local
development, and promoting cultural and contextual awareness.4
Possible Approaches To Address Evaluation Needs
Frameworks for evaluating
partnerships have been developed.4 Partners should work collaboratively to design and implement evaluations and interpret and apply their
findings.4
CONCLUSIONS
The goal of GCH education, clinical practice, and research is to improve child health globally. Robust
evaluation can help support and strengthen GCH programs in their efforts.
ACKNOWLEDGMENTS
We thank Valerie Haig and Donna Crisp of the American Board of Pediatrics for administrative support, and Mina Halpern, Cynthia Howard, and Virginia Moyer for their critical review of and advice regarding a draft of this article. We also thank the members of the Global Health Task Force of the American Board of Pediatrics: Linda Arnold, MD;
Maneesh Batra, MD, MPH; Sabrina M. Butteris, MD;
Christopher A. Cunha, MD; Jonathan D. Klein, MD, MPH; David G. Nichols, MD, MBA; Cliff M. O’Callahan, MD, PhD; Michael B. Pitt, MD; Nicole E. St Clair, MD; and Andrew Steenhoff, MBBCh, DCH.
ABBREVIATIONS
GCH: global child health HIC: high-income country LMIC: low- and middle-income
countries
FINANCIAL DISCLOSURE:In the past 3 years, Dr Lowenthal was the principal investigator of a research grant from Gilead Sciences for HIV-related work, and she is currently the principal investigator of a research study funded by The Lucky Iron Fish for iron deficiency anemia research; the other authors have indicated they have nofinancial relationships relevant to this article to disclose.
FUNDING:Supported in part by the American Board of Pediatrics Foundation
POTENTIAL CONFLICT OF INTEREST:The authors have indicated they have no potential conflicts of interest to disclose.
REFERENCES
1. Butteris SM, Schubert CJ, Batra M, et al. Global health education in US pediatric residency programs.Pediatrics. 2015; 136(3):458–465
2. Budhathoki SS, Gurung R, Ewald U, Thapa J, Kc A. Does the Helping Babies Breathe Programme impact on neonatal resuscitation care practices? Results from systematic review and meta-analysis.Acta Paediatr. 2019; 108(5):806–813
3. Royal College of Paediatrics and Child Health. Global child health: our UK courses. Available at: https://www. rcpch.ac.uk/resources/global-child-health-our-uk-courses. Accessed on January 12, 2019
4. Steenhoff AP, Crouse HL, Lukolyo H, et al; GH Task Force of the American Board of Pediatrics. Partnerships for global child health.Pediatrics. 2017;140(4): e20163823
5. St Clair NE, Pitt MB, Bakeera-Kitaka S, et al; Global Health Task Force of the American Board of Pediatrics. Global health: preparation for working in resource-limited settings.Pediatrics. 2017;140(5):e20163783
6. Stanton B, Huang CC, Armstrong RS, et al. Global health training for pediatric residents.Pediatr Ann. 2008; 37(12):792–796
7. Crump JA, Sugarman J, Barry M, et al; Working Group on Ethics Guidelines for Global Health Training (WEIGHT). Ethics and best practice guidelines for training experiences in global health.
Am J Trop Med Hyg. 2010;83(6): 1178–1182
8. Butteris S, Gladding S, Eppich W, Hagen S, Pitt M; SUGAR Investigators. Simulation use for global away rotations (SUGAR): preparing residents for emotional challenges abroad–a multicenter study.Acad Pediatr. 2014; 14(5):533–541
9. Scott KW, Jha AK. Putting quality on the global health agenda.N Engl J Med. 2014;371(1):3–5
10. Russ CM, Tran T, Silverman M, Palfrey J. A study of global health elective outcomes: a pediatric residency experience.Glob Pediatr Health. 2017;4: 2333794X16683806
11. Sykes KJ. Short-term medical service trips: a systematic review of the
evidence.Am J Public Health. 2014; 104(7):e38–e48
12. Oettgen B, Harahsheh A, Suresh S, Kamat D. Evaluation of a global health training program for pediatric residents.Clin Pediatr (Phila). 2008; 47(8):784–790
13. Gladding S, Zink T, Howard C, et al. International electives at the university of Minnesota global pediatric residency program: opportunities for education in all accreditation council for graduate medical education competencies.Acad Pediatr. 2012;12(3):245–250
14. Canadian Pediatric Society. Global child health curriculum modules. Available at: https://www.cps.ca/en/curriculum. Accessed August 24, 2018
15. Koplan JP, Bond TC, Merson MH, et al; Consortium of Universities for Global Health Executive Board. Towards a common definition of global health.
Lancet. 2009;373(9679):1993–1995
17. Eckerle M, Crouse HL, Chiume M, et al. Building sustainable partnerships to strengthen pediatric capacity at a government hospital in Malawi.Front Public Health. 2017;5:183
18. Singal AG, Higgins PDR, Waljee AK. A primer on effectiveness and efficacy trials.Clin Transl Gastroentnerol. 2014; 5(1):e45
19. Theobald S, Brandes N, El-Saharty S, et al. Implementation research: new imperatives and opportunities in global health.Lancet. 2018;392(10160): 2214–2228
20. Yoshida S, Martines J, Lawn JE, et al. Setting research priorities to improve
global newborn health and prevent stillbirths by 2025.J Glob Health. 2016; 6(1):10508
21. Keating EM, Haq H, Rees CA, et al. Global disparities between pediatric
publications and disease burden from 2006 to 2015.Glob Pediatr Health. 2019; 6:2333794X19831298
22. Joseph PD, Caldwell PH, Barnes EH, Craig JC. Disease burden-research match? Registered trials in child health from low- and middle-income and high-income countries.J Paediatr Child Health. 2017;53(7):667–674
23. Russ CM, Ganapathi L, Marangu D, et al. Perspectives of host faculty and
trainees on international visiting faculty to paediatric academic departments in East Africa.BMJ Glob Health. 2016;1(3): e000097
24. Lukolyo H, Rees CA, Keating EM, et al. Perceptions and expectations of host country preceptors of short-term learners at four clinical sites in Sub-Saharan Africa.Acad Pediatr. 2016; 16(4):387–393
25. Pitt MB, Gladding SP, Majinge CR, Butteris SM. Making global health rotations a two-way street: a model for hosting international residents.Glob Pediatr Health. 2016;3:
DOI: 10.1542/peds.2018-3716 originally published online November 12, 2019;
2019;144;
Pediatrics
BOARD OF PEDIATRICS
ON BEHALF OF THE GLOBAL HEALTH TASK FORCE OF THE AMERICAN
Sophia P. Gladding, Parminder S. Suchdev, Sarah Kiguli, Elizabeth D. Lowenthal and
Practice, and Research
Increasing Impact: Evaluation in Global Child Health Education, Clinical
Services
Updated Information &
http://pediatrics.aappublications.org/content/144/6/e20183716 including high resolution figures, can be found at:
References
http://pediatrics.aappublications.org/content/144/6/e20183716#BIBL This article cites 23 articles, 5 of which you can access for free at:
Subspecialty Collections
alth_sub
http://www.aappublications.org/cgi/collection/international_child_he
International Child Health
following collection(s):
This article, along with others on similar topics, appears in the
Permissions & Licensing
http://www.aappublications.org/site/misc/Permissions.xhtml in its entirety can be found online at:
Information about reproducing this article in parts (figures, tables) or
Reprints
DOI: 10.1542/peds.2018-3716 originally published online November 12, 2019;
2019;144;
Pediatrics
BOARD OF PEDIATRICS
ON BEHALF OF THE GLOBAL HEALTH TASK FORCE OF THE AMERICAN
Sophia P. Gladding, Parminder S. Suchdev, Sarah Kiguli, Elizabeth D. Lowenthal and
Practice, and Research
Increasing Impact: Evaluation in Global Child Health Education, Clinical
http://pediatrics.aappublications.org/content/144/6/e20183716
located on the World Wide Web at:
The online version of this article, along with updated information and services, is
by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.
