SMGL and comparison groups with the pre- to post-test changes shown for outcomes
such as service use intent and service use behaviors. Within each experimental group, I
compared the proportions for the pre and post time periods using the Chi-squared test.
I used a similar analysis strategy to examine supply-side indicators from the HFA
focusing on signal functions and other facility-level indicators of maternity care provided.
Within each experimental group I compared the proportions for the pre and post time
periods using the Chi-squared test. To test changes in the mean signal function, I made
similar comparisons using the t-test.
Qualitative Evaluation
2.1.7 Purpose
My purpose in conducting qualitative key informant interviews (KII) with
stakeholders was to generate more detailed data on factors affecting SMGL program
implementation in Kalomo District. This allowed me to contextualize the quantitative
findings and to identify factors that may facilitate or prevent program sustainability,
replication, and scale-up. The KIIs focused on the respondents’ experiences as policy or
program-level decision-makers, program implementers, or participants in the delivery of
2.1.8 Sample Selection and Recruitment of Informants
In Table 7, I list the type and number of participants recruited for the qualitative
interviews. For each international agency and provincial and district-level MOH office, I
sought to interview the main point-of-contact for the SMGL program (one or two persons
at each). For the facility-level interviews, I sampled eight facilities, including seven rural
health centers and one hospital (approximately 23% of the 35 total for the district) to
interview both senior clinicians (nurses, clinical officers, or other health providers) and
one of the SMAG members. For facility-level informants at the seven rural health
centers (not hospitals), I selected two types of facilities: three “high-functioning”
facilities and four “low-functioning” facilities, which were classified on the basis of the
pre-SMGL HFA that assessed facility capacity to handle obstetric emergencies. I
interviewed available clinicians and SMAGs, provided that the individuals met the
criteria outlined above.
I identified potential informants by creating a comprehensive list of candidates,
including representatives from the implementing partners in Kalomo, donor
representatives, provincial-level MOH representatives, district-level health staff, SMGL
project staff, facility-level clinicians, and community health workers in the SMAGs. I
then worked with ZCHARD/BU leadership to clarify the roles and responsibilities of
during the time I proposed to conduct interviews. The only informant I was unable to
interview was a provincial-level government official from Southern Province, due to time
limitations.
Table 7. Key Informants for SMGL Evaluation
2.1.9 Instrumentation and Data Collection
I conducted in-person interviews using a semi-structured interview guide. (See
Appendix D). The interviews with all informants except for two SMAG members were
conducted and transcribed in English; I worked with translators to interview the two
SMAG members in their native language, Tonga. Each interview took place in a private
location convenient to the participant, such as a private room at the facility or outside
away from other individuals. The interviews lasted between 35–60 minutes.
Approval for the interview procedure was granted from the Internal Review
Board (IRB) at both Boston University and ERES Converge in Zambia. I obtained
Type of Key Informant Number
A Kalomo District Community Health Workers (CHW) from SMAG
5
B Kalomo Health Facility Lead Clinicians (Doctor, Clinical Officer or Nurse/Midwife)
8 (2 hospital and 6 rural health center)
C Kalomo District Health Officers 2
D SMGL Project Director, Kalomo 1
E SMGL Senior Clinical Mentor (1) and Clinical Mentors (3) 4
F MOH and MCDMCH Representatives 2
G USAID/CDC Representatives 3
H Representative from Zambia Integrated Systems Strengthening Project (ZISSP) another Kalomo SMGL implementing partner
1
written informed consent from all participants prior to conducting the interviews.
2.1.10 Data Analysis
The purpose of the qualitative analysis was to: 1) create an explanatory account
for the quantitative analysis by examining context, program implementation factors, and
observed outcomes, and 2) develop a set of recommendations regarding future scale-up
and implementation. I conducted all qualitative data analysis using Nvivo10.(QSR
International Pty Ltd., 2012)
I followed the “framework” approach for the analysis of qualitative data in
applied research, which uses the following steps: 1) familiarization; 2) identifying a
thematic framework; 3) indexing; 4) charting; and 5) mapping and interpretation (Ritchie
& Spencer, 1993). I personally transcribed all interviews, and then reviewed the
transcripts to identify a general set of initial themes and topics, taking notes and
identifying variables to explore as additional indicators in the quantitative analysis.
Next I began constructing a codebook, using questions from the interview guide
and components from the logic model and the Three Delays Model. I preliminarily coded
one transcript from each of the five types of respondents: SMAG member, health facility
provider, SMGL program implementer, government official, and lead implementing
until it reflected all of the themes and subthemes present.
Next, I sorted the data by themes and synthesized the findings. I analyzed both
for common themes as well as for atypical responses that were unique to particular types
of individuals. I conducted queries using outcome frequencies and then framework
matrices to compare topics by type of respondent. I revisited the quantitative data to
identify additional questions to exposure using the informants’ responses. I used these
3 CHAPTER THREE: RESULTS