7.1 Overview
The previous three chapters summarized findings from different data collection efforts undertaken in this research to examine the implementation of P4HB®. These findings reflect multiple perspectives about how this program was conceptualized, planned, and implemented, with explicit details about whether the program was
implemented with fidelity and what barriers to successful program implementation still exist. By using a mixture of research methods in an evaluative case study design, we were able to examine whether implementation has occurred in compliance with it statutory goal, as well as the role that resources, and health system factors play into successful implementation. Finally, this approach enabled us to identify facilitators and problems with the implementation of P4HB®.
This current chapter presents results from a cross-method analysis of all data collection efforts (qualitative and quantitative). While each component was explored individually, a mixed methods analysis contributes to a deeper understanding of policy implementation as applied to a Medicaid family planning waiver program. In particular, greater insight is achieved by studying the similarities and differences observed across the various methods. Such observations can contribute to a clearer understanding of the different actors involved in policy implementation, the processes required to plan, design,
182
and implement a program, and the factors that guide or impede successful implementation.
This chapter begins with an overview of our approach to integrate the various qualitative and quantitative data. We describe this process first and then present a clear picture of how the data were used to compare and contrast our study findings. Next, we present a descriptive summary of the findings from each data collection method. Using the process evaluation typology described in Chapter 3, we examine the findings from each methodology by applying these specific measures. Next, we summaries the findings of these process evaluation measures across all three methodologies. Finally, we present the mixed methods results, organized by the research questions identified in Chapter 1.
As identified, we use our theoretical framework to identify these research questions. We align our theoretical framework with these research questions and use a matrix to display the intersection of our theory with the study’s research questions.
7.2 Technique for Integrating Mixed Methods Data
As described in Chapter 3, we employed concurrent analysis of our data, whereby each source (interviews, document review, focus groups, provider surveys) was analyzed separately and then merged for comparison across themes. This approach, also described as “triangulation,” allowed us to capture a more complete, holistic, and contextual portrayal of the implementation of P4HB®. Key to this process, however, is utilizing a strong and consistent technique to integrate the data. As described by Wolf (2010), the
“nexus between the qualitative and quantitative analyses needs to be carefully
established” (pg. 160). Indeed, we searched for this connection during the analysis phase
183
but we also searched for divergent findings as well. A common mistake in mixed methods research is to assume that the findings from each data source are simply aggregated and merged only to find common ground (Jick, 1979). We followed the technique described by Creswell and Clark (2007) to transform the quantitative data (provider surveys) into the qualitative data. The advantage of this approach is to allow for the data “to speak” on common ground but also to illuminate similarities and findings in the data (Heese-Biber and Leavy, 2006; Creswell and Clark, 2007).
To prepare for transformation, we first explored and analyzed each source of data and presented the results separately. This stage included conducting the thematic analysis of the qualitative data and the statistical analysis of the provider survey data. Next, we presented the results of each data source, highlighting the major findings and noting which themes emerged across each type of data. For instance, when we reviewed findings about provider outreach and education (from interviews and document reviews for
example), we noted what were common themes but also differences. Our interviews with providers revealed an almost complete lack of information sharing from Medicaid and the CMOs; yet, the document reviews indicated that provider outreach had occurred to some extent and throughout the first year of the program. So we noted that provider outreach and education was a major theme throughout our qualitative analysis but that findings about its importance and prominence in the first year of implementation were not conclusive.
Then we were able to move to our next step in the transformation process, which was to examine the provider survey data in a qualitative manner. By transforming our main findings from the statistical analysis of the provider survey, we could compare the
184
results more easily. Using the example of provider outreach and education, the provider survey data were transformed into themes or factors that were then compared with the themes developed from the other qualitative data. We noted that little provider education had occurred among our survey respondents and that family physicians were the least likely to have received information about P4HB®. Title X providers were the group most likely to receive any training on this program, a finding we would not have been privy to, if we had relied only on the interview and document review data. So the data
transformation was important for not only observing concordance with some thematic analysis but also to highlight new and unexpected findings as well.
A final stage in our data transformation process was to cross link the findings to our process measures and research questions. This provided the opportunity to map our findings to the theoretical basis of our research as well as a systematic approach to integrate the data. We wanted to make sure that our findings on process measurement and the research questions were addressed using all the data that had been analyzed and appropriately transformed. The next sections present the findings by data collection method, process measure, and research question.
These discussions are aided by matrices that illuminate the key findings of the integrated data. Then we conclude with a summary of the overall findings of this mixed methods approach.