Data analysis and power analysis 1 Aim

Analyses were conducted using Stata 10 (Stata Corp., College Station, TX).

Missing data management. One percent of observations were missing data for the top

managers’ support scale; 22 percent for the access to financial resources scale; and 37 percent for the access to human resources scale. Little’s (1998) test indicates that data were missing at random. I imputed missing data using multiple imputation (Stata 10’s ice

command), a consistent and efficient method of handling missing data (Schafer & Graham, 2002). Multiple imputation produces standard errors that reflect the uncertainty associated with imputed values. Multiple imputation involved creating 20 data sets, each with different imputed values. Missing values for each variable were imputed based on a set of predictor variables. Analysis was then conducted with each of the 20 imputed data sets. These analyses were then combined to construct a single set of results.

71

Consideration of common method variance. To minimize the bias associated with

constructing both dependent and independent variables using middle managers’ responses to survey items, I accounted for common method variance. The common method variance model asserts that “the observed variables are contaminated by a single unmeasured factor that has an equal effect on all of them” (Lindell & Whitney, 2001). In this case, the

unmeasured factor is the variance associated with responding middle managers. To account for this variance, in each ordinary least squares regression model, I included a survey item that was not theoretically related to implementation effectiveness or middle managers’ commitment. These marker variables were self-reported attitude items that had the smallest possible positive correlations with dependent variables after artifactual negative correlations were removed: (1) An item relating to middle managers’ attitudes about boards of directors’ level of involvement in the HDC was used as a marker variable for the following models: (a) community linkages implementation effectiveness; (b) self-management implementation effectiveness; and (3) middle managers’ commitment. (2) An item relating to middle managers’ attitudes about the extent to which HDC learning sessions helped to motivate HDC teams was used as a marker for delivery system design implementation effectiveness. The items were imperfect marker variables in that, theoretically, they may have been related to the dependent variables; however, I chose these items based on three criteria: (1) Of all the survey items, they were the least theoretically related to the dependent variables; (2) they had the smallest possible positive correlations with dependent variables after artifactual negative correlations were removed; and (3) they had low correlations with the other predictor

72

the survey: it was short enough to avid transient mood states such as boredom and fatigue (Lindell & Whitney, 2001). For all analyses, middle managers were the unit of analysis.

Analysis

The study sample contained 120 observations. The small size of the sample limited the statistical power of the analysis and increased the possibility of Type I error (i.e., failure to support the hypothesis when the hypothesis is true). As an alternative, bivariate analyses were used to determine which independent variables to include in multivariate analyses (p<.1). I chose this approach to achieve adequate statistical power and to ensure stability in coefficients and standard errors. For bivariate and multivariate analyses, p-values of less than .1 were considered significant.

In each multivariate OLS regression model, power to detect statistically significant relationships was sufficient (>.8). Shapiro-Francia tests, skewness and kurtosis tests, and visual inspection of the distribution of multivariate OLS regression residuals indicated approximate normality. I used Stata’s “robust” option to adjust for heteroskedasticity. Correlations and variance inflation factors were examined to assess multicollinearity (see table 5.3).

Main effects. The hypotheses that I tested are as follows:

Hypothesis 1: Middle managers’ commitment to innovation implementation is positively related to implementation effectiveness.

Hypothesis 2a: Incentives are positively related to middle managers’ commitment to innovation implementation.

73

Hypothesis 3a: Performance reviews are positively related to middle managers’ commitment to innovation implementation.

Hypothesis 4: Access to financial resources is positively related to middle managers’ commitment to innovation implementation.

Hypothesis 5: Access to human resources is positively related to middle managers’ commitment to innovation implementation.

Hypothesis 6: Access to training resources is positively related to middle managers’ commitment to innovation implementation.

Hypothesis 7: Local social network involvement is positively related to middle managers’ commitment to innovation implementation.

Hypothesis 8: Top managers’ support for innovation implementation is positively related to middle managers’ commitment to innovation implementation.

The statistical significance of the partial effects of the independent variables on the dependent variables in multivariate OLS regression analyses were used to test hypotheses 1, 2a, 3a, and 4-8.

Interaction effects. The effects of incentives and performance reviews on middle

managers’ commitment to innovation implementation are likely to depend on other IP&Ps, which are necessary to commit to innovation implementation. The hypotheses that I tested are as follows:

Hypothesis 2b: The relationship between incentives and middle managers’ commitment to innovation implementation depends upon performance reviews; access to financial, human, and training resources; local social network involvement; and top managers’

74

support, such that the relationship will be weaker when any of these other IP&Ps are available.

Hypothesis 3b: The relationship between performance reviews and middle managers’ commitment to innovation implementation depends upon incentives; access to financial, human, and training resources; local social network involvement; and top managers’ support, such that the relationship will be weaker when any of these other IP&Ps are available.

The statistical significance of the partial effects of interactions between incentives and other IP&Ps and between performance reviews and other IP&Ps in multivariate OLS regression analyses were used to test hypotheses 2b and 3b.

4.5.2. Aim 2

All interviews were transcribed verbatim, resulting in more than 300 pages of text. I employed template analysis, which combines content analysis methods with grounded theory, to identify some themes a priori from interview questions and to allow additional themes to emerge as analysis proceeds (N. King, 1998). A multifunctional qualitative data analysis software program (Atlas.ti 5.0) was used to code interview data and to identify emergent themes associated with moderators and mediators of the relationship between IP&Ps and middle managers’ commitment to innovation implementation in health care organizations. Using software to record coding steps and document coding decisions enhanced the reliability of study findings.

The text units were coded using a coding manual with definitions, decision rules, and examples to ensure consistency of data analysis and increase internal validity. The theoretical

75

framework for middle managers’ role in innovation implementation (figure 3) provided a starting list of codes, which were supplemented with emergent codes as the analysis proceeds. Using the coding manual addressed potential instrumentation bias (Cook & Campbell, 1979).

I began by coding a sample of three transcripts. Based on the results, I sharpened the coding manual’s code definitions, decision rules, and examples. I then coded the remaining documents, developed additional codes, and generated new propositions. Using ATLAS.ti, reports of all text segments for each code were generated. To enhance coding validity, I sought the perspective of a colleague with expertise in qualitative methods in health services research (Leah Masselink PhD, National Research Service Award Postdoctoral Fellow at the University of North Carolina at Chapel Hill’s School of Nursing) on my interpretation and application of the codes. Dr. Masselink and I independently coded three of my interview transcripts (a 20 percent sample). We then compared our coding and reconciled

disagreements until we reached consensus. In most cases, the reason for the discrepancy was obvious (e.g., misapplication of inclusion/exclusion criteria for a code, lapse of attention). We resolved remaining discrepancies by collaboratively refining code definitions. The degree to which the construct emerges in the data (its strength), and the degree to which the construct affects strategic responses (its valence) were examined to assess relationships between IP&Ps and middle managers’ commitment and mediators and

moderators of the relationships. Specifically, I conducted within- and across-unit analysis to assess construct strength and valence. First, I assessed the valence of themes within each middle manager’s interview text. For example, one middle manager discussed access to financial resources, access to human resources, and top managers’ support most frequently

76

and emphatically. The middle manager indicated, however, that top managers’ support was necessary for her to make use of access to financial and human resources; if top managers were unwilling to give the middle manager the authority to use financial resources to access useful implementation tools and to manage human resources in a way that would facilitate HDC implementation, the middle manager was unable to make use of financial and human resources. This within-unit analysis allowed me to assess the relative and combined effects of IP&Ps on her commitment to innovation implementation. Second, I assessed construct

strength through across-unit analysis. For example, I determined that interviews provided little support for the moderators proposed in the conceptual framework (figure 3) by comparing responses across middle managers’ interviews.

CHAPTER 5

QUANTITATIVE RESULTS