4.6 SUMMARY EFFECT SIZE COMPUTATION

4.6.1. Averaging correlations within studies

Several incorporated studies reported multiple correlations for the relationship

under investigation (e.g., Akgün et al., 2009; Chen et al., 2009; Coombs &

Bierly, 2006). The correlations from such studies were combined into a single effect size before they were synthesised. Reporting of multiple correlations in a single study was generally a result of deploying diverse measures for either PIC or firm performance, or both. For example, Vorhies and Morgan (2005) reported separate correlations for two firm performance measures, namely, profitability and return on assets. Similarly, Ar and Baki (2011) reported separate correlations for product and process innovation that together constitute PIC (see O‘Cass & Ngo, 2012 for PIC definition). In such cases, the following arguments underpinned the averaging of multiple correlations reported in a single study.

It is a common practice in meta-analyses to synthesise effect sizes based on different IV or DV measures (e.g., Kirca et al., 2005; Kirca et al., 2011; Rosenbusch et al., 2011). This practice supports the argument for averaging multiple correlations that are based on different but conceptually similar measures that are reported in a single study. Lipsey and Wilson (2001: 101) highlight this convention and its appropriateness by asserting that ―the usual ways of handling multiple effect sizes […] are to either select a single effect size from amongst them or average them into a single mean value‖. Therefore, multiple effect sizes for the PIC–firm performance relationship reported in the incorporated studies were averaged to obtain a single effect size value, as in Ar and Baki (2011), Wolff and Pett (2006) and Yam et al. (2011). This also ensured that every study reporting multiple effect sizes was included only once to preclude their overrepresentation in the summary effect size. The imperative of preventing overrepresentation has been underscored in several

meta-analyses (e.g., Read et al., 2009; Rosenbusch et al., 2011;

Sivasubramaniam et al., 2012). It has also been clearly articulated by Rosenbusch et al. (2011: 448), who state that:

Page | 101

Where articles based on the same sample reported different effect sizes because they linked different innovation measures to different performance measures, we calculated average effect sizes and included each sample only once based on average effect sizes.

Conversely, separate publications investigating the same constructs and firm sample, but reporting multiple effect sizes for the relationship of interest, were included only once in the summary effect size computation, as in Li and Atuahene-Gima (2001; 2002). Inclusion of such studies only once was achieved through averaging the reported correlations. Consequently, the possibility of any firm sample being overrepresented through multiple inclusions in the dataset was eliminated. This averaging of multiple correlations reduced the dataset from the original 81, to 58 correlations. The 58 correlations were subsequently aggregated, as discussed in the next Section.

4.6.2. Aggregating correlations across studies

As the summary effect size is analogous to a weighted average, which is commonly used in descriptive statistics, and represents the systematic aggregation of the disattenuated effect sizes (i.e., the correlations that have

been corrected for artifacts) (Borenstein et al., 2009). Fisher‘s z-

transformation and Hunter and Schmidt are the most commonly used approaches for obtaining the summary effect size (see Borenstein et al., 2009). The Hunter and Schmidt approach advocates that summary effect size calculations should be directly performed on correlations. On the other hand, Fisher‘s z-transformation involves converting correlations into z-coefficients (Hedges & Olkin, 1985; Kirca et al., 2005). Importantly, the standard-error of a z-coefficient is exclusively contingent upon the sample size and is unaffected by the magnitude of the z-coefficient itself, making the z-transformation a potentially superior method (see Geyskens et al., 2009). Several meta-

analyses have employed Fisher‘s z-transformation method (e.g., see

Grinstein, 2008b; Kirca et al., 2005; Kirca et al., 2011; Rubera & Kirca, 2012). Hence, Fisher‘s z-transformation was preferred over the Hunter and Schmidt

Page | 102

approach for obtaining the summary effect size in the current study. The formula used for computing z-coefficients is:

z-coefficient = 0.5* ln {(1 + Correlation)/(1 – Correlation)}; with the standard error of z-coefficient = 1/(N–3)1/2 _{, N = sample size of the study, ln = natural log, and ‗–‘}

(en dash) denotes subtraction. Exponent ‗1/2‘ and * (Asterisk) represent square root and multiplication respectively. (Borenstein et al., 2009)

The z-coefficients were then weighted by an estimate of the inverse of their variance and subsequently averaged (see Hedges & Olkin, 1985). This weighting ensured that studies with large samples were conferred proportionately greater importance. Finally, the weighted average of the z- coefficients was transformed back into the original correlation metric for reporting as the summary effect size (see Hedges & Olkin, 1985). This step was performed via a suitable software program (discussed later in the Chapter).

Some studies reported effect sizes that fell outside the usual range (approximately, from 0.00 to 0.60) of effect sizes extracted. For example,

Penner-Hahn and Shaver (2005) reported a correlation coefficient of ‗–0.11‘,

and this study could be considered an outlier by many researchers. Outliers are ―studies whose effects differ very substantially from the others‖ (Borenstein et al., 2009: 368). While outlier values such as the correlation reported by Penner-Hahn and Shaver (2005) are generally substantive, they can also be consequent upon the presence of transcriptional and computational errors (Gulliksen, 1986). Hence, the method adopted for a sensitivity analysis of outliers is discussed next, in addition to other types of sensitivity analyses that were undertaken in the current meta-analysis.