CORRECTING THE EFFECT SIZES FOR ARTIFACTS

Artifacts generally produce attenuation in study results, which means that artifacts cause effect size values to become smaller in value (Borenstein et al., 2009). Therefore, the artifact correction framework prescribed by Hunter and Schmidt (1990) was employed for corrections to estimate the disattenuated (i.e., artifacts-corrected) correlations. The disattenuated correlations were greater in magnitude than the reported (attenuated) correlations, as is

Page | 97

characteristically the case in all meta-analyses. The elimination of artifacts enabled the computation of a summary estimate of the PIC–firm performance relationship that is largely free from sampling and reliability (of measurement) errors (see Aguinis & Pierce, 1998).

Additionally, a more accurate assessment of the moderation effects is possible after the corrections, because a large proportion of the observed variance in the effect size dataset potentially stems from the presence of artifacts (Hunter & Schmidt, 1990; 2004). Thus, corrections were made to eliminate the sources of spurious variation before attempting to estimate moderation effects as prescribed by Hunter and Schmidt (1990; 2004). As meta-analyses generally account for sampling and measurement errors (e.g., see Grinstein, 2008a; 2008b; Kirca et al., 2011; Leuschner et al., 2013; Rubera & Kirca, 2012; Sivasubramaniam et al., 2012), these two artifacts are now discussed in the context of the current study.

4.5.1. Correction for sampling error

Sampling error is present in all primary studies and it indicates the extent to which the firm samples studied do not accurately represent the populations from which they are drawn (Särndal, Swensson & Wretman, 1992). Sampling error causes deviations in study findings from what would be the case if no sampling error was present, and its influence on correlations is essentially unsystematic (Hunter & Schmidt, 1990). Due to the unsystematic effect of the sampling error, no corrections are possible in individual correlations (Hunter & Schmidt, 2004).

Thus, no specific corrective computations could be performed in this study to eliminate or minimise sampling error. The magnitude of the unsystematic effect of sampling error is chiefly determined by the size of the overall firm sample in a meta-analysis (Hunter & Schmidt, 2004). Consequently, it is expected that the present study is not entirely free of sampling error. However,

due to an appreciable cumulative sample size (N) of 13,911 firms, sampling

error is not expected to distort the meta-analytic results considerably. The

Page | 98

sample sizes in many other meta-analyses (e.g., see Büschgens et al., 2013;

Read et al., 2009).

By contrast, the correction for measurement error can be undertaken on individual effect sizes if reliability estimates are reported in incorporated studies. Most meta-analyses employ corrections for measurement error and such corrections were undertaken in the current study, as outlined next.

4.5.2. Correction for measurement error

The correction for measurement error was undertaken by adopting the guidelines prescribed by Hunter and Schmidt (1990). This error was corrected by factoring in reliability estimates reported in studies. Reliability estimates are squares of corresponding factor loadings (Grawe et al., 2009). Unfortunately, several incorporated studies did not report the reliability estimates for their measures. This problem is commonly encountered and the general practice is to either compute a simple average (e.g., see Kirca et al., 2005; Geyskens et al., 1998; Stam et al., 2014), or sample size-weighted average of the reported reliability estimates (e.g., see Kellermanns et al., 2011; Sivasubramanian et al., 2012). Either of the two averages is assigned to the studies not reporting this data.

While both approaches have scholarly acceptance as outlined above, the latter (weighted average) was used in this study. The premise underlying the preference for a weighted average was that studies with large samples are likely to report more accurate reliability estimates. Thus, a weighted average factors in the relative precision of individual studies, as indicated by their respective sample sizes. The values were computed using a generic formula for weighted averages as shown below (and contextualised for the current study):

Weighted average of reported reliability estimates=

∑(Sample size of study reporting reliability X corresponding reliability estimate) / ∑(Sample sizes of studies reporting reliability estimates) -Formula-4.3 Where, X and ∑ represent multiplication and summation respectively.

Page | 99

Incorporated studies with unreported reliability estimates were assigned reliability values of 0.834 for PIC and 0.908 for firm performance (e.g., Penner-Hahn & Shaver, 2005; Thornhill, 2006; Lööf & Heshmati, 2006). The rationale for assigning the average of reliability estimates to missing (unreported) values was that the overall correction is superior with, rather than without, the assignment of average reliabilities to studies missing this information.

All effect sizes were individually disattenuated by dividing them by the product of the square root of the IV and DV reliabilities (see Hunter & Schmidt, 1990; 2004; Schmidt & Hunter, 1996). Hence, in accordance with scholarly recommendations, the formula used for disattenuation of PIC–firm performance effect sizes was:

rcorrected = rreported / (RPIC x RFirm performance)1/2 -Formula-4.4

where, ‗RPIC‘ and ‗RFirm performance‘ denote reliabilities for the IV and DV respectively;

‗rreported‘ is the attenuated correlation reported in the study and rcorrected is the

disattenuated correlation corrected for measurement errors. Exponent ‗½‘ denotes the square root of the denominator and ‗x‘ denotes multiplication.

(Hunter & Schmidt, 2004)

The formula-4.4 is essentially identical to Formula-4.1 (presented in Section- 4.3), but used here to make corrections for measurement errors (it was earlier used for adjusting correlations in compliance with the weighting scheme). Incorporated studies using archival data, such as Artz et al. (2010) and Schoenecker and Swanson (2002), were accorded reliabilities of one, as the data used for analysis in such studies was objective and not subjective (e.g., see Read et al., 2009). The effect sizes were synthesised subsequent to undertaking the adjustments based on the weighting scheme, and corrections for measurement errors. The procedure used for obtaining the summary effect size is now outlined.

Page | 100