Data Screening and Preliminary Analysis

5.4.1. Overview

The extent to which the data meets psychometric assumptions was addressed before the appropriate data analysis techniques were employed, since some of the criteria (i.e., data distribution, and sample size) have direct bearing on the choice of analysis

techniques and tests.

5.4.2. Missing Data

Missing data is a concern of most researchers and can affect empirical research (Malhotra, 1999). Sixty-nine returned supplier surveys (41.6% of supplier surveys), and 55 (40.1%) of the customer surveys returned had missing data. Respondents who returned surveys with missing data were contacted by telephone and offered the opportunity to complete the survey over the telephone or via email. In all cases the participant either had not understood the original question requirement and needed clarification, or had simply overlooked that portion of the survey. The most common item of missing data in the supplier surveys was the first item of Section A on the second page asking the extent of CRM use. Thirty-seven respondents overlooked this item on the survey, accounting for 53.6% of the number of surveys with missing data. It was concluded that respondents simply overlooked the ‘extent of CRM use’ item due to the physical position of this item on the page. For the balance of the surveys with missing data the missing data was evenly distributed across 41 items.

In the customer survey, the very first item in Section A was most often omitted; 46 respondents (83.6% of the surveys missing data) failed to answer this item. This item referred to the strength of the relationship with the nominating firm. It was concluded that due to the declarative statement itself and the relative position of the item on the page the respondents simply overlooked this item. In all situations the requirements were appropriately clarified and the surveys completed over the phone (or email). In five cases of missed data, specific sections of the supplier survey were emailed to the

respondent and the surveys were completed electronically and returned. This follow-up process resulted in no missing data for the 140 matched surveys.

5.4.2.1. Non-eligible Respondents

Since the objective of the study was to investigate the effect of CRM technology adoption on customer relationships, response from suppliers who indicated they did not currently have or use CRM technology (or a CRM system) were discarded from further analysis. This resulted in the elimination of 25 dyads, leaving 115 dyads.

5.4.2.2. “Do not know” Response – IT Management Orientation section

As discussed in Chapter 4 the “Do not know” (DNK) option was included only in the IT management orientation (ITMO) section of 20 items. DNKs were anticipated in the ITMO section since it was considered that not all of the marketing and sales respondents would have the appropriate level of IT operational knowledge required to answer this series of questions (Durand & Lambert, 1988). Although the use of DNKs creates additional problems for data analysis (Poe, Seeman, McLaughlin, Mehl, & Dietz, 1988), DNKs are considered acceptable responses and should not be ignored (Leigh & Martin, 1987). The number of respondents choosing at least one DNK response in the ITMO section was 16 (10.7%).

In order to understand the effect of DNK responses in the ITMO section of the survey, the pattern of DNKs was analysed using SPSS Missing Value Analysis (MVA) and then the DNK’s were treated as missing data. Based on MVA there was no overall discernable pattern to the DNK responses. DNK respondents were compared, using independent t-tests, to the balance of the supplier respondents with respect to the two sub-constructs of the CRM technology adoption (CTA) variable (CKN and USF scales). Table 5.5 shows no significant difference between the two group’s responses to the CTA items. Refer to Appendix A8 for details of the DNK analysis.

Table 5.5: Comparing CTA Responses Between “Do not know” and General Respondent ITMO Responses (n = 115)

t-test for Equality of Means

Construct t-value df Sig. (2-tailed)

CKN 1.612 113 0.110 USF -0.318 113 0.751

Since the overall number of DNKs was relatively small, and restricted to the ITMO section only, in order to preserve the data properties to allow full analysis of the data, the DNK responses were treated as missing values and were substituted using series means. Two cases were deleted from subsequent analysis since they exhibited extremely high DNK responses in the ITMO section at 70% and 100% respectively. This left 113 dyads as the final number of cases for analysis.

5.4.3. Assumptions Underlying Statistical Procedures

5.4.3.1. Normality of the Data

Normality of the data was assessed using the Kolmogorov-Smirnov (KS) test, visual examination of normal probability plots, and by computing skewness and kurtosis measures (Carver & Nash, 2005). Skewness ratings of ± 1 and kurtosis scores of ± 2 are considered mild and fall within the ‘normal’ range, while scores outside of this range have the potential to restrict the data analysis and subsequent interpretation of results (Heck, 1998; Kline, 2005). Considering the relationship strength (RS) and relationship performance (RP) questions related to relationships with a current supplier, it was anticipated that customers would tend to respond more positively than negatively toward their supplier, resulting in a skewed data distribution on these specific items. As expected, the data distribution from a number of customer items relating to relationship strength and performance was highly skewed with corresponding high kurtosis. (The survey criteria did specify that supplier firms nominate their third or fourth most

important customers in an attempt to reduce or neutralise this effect.) Six items from the supplier data regarding CRM technology adoption (CTA) also exhibited similar

distribution concerns (see Appendix A9 for survey data distribution details).

When confronted with non-normal data distributions for factor analysis, attempts should be made to normalise the data in order to conduct appropriate and valid data analysis (Rummel, 1970). However, the result of data transformations, although facilitating data analysis, can restrict and alter subsequent interpretation of the results and therefore should not be conducted unnecessarily (Hair et al., 2006; Rummel, 1970). Although many multivariate analysis techniques are known to be robust with respect to data distributions, (i.e., some deviation from normality is acceptable) (Hau & Marsh, 2004; Muthén & Kaplan, 1985; S. Sharma, Durvasula, & Dillon, 1989; Yanagihara & Yuan, 2005), as a precaution the CTA principal components exploratory factor analysis

was conducted twice, once using untransformed data, and a second time using

transformed data. Both analysis approaches led to similar conclusions8, therefore only the untransformed data is reported in the exploratory factor analysis (EFA) results. Since PLS is not restricted or constrained by the distribution properties of the data, and for the sake of consistency with the factor analysis results, only the untransformed data was used to conduct the PLS analysis (Chin, 1995; Wold, 1985).

5.4.3.2. Sample Size and Power

Since little guidance is available from the SEM and PLS literature regarding

statistical power, factor analysis criteria were adopted (Chin, 1998a). Hair et al. (2006) provide guidelines for identifying significant factor loadings for factor analysis

dependent on sample size. Given a sample size of 113 cases, a factor loading of 0.55 or greater is considered significant, assuming a 0.05 significance level for a Type I error (α), and a power level of 80 percent (Cohen, 1992). (A Type I error is the probability of accepting a “false positive” as true.) For this study the more conservative 0.60 level was used as the minimum criterion for assessing factor loadings.

5.4.4. Common Method Variance

Common-method bias is recognised as a major source of measurement error and can have a substantial impact on observed relationships between the measured variables (Bagozzi & Yi, 1991; Nunnally & Bernstein, 1994). Although there are a number of sources of common method bias, including context and item characteristics, the use of the same respondent for independent and dependent measures is a common source and has been shown to produce significant artificial covariance (Podasakoff, MacKenzie, Lee, & Podsakoff, 2003). Podasakoff et al. suggest a set of procedures to control for common method bias and recommend in the first instance that “predictor and criterion variables [can] be measured from different sources….Additional statistical remedies could be used but in our view are probably unnecessary in these instances” (p. 897). The use of supplier respondents for the dependent variables and customer respondents for the dependent variables helped reduce common method bias (Reinartz et al., 2004).

Another potential source of common method bias, related to the general measurement context specific to this dyadic research, is the existing relationship between supplier and customer. There was the possibility that suppliers colluded with

customers in order to present their relationship in a more favourable light. Survey research assumes random measurement error across informant questionnaire responses (Nunnally, 1959; Viswanathan, 2005). In order to test for this possibility a correlation matrix was generated comparing supplier RS and customer RS responses, and again for supplier RP and customer RP responses, see Table 5.6 (J. C. Anderson & Narus, 1990). There was an expectation that the perception of the relationship would be similar

between the two parties, but should not be in complete agreement. The a priori decision was that correlations between 0.00 and 0.40 would be an acceptable range of

correlations, correlations beyond 0.70 would be suspect, while correlations between 0.40 and 0.70 would be subject to closer scrutiny. The results indicate no strong correlations between the relationship variables across the two respondent groups and therefore no apparent collusion on behalf of the two parties was evident.