Reliability

Reliability reflects the precision of the survey instrument, i.e. how reproducible the data of the survey instrument is628. In mathematical terms, reliability is the ratio of true score variance (i.e. non-random variance) to the observed variance629. To make a study more reliable, multi-item measurement constructs are used. Internal consistency reliability can be calculated for the group of measurement items that measure different aspects of the same phenomenon. Internal consistency can be expressed in the form of Cronbach’s coefficient alpha, which measures reliability among a group of items combined to form a single scale. Cronbach’s alpha reflects how well different items complement each other in measuring different aspects of the same concept.630 The value of Cronbach’s alpha is determined by the average correlation of each item with 624 Bohrnstedt 1983 p. 70 625 Litwin 1985 626 Bohrnstedt 1983 627 Bohrnstedt 1983 628 Litwin 1985 629 Bohrnstedt 1983 630 Litwin 1985

every other item in the group and the number of items used in the measurement construct631. Cronbach’s alpha level .70 is the generally accepted threshold value for good reliability632. However, lower reliabilities such as .60 may be appropriate in some research studies633. Low reliability is problematic since it tends to attenuate correlations between investigated measurement constructs, and it leads to underestimating the relationships between constructs634.

Table 12 summarizes the results of the factor analysis including Cronbach’s alpha values for each measurement construct used in this study. Ten multi-item measurement constructs including 8 independent variables and 2 dependent variables are used in this study. In addition, one independent variable is measured by using only a single item. All the Cronbach’s alphas are above the general threshold value .70 expect one variable (product concept superiority) which has the alpha value .69. The highest alpha values were reported in the informal communication (.91) and technology uncertainty (.84) measurement constructs.

Table 12. Internal consistency coefficients of measurement constructs.

Measurement construct Number of items Cronbach's α Number of cases Input control 4 .79 132

Front-end process formalization 4 .79 126

Outcome-based rewarding 3 .76 119

Strategic vision 1

Informal communication 3 .91 131

Participative planning 3 .73 127

Intrinsic task motivation 3 .74 130

Market uncertainty 4 .76 128

Technology uncertainty 4 .84 130

Product concept superiority 5 .69 129

Strategic renewal 4 .76 131 631 Nunnally 1978, Dooley 1980 632 Litwin 1985, Nunnally 1978 633

Hair et al. 1998, Dooley 1980 634

Clarity of the survey and understandability of questions were confirmed in several ways. This was done to increase the reliability of the study635. Existing items and variables were used whenever possible. The questionnaire was also thoroughly tested with practitioners and academics as discussed earlier. These tests led to minor modifications in the questionnaire to make it more understandable. The final feedback indicated that both instructions and questions were clear and that respondents knew how to answer the questionnaire. The clarity and understandability of the questionnaire is indicated by the small amount of missing data. The used measures pointed to only 1.56% of missing data. The influence of this missing data was found to be insignificant. The items of independent variables concerned the intensity of the use of certain control mechanisms, which was considered to give a more concrete evaluation point compared to simply asking for an opinion on a certain statement. In addition, the exact objective figures were used in control variables whenever possible.

This survey study relies on the judgment of a single respondent, which increases the possibility of lower reliability. The bias is less significant the more competent the key informants are636. Special attention was put on informant selection. The questionnaire was sent to the appropriate director-level person or R&D-responsible person in the selected companies. These persons were considered to have the best possible knowledge of the investigated phenomenon. If the person was not in the rightly defined position in the organization to respond to the survey, that person was requested to forward the questionnaire onto the correct person. The study relies on perception-based measures of informants both in dependent and independent variables when objective measures were not available. This is not necessarily, however, a serious threat for reliability. For example, Dess and Robinson have found subjective perception and objective measures to strongly correlate in measuring organizational performance in terms of return on assets and growth in sales637. Further, they emphasize that subjective measures are useful especially in attempting to operationalize broader, non-economic dimensions of organizational performance638.

635

Nunnally 1978 636

Ernst and Teichert 1998 637

Dess and Robinsson 1984 638

The literature analysis revealed that tested measurement constructs available for studying management control in the front end of innovation are limited. Lack of verified constructs resulted in the need to create new items and measurements constructs. In addition, this study relies on a single management representative’s own report of both independent and dependent variables in each company. This self- reporting may cause common method variance i.e. cause additional correlation among variables639. Several preparations and remedies were used to remove common method variance. First a priori verified measurement items were used whenever possible. Different items belonging to the same construct were asked in different places in the questionnaire form in order to avoid ‘consistency motif’, i.e. a tendency to maintain a consistent line in a series of answers640. As a post hoc remedy, some scale trimming was made, i.e. some measures causing overlapping constructs were eliminated641.

Herman’s one-factor test was also used to analyze common method variance. All the independent variables, and dependent variables separately, were entered in the factor analysis simultaneously. This resulted in 7 independent factors and 2 dependent factors as expected. In addition, the first general factor accounted only for 23.45% of the covariance of independent variables and 32.52% of the covariance of dependent variables respectively. This gives some indication that common method variance is not a serious problem in this study.642 However, an upward shift in the distribution of responses give reason to doubt that some ‘social desirability problems’ may exist, i.e. respondent may have answered questions in a manner that present a respondent in a favorable light643. This was despite the fact that full confidentiality and anonymity, if desired, were promised to respondents.