Quality of the questionnaire

The quality of the questionnaire can be determined by examining the reliability of the findings, the validity of the questionnaire, and the research ethics of the questionnaire. These three parts for determining the quality will be addressed in the following sub-paragraphs.

4.3.1.

Reliability

Saunders et al. (2009) refer to reliability as the extent to which the data collection techniques or analysis procedures will yield consistent findings. According to Babbie (2010) reliability refers to the quality of the measurement method that suggests that the same data would have been collected each time for repeated observations of the same phenomenon. Since this research is conducted at one-point in time there is no way of telling whether a repetition of the research will result in the same findings. However, timing will not influence the results of the participants much since the subject of the survey is not changing rapidly. The strategic alignment experiences of organisations nowadays are not going to change in a week or a month. Even though there is no way of telling whether another research will have the same results, it is tried to make the survey as reliable possible. Since the questionnaire is entirely anonymous and conducted online there is no threat of subject or participant bias, which means that participants do not have to be afraid of what others might think. The survey is set up with a clear structure and has been several times tested to make sure that there is no wrong interpretation possible of the questions; therefore observer error is avoided.

The actual questionnaire is, just like in the pre-test, tested for reliability with the use of Cronbach’s Alpha. The internal consistency of the constructs is measured and is expressed with a number between 0 and 1. The Cronbach’s Alpha gives an indication whether the items assigned to a construct are consistently reflecting the construct. The scores of the Cronbach’s Alpha test can be found in Table 11. For all the constructs the Cronbach’s Alpha’s lie between the range of 0.7 and 0.95, suggesting that there internal consistency for the constructs.

Table 11 - Cronbach’s Alpha for the actual questionnaire constructs Construct α (Cronbach's Alpha)

Strategic alignment (STA) 0.743 Culture and shared beliefs (CSB) 0.851 Organisational capabilities (ORC) 0.890 Communication (COM) 0.927 Return and risk (RAR) 0.864

4.3.2.

Validity

While reliability is concerned with the ability of an instrument to measure consistently, the validity is concerned with the extent to which an instrument measures what is intended to measure (Tavakol & Dennick, 2011). Consequently, validity is concerned with the extent to which the measure accurately reflects the concept that is intended to be measured (Saunders et al., 2009). Babbie (2010) explain that there are several criteria which help determine the validity of the measure. The first criterion, face validity, is about the quality of an indicator that makes it seem a reasonable measure of a variable. Through the literature review an assessment is made to subdivide the variables under four main variables. This assessment is made by looking critically at the indicators; each indicator has aspects which are relevant for the main variable. During the pre-test there were no indications that the participants found the indicators for a certain concept confusing. The second criterion is criterion-related validity, which is the degree to which a measure relates to some external criterion, sometimes called predictive validity. It is concerned with the ability of the measures (questions) to make accurate predictions (Babbie, 2010). In some cases, answering a simple question like whether an organisation successfully implements strategies can lead to some forecasting for the other variables. With the analysis of the data these criterion-related variables are kept in mind. The third criterion, construct validity, is the degree to which a measure relates to other variables as expected within a system of theoretical relationships. It is about the logical relationships among variables. Construct validity refers to the extent to which your measurement questions actually measure the presence of those constructs you intended them to measure (Babbie, 2010). Since the measures are directly related to the indicators, by conducting the operationalisation, it is sure that a question actually measures an assigned indicator. The last criterion is content validity and it is about the degree to which a measure covers the range of meanings included within the concept (Babbie, 2010). Content validity refers to the extent to which the measurement device provides adequate coverage of the investigative questions. Some of the indicators consist of multiple concepts which might be confusing for a respondent because they do not know to which concept to respond. Therefore, some indicators are split up into multiple questions to make sure that all the concepts are covered but do not get confused with each other.

To test whether the indicators measure what is intended to be measures a bivariate correlation analysis has been used. The direction of the relationship of the constructs with strategic alignment is determined, the constructs influence strategic alignment. Therefore a one-tailed test applies to determine the significance. The cross-correlation matrix can be found in

Appendix D - Correlation between indicators. This cross-correlation matrix usually exists of all indicators of the constructs. Since this matrix is too large to fit on the page only the indicators of strategic alignment (STA) and culture and shared beliefs (CSB) are shown. The green areas indicate the correlation between variables within a construct. From the table can be seen that usually the correlations between indicators of a certain construct are a higher than with indicators of other constructs. However, this is not always the case. For instance, the correlation between the direct influences on strategic alignment, like STA3 and STA4b (a score of 0.101), is lower than the correlation between STA3 and CSB1 (a score of 0.167). Nevertheless, based on the other determinations of validity the indicators seem to be valid enough to continue the analysis.

4.3.3.

Data distribution

Skewness is the measure of asymmetry of the distribution; it measures the degree and direction of the asymmetry. A positive value indicates that the long tail of the distribution is with the higher values, thus the distribution is skewed to the right. Accordingly, a value of zero indicates a symmetrical distribution such as a normal distribution. From Table 12 can be seen that most of the constructs are not normally distributed but have a distribution which is skewed to the right, thus the mean is higher than the median. Only the CSB construct has almost a normal distribution with a score of 0,073.

Kurtosis is the shape of the distribution in comparison with the normal distribution; it measures the heaviness of the tails of a distribution. If the kurtosis is positive it means that the top of the distribution is higher than the normal distribution. A value of zero indicates that the shape is similar to the shape of a normal distribution. From Table 12 can be seen that the kurtosis scores for each construct is larger than zero, which means that the tails of the distributions are heavier than for a normal distribution.

Table 12 - Skewness and kurtosis of the questionnaire constructs Construct Skewness Kurtosis

Strategic alignment (STA) 0,428 2,740 Culture and shared beliefs (CSB) 0,073 1,067 Organisational capabilities (ORC) 0,306 1,003

Communication (COM) 0,271 0,890

Return and risk (RAR) 0,521 0,080

Based on scores of the skewness and kurtosis can be determined that the constructs do not have a normal distribution. Therefore, a parametric test does not apply for this research. For a parametric test assumptions are made about the distribution of the variables, like that there is a normal distribution. Since there is no normal distribution the non-parametric tests can be used to analyse the data, which are not based on strong assumptions.

4.3.4.

Research ethics

There are some general ethical issues which are considered in this research. The first is the privacy of possible and actual participants which is harboured since the questionnaires can be answered anonymous and are completed online. The respondents can choose anonymity by not leaving their e-mail address behind at the end of the questionnaire. Anonymity exists when neither the researchers not the readers can identify a given response with a given respondent. A respondent can choose for confidentiality when they leave their e-mail address behind. In this case confidentiality means that the researcher can identify a given person’s response but promises not to do so publicly (Babbie, 2010). The second ethical issue is the 34

voluntary nature of participation and the right to withdraw partially or completely from the process (Babbie, 2010). The respondents are asked to fill in the questionnaire, whether they do so is entirely their own choice. Even when they have started to fill in the questionnaire they can choose to stop at any moment. The third ethical issue is consent and possible deception of participants. With every questionnaire a clear introduction is given about what is asked from the respondents and what is done with the results. When they start with the questionnaire they give their consent to the indicated terms. The fourth ethical issue is the maintenance of the confidentiality of data. The gathered data is not distributed to others without a clear reason; the names of the respondents are not known and will not be registered without them choosing so. The fifth ethical issue is the reactions of participants to the way data is collected. The data is collected through an internet-mediated questionnaire. Therefore, there is no physical contact with an interviewer or someone conducting the questionnaire. In this way embarrassment, stress, discomfort, pain and harm are avoided. The sixth ethical issue is the effects on participants of the way in which data is analysed and reported. It is made sure that the respondents are clear about what will be done with the data from the questionnaire, if they feel uncomfortable in any way they can choose not to complete the questionnaire. The last ethical issue which is considered in this paper is the behaviour and objectivity of the researcher. There will be no direct contact, only through an e-mail. With use of the appropriate writing language to the respondent the behaviour will not be a problem.