Pre-Test and Pilot Study

The data collection stage in this research study was divided into three phases, namely the pre- test, pilot study, and main study. After designing the questionnaire, around 10 fellow Iranian Ph.D. students from universities in London were invited to a cognitive interview and asked about the questions. According to Willis (1994, 1999), the cognitive interview can be conceptualised as a modification and expansion of the usual survey interviewing process. The researcher has been specially trained to conduct cognitive interviews and was familiar with the procedures. Cognitive interviews can be differentiated from field interviews through the application of two varieties of verbal report methods: 1- think-loud and 2- verbal probing. For the purpose of this study and to have a better understanding of respondents’ response to each question, the researcher has applied both these methods. At the first stage participants were asked to verbalise his or her thinking as he or she answered the questions (Davis and DeMaio, 1993; Bickart and Felcher, 1996). Then after respondents provided their answer to the relevant question, the researcher asked additional probing questions to further elucidate the subject’s thinking (Belson, 1981; Willis, 1994, 1999).

After analysing the results from the pre-test and doing some adjustments on the questions, exactly 85 copies of the new version of the translated questionnaire were sent to three organisations, one from each size category, for the purposes of the pilot study (Table 4.3). In total, 50 individuals replied with fully completed questionnaires, producing a response rate of 59%. The researcher was advised not to have too large a sample for the pilot study, as this may increase the possibility of losing potential respondents for the final data collection. The respondents were not aware that they were chosen for a test. After conducting the pilot study,

154

10 out of the 50 people who participated in the pilot study were chosen for face-to-face short cognitive interviews to obtain more knowledge and understanding of the questions and to identify whether there was any need for any change in wording or structuring of the questions. Then, final amendments were made to the questions before the main study was conducted.

Table 4.4: Number of Questions Sent and Received for the Pilot Study

Pilot Study Size No. of questions

sent No. of questions received Small 10 9 Medium 25 15 Large 50 26

According to Powney and Watts (1987), Creswell (2009), and Bryman and Bell (2011), a pilot study with a small sample helps to test three functions by:

 Checking whether or not the organisation under study meets the research requirements

 Putting the interviews’ structure and logistics to a practical test

 Acting as an opportunity for the researcher to develop his/her communication skills The motives for choosing to do a pre-test and pilot study before the final data collection were based on the work of Converse and Presser (1986), Bryman and Bell (2011), and Bryman (2012), who all agree that a researcher who can conduct more than one pre-test would be in a better position if they use a participatory pre-test first and an undeclared test second. Also, the aim was to test the clarity of the questions and to ascertain the reliability of the instrument used in order to achieve the research objectives in general. Additionally, the pilot study helped the researcher estimate the time needed to answer questions and if there was a need for any rewording on any of the questions.

The pilot study data collection was divided into two phases: the first phase consisted of a survey on national culture with 29 questions, and a second phase in which data was collected

155

on organisational culture, leadership style, and organisational effectiveness. The pilot study started at the beginning of June and ended by the mid July in 2012. Questions were sent by mail to designated persons in three different organisations to be distributed, which out of that, 50 replied to the researcher. The average time for answering questions was around forty five minutes in total, of which the national culture section took around 10 minutes, leadership around 10 and organisational culture and organisational effectiveness took around 25.

In order to measure the reliability of the questionnaire, the researcher measured internal consistency using Cronbach’s alpha as this instrument has been widely used to measure the reliability of scales. Although Bryman (2012) suggests that an acceptable Cronbach’s alpha is around .80, De Vaus (2002) proposed that 0.6 should be considered as indicative of acceptable reliability. Also, according to Sekaran (2003), the reliability of scales increases as alpha approaches 1.0. In general, an alpha less than .6 would be considered as indicative of poor reliability, with 0.7 considered acceptable, and above 0.8 considered high.

Table 4.5: Pilot Study Questionnaire Internal Reliability

No Description No of Cases No of items Cronbach’s alpha No of items deleted Cronbach’s Alpha 1 PDI 50 6 .83 None .83 2 UAI 50 5 .79 None .79 3 IDV 50 6 .50 1 .75 4 MASI 50 5 .85 None .85 5 Clan 50 6 .95 None .95 6 Adhocracy 50 6 .73 None .73 7 Market 50 6 .86 None .86 8 Hierarchy 50 6 .89 None .89 9 Transformational 50 20 .80 None .80 10 Transactional 50 8 .78 None .78 11 Passive 50 8 .72 None .72 12 Leader (comb of all 3) 50 36 .76 None .76 13 OE 50 41 .90 None .890

156

After analysing the data from the national culture section, the results indicated that Cronbach’s alpha for three scales was acceptable and that one was very low. The scores, before deleting any item, were in the range of 0.50 for IDV to 0.85 for MSI. After deleting one item from IDV, the range was acceptable, with alpha ranging from 0.75 for IDV to 0.85 for MSI (table above). In order to increase the reliability of the national culture questionnaire, the researcher had to delete item IDV6. Additionally, after collecting data for the pilot study, the researcher decided to totally delete 7 questions from the paternalistic category from the main study due to their insignificant contribution to this research and the lack of previous data on this variable. Therefore, the national culture questionnaire was reduced from 29 questions in the pilot study to 21 for the main study.

The second part of the pilot study was based on organisational culture, leadership styles and organisational effectiveness questions with 24, 36, and 41 questions included, respectively. After analysing the data on organisational culture, the result showed that Cronbach’s alpha for all scales varied within an acceptable range from .73 to .95 (Table 4.5). The results also showed that Cronbach’s alphas in the leadership styles section for all scales varied within an acceptable range from .72 to .80 and for all scales together (36 items, as leadership style was taken as one mediator), the range was .73 which suggests that they can be taken as one variable. Finally, after analysing the data from organisational effectiveness, the result indicated that Cronbach’s alpha was very high at .890 (Table 4.5).

The instrument also had face validity as the items in the questionnaire, on the face of it, appeared to measure the concepts that the researcher wants to study (Sekaran, 2003). Face validity can simply be improved by rewording and restructuring items in terms of what appears relevant and plausible in the particular setting in which it is intended to be used (Anastasi, 1983). Three questions, one focused upon organisational culture and two focused upon organisational effectiveness, have been rephrased and restructured after receiving comments from Iranian academics, who are experts in these concepts.