Reliability

After the validity assessment and pilot study were completed and the suggested changes made, the data collected from the 10 participants were entered into the computer

116

software (SPSS: Statistical Package for the Social Sciences). The aim was to check the reliability of the instrument. Reliability has been defined by Neuman (2000: 164) as:

The extent to which the results of the study would be similar if carried out again by a different researcher.

In another definition, reliability refers to the consistency of a measure, and this often involves two separate aspects: external and internal reliability. External reliability refers to the degree of consistency of a measure over time; that is, the researcher should expect the same results if s/he applied the same scale on different occasions or with a different set from an equivalent population. Internal reliability, which is the more common and connected with multiple-item scales, raises the question of whether each scale is measuring a single idea and hence whether the items that make up the scale are internally consistent (Bryman and Cramer, 2001).

After consultation, the researcher decided to apply an internal reliability test to assess the questionnaire’s consistency. A number of methods exist for estimating the internal reliability of a questionnaire. Among these are the split-half (sub-divided test), the Kuder-Richardson method of rational equivalence, Guttman’s test of reliability, Hoyt’s Analysis of Variance Procedure and Cronbach’s Coefficient Alpha. Each method is used with a specific type of data, according to the aims. The currently widely-used method is Cronbach’s Alpha, which calculates the average of all possible split-half reliability coefficients. A correlation coefficient is then generated that varies between 0 and 1 and the nearer the result to 1 (preferably at over 0.8), the more internally reliable the scale (Oppenheim 1996: 159; Bryman and Cramer, 2001: 63).

The researcher decided to use the Cronbach’s Alpha method to test the internal consistency of the scales, primarily because this test is the most common and can easily be computed using SPSS. To conduct this test, the questionnaire items were arranged into four dimensions, each intended to measure a particular aspect of the modified NPM reform model. The dimensions and issues were as follows: 1) structure: “organizational decentralization”; 2) the organizational process: “introduce private sector management techniques”; 3) the budgetary process; and 4) human resource management (HRM) and total quality management (TQM) issues.

117 3.6.7 Implementation

Before conducting the field study, the researcher was aware that the research environment in Bahrain required attention. Blaxter et al. (1996) emphasise the importance of context and advise that researchers should base their strategies on the realities in which the research is being conducted. This is absolutely true; conducting a field study within the Arabian context is significantly different from that in the West. In Bahrain, the cultural setting imposes a number of constraints and limitations. Government officials are usually hesitant to participate, particularly in studies that relate to public sector management, unless researchers obtain written letters from the authorities concerned indicating approval. Further, Omani researchers (e.g. Muharami, 1993; Maawali, 2000) indicate that it is difficult to investigate organisational issues in Oman (a similar environment to Bahrain) from a position outside the organisation. They advise researchers to benefit from personal and informal contacts, as nothing meaningful can be achieved through official means alone. They also advise that researchers should administer procedures in person to obtain good results, although doing so may require more time and effort.

In consequence, the following procedures were applied in distributing and collecting the questionnaires:

1. In Bahrain, a support letter was issued by the Ministry of Foreign Affairs and Information (the researcher’s Ministry), which indicated that the researcher had been given permission to conduct his field study and encouraging public Ministries and agencies to cooperate. These letters were used to support the fieldwork, although they had little influence; this led the researcher to use his access as an insider to benefit from this network of relationships.

2. After determining the number of questionnaires to be distributed to each Ministry and agency (according to CSB 2008 statistical information), a series of arrangements were made to meet the Ministries’ HR officials, in person, and 3. explain to them the objectives of the study and the kind of help and cooperation

needed. Ministries were provided with a guide explaining how questionnaires should be distributed and when they would be collected.

4. In the covering page of the questionnaire, participants were asked to enclose completed questionnaires in an attached envelope addressed to the researcher, to

118

ensure that envelopes were sealed and return them to the authority from which they were received. Participants were assured of confidentiality and requested to take their time in answering all questions carefully. In addition, the researcher’s telephone, fax and e-mail details were provided for use with any inquiry in relation to the study.

5. To ensure a smooth process, one “contact person” was identified in each of the Ministries or agencies where questionnaires were distributed. They were individually briefed about what was required and were asked to follow up questionnaire collection.

The distribution of the questionnaire (in both English and Arabic) started in August 2008 and a period of four months was scheduled for the collection of completed questionnaires. During this period, lines of communication were opened with the contact people. Two means of distribution were adopted: hard and soft copy. The researcher used an Internet website: www.nesdbh.com (Appendix A-7) hired for 3 months specifically for this reason. English and Arabic language instructions accompanied each set of the questionnaire, which was intended to exploit the increasing popularity of Internet usage. Accordingly, the website address was distributed and SMS messages used to inform the Ministries’ coordinators to inform their managerial staff and provide an opportunity to use either a hard or soft copy. Online distribution, however, is in common use within public managerial research in Bahrain or perhaps even in the GCC. The online version was designed to be accepted only if completed answers were obtained; otherwise, it would be automatically rejected. The website was designed to allow each respondent only one attempt, to avoid the duplication or triplication of use that could mislead the study. This approach resulted in 57 respondents in addition to 301 respondents who preferred to use the hard copy (although the researcher had hoped the soft copy method might generate more attention in this globalised era of electronic transformation).

Distribution and collection of the questionnaires required real effort and time. It involved following up the contacts in the Ministries by telephone and visiting them every week or so to collect the completed questionnaires. As a result, out of the 691 questionnaires distributed, 358 (52%) were completed and returned, which supports the research sample’s validity. The main factor behind this high response rate was the fact that the researcher had access to contact people who were instrumental in encouraging

119

participants in their organisations and Ministries to complete the questionnaires and in following up collection.