The Questionnaire

One set of questions was developed. The questionnaire included questions relating to all variables in the present research model and some general questions such as the personal details of the manager. The development of the questionnaire followed the guidelines of de Vaus (1992) and Dillman (2007). The empirically based suggestions from Andrews (1984) were used where considered relevant. The set of questions is included in Appendix I – Part A.

5.5.1 Development of the Questionnaire

The questionnaire was developed adhering to the following criteria.

a. For the measurement of most variables in the research framework, a number of items for each variable were included so that multi-item scales could be developed. Operationalisation of a variable in this way captures the complexity of the construct, simplifies data analysis, increases reliability, enables more precision, and increases validity (de Vaus, 1992). However, part of one variable (managerial performance) employed a single-item. The use of a single-item was justified in Chapter 4. Questions were kept as short as possible. The questionnaire contained 79 questions in eight parts.

b. Question clarity is important. Language was kept as simple as possible, instructions were carefully worded, and definitions were given when considered necessary.

c. Questions were asked in a direct fashion and most were closed.22 Care was taken to avoid double-barrelled and ambiguous questions. Some questions were reverse-worded to avoid response set bias.

d. The characteristics of the response scales were carefully considered in the light of Andrews (1984). Five-point Likert scales from one to five were used for most of the items requiring an opinion. Although not in accordance with Andrews’ (1984) assertion that more response categories leads to higher validity and lowers the measurement error, five-point Likert scales are widely used and accepted in social science research. In fact, the present researcher argues that they are the appropriate technique for the type of questions asked in this survey. Contrary to Andrews’ (1984) finding that data quality increases when only the end points and some intermediate points are labelled,23 the present survey scales were all labelled. It is expected that with the explicit meaning labelled in all categories, the indication for every possible answer will be much clearer. Consistent with his suggestion, the answer categories include an explicit “Don’t know” option for most of the questionnaire items. In the present study the “Don’t know” option was modified into “No Basis for Answering” option. Andrews (1984) found that the inclusion of an explicit “Don’t know” option increases data quality as it provides an opportunity for respondents not to answer if they lack information to do so.

e. In respect of the length of both the introduction and questions, the current study, for the most part, followed Andrews’ (1984) suggestion. The introduction to each part was within the recommended 16-24 words unless a clarification of terms was necessary. Most of the questions were medium length (16-24 words) although some were shorter (i.e., less than 16 words) and some were longer (i.e., more than 24 words). Most questions (including all questions using a Likert scale response) were phrased in comparative terms.

22 _{Questions of financial and non-financial measures commonly used to evaluate managerial and}

divisional (business unit) performance (Part 6) invite divisional (business unit) managers to add other performance measures if they wish, and many did.

23 _{Andrews (1984) suggested that this finding was surprising and not yet fully understood and}

f. The length of each part was not considered explicitly in the development of the questionnaires; rather, the items relating to each variable were grouped together.

g. In accordance with Andrews (1984), the position of items within the questionnaire was carefully considered. Andrews found that data quality was lower when items were at the beginning or at the end of the questionnaire. In the present study, the ‘easy’ question (i.e., demographic data) was placed at the end of the questionnaire.

5.5.2 Pilot Testing

A thorough literature review related to each instrument was presented in Chapter 4. Accordingly, most of the instruments in the current study were adapted and modified from previous studies, while some of the items were developed where necessary. For example, the instruments for variable participation in the development of performance measures are developed based on Kenis’s (1979) questionnaire, while the instruments for variable procedural fairness are derived mostly from Little et al. (2002) (see Chapter 4, Section 4.5 for the discussion of the operationalisation of the key variables).

There were three steps taken during the questionnaire development process. First, group discussions were held with up to ten fellow academics and fellow PhD students in the School of Accounting and Finance. These discussions focussed on both the reliability and validity of the proposed items for the instruments.

Second, a mini-pilot project was conducted where the draft survey questionnaire was sent to a few academics outside the University and a few managers for feedback. Three academics and three managers (i.e., business director, business analysis manager, and senior business banking manager) had agreed to participate in the mini-pilot project. This mini-pilot project focussed on the wording and understandability of the questions and the covering letter, the setting out of the questionnaire, and the time estimates to complete the answers. Some minor changes to the questionnaire were made as a result of this mini-pilot project.

Application to the human research ethics committee of the University for approval also resulted in some minor changes. Conditions of approval included a guarantee of confidentiality, an outlined procedure for safeguarding the data, and an emphasis on the voluntary nature of the responses to complete the questionnaire.

Finally, another pilot test was undertaken where the survey questionnaire, along with a feedback questionnaire evaluation form, was sent to a few division (business unit) managers. The feedback questionnaire evaluation form is also included in Appendix 1 – Part A. The pilot project was intended to get feedback from actual targeted respondents and to test whether the method to determine the name of each division (business unit) and the address were correct. The survey was printed in two different colours – white and yellow. The white paper survey was sent directly to the manager division (business unit) while the yellow paper survey was sent to HR managers, asking them to distribute it to the manager division (business unit). This approach has been used due to the unavailability of locating all the relevant addresses and also provided the opportunity to explore any differences in response rates for the different distribution methods.

Eighty-two surveys were sent out for the final survey questionnaires for pilot testing. These amounts consisted of 60 surveys containing the address of each division (white paper) and 22 surveys that were sent to the HR manager (yellow paper) due to address unavailability. Nine managers (11%) completed the questionnaire and provided valuable feedback. The feedback includes the length of the questionnaire; the readability/difficulty of questions; the questions that should be omitted or included (if any); and additional comments from the respondents. The additional comments from the respondents were mostly about the inclusion of performance measures used in their division. Those responses consisted of white and yellow paper surveys. Consequently, the method to determine the name and the address of the division (business unit) managers seemed appropriate.

The questionnaires were then amended as necessary and administered as detailed in Section 5.7.