The Research Method

In social sciences, the most commonly used methods to examine the characteristics and interrelationship of sociological and psychological variables is the survey method (Robert, 1999; Nazari, Kline and Herremans, 2006). Researchers have used surveys to collect data on a variety of topics, for example, performance measurement with budgeting, managers’ perceptions, managers’ participation, etc. The present study also employs the survey method to collect its data. The justification for the survey method along with a mail questionnaire is provided in sub-sections 5.2.1 and 5.2.2.

5.2.1 Why a Survey Method?

Most of the previous research in the balanced scorecard (BSC) area employs an experimental research design (see, for example, Lipe and Salterio, 2000, 2002; Libby et al., 2004; Roberts et al., 2004; Banker et al., 2004; Dilla and Steinbart, 2005) to evaluate the performance evaluation processes. However, in those

experiments they do not explain the development of the performance measures. In those experiments, participants acted as if they were managers in the performance evaluation process where performance measures were imposed on them. The results may have been different had the managers been involved in the development of the performance measures. Hence, this study will use a survey research method to address the research question by explicitly incorporating manager involvement; and to test the developed hypotheses.

In the social sciences, the survey method is used widely to examine empirically the characteristics and interrelation of sociological and psychological variables (Roberts, 1999; Nazari et al., 2006). Its development and application in the twentieth century has ‘profoundly influenced the social sciences’ (Kerlinger, 1986). It has many advantages such as being a cost-effective manner of collecting a large quantity of generalisable data while avoiding interviewer bias (Roberts, 1999).

Why is a survey method appropriate for this study? As Nazari et al. (2006) state, there are several underlying assumptions in survey research using self-report of

attitudes, values, beliefs, opinions and/or intentions. These self-report assumptions, discussed below, reflect the present research’s central purpose; which is to examine the perception of division (business unit) managers on the performance measures used in the performance evaluation process.

First, the respondents are the most reliable source for certain types of information (Nazari et al., 2006). In the performance evaluation process, the

fairness perception of the performance measures used in the process is crucial. Division (business unit) managers are the most reliable source of information since they are involved in the performance evaluation process both as an evaluator as well as the one being subject to the process.

Second, those subjective perceptions actually matter. One can argue that

powerful than reality itself since very often people act on their perceptions (Nazari et al., 2006).

Third, perceptions can be demonstrated to be linked to outcomes of interest to organisations (Nazari et al., 2006). In other words, perceptions influence the

behaviours that have real consequences for organisations. The common-measure bias found in previous studies might increase the unfairness perceptions of the performance measures in the performance-evaluation process. Those perceptions can negatively influence the behaviour of divisional (business unit) managers, such as lowering their performance, which impacts on the organisation. Given the main objective in this study, as well as considering the above assumptions of the self-report survey, a survey method is appropriate for this research.

However, as one would expect, this method is not free of criticism (Marsh, 1982; de Vaus, 1992); furthermore, Young (1996) questions its contribution to management accounting research. The main fundamental concern of those critics is the validity and reliability of the survey method (Van der Stede, Young and Chen, 2005; Young, 1996). Therefore, in order to minimise any potential problems, every effort will be undertaken to obtain qualified data to assess adequately the phenomenon of interest. This will be done with the appropriate survey questionnaire development and administration of the survey. These issues are discussed in Sections 5.5 and 5.7.

5.2.2 Why a Mail Questionnaire?

Mail questionnaire as a technique of data collection has been criticised due to its possible lack of response and the inability to verify the responses given (Kerlinger and Lee, 2000). However, the number of surveys conducted by self- administered mail questionnaires exceeds the number of interview surveys conducted each year, although, it is difficult to quote the exact numbers (Dillman, 2007). In managerial accounting research, a mail questionnaire survey is the survey method most frequently used (Van der Stede et al., 2005).

Nazari et al. (2006) stated that the aim of survey research in management accounting is to measure certain attitudes and/or behaviours of a population or a sample, and can be used either for exploratory or confirmatory purposes. Exploratory research is a study to find out basic facts and become familiar with the subject of the study. It usually focuses on finding out what construct to measure and how to measure them (Pinsonneault and Kraemer, 1993). On the other hand, confirmatory research is a theory testing study that assesses relationships between constructs that have been defined in prior research studies (Nazari et al., 2006).

With regard to the purposes of the survey, one of the characteristics of this present study is to confirm whether common-measure bias found in prior research experiments truly exists in organisations. If it exists, one must determine what the impact on the managerial performance is and how to reduce it. To confirm prior research findings, a large data set is needed from real organisations where the most relevant technique to gather such data is via a mail questionnaire survey.