Methodology I (Qualitative): Phase 1- Conjoint Development

A preliminary qualitative assessment of the different balanced scorecards was carried out to identify performance measures which are common to balanced scorecards as well as how these measures are usually presented. As suggested by Kaplan and Norton (1992), balanced scorecards consisted of four perspectives; Financial, Customer, Internal, Learning

& Growth. Each of these perspectives in turn consists of up to four performance metrics.

Although there have been a number of studies examining differences in the use of common and unique measures of balanced scorecards by organisations (Banker et al., 2003; Gagne et al., 2006), the researcher has considered the use of the common measures, as the study is primarily concerned with how these measures are visual ordered and presented. A list of metrics based on the extensive work of Cohen et al., (2005) in this area is presented in Table 7.1. A further compilation of balanced scorecard metrics can be found in Appendix A l.

Ratio Abbrerâlion Literature

Return on assets ROA Evans (200-1) and Ittner et oL (2003)

Return on equity ROE E\nns (200-1), Kaplan and Atkinson (1998) and Kaplan and Norton (1996a)

Inventory turnover rr Banker et al (200-1) and Lipe and Salterio (2002) Debtois turnover DT Banker et al (20&1) and Lipe and Salterio (2002) Sales margin SVI Lipe and Salterio (2002) and Libbj' et al (200-1)

Assets turnover AT Banker et al (2001)

Qualitative variables Abbreviations Literature itlem al business and production process pers{)cctivc Effective dispatching of orders (in EDO

Perceived level of trust to the products PLoT After-sales service

Johnson et a l (2005) and Kaplan and Norton (1996a)

Johnson et al (2005), Laudon and Laudon (200-1), Cherhall (2005), Aidemark (2001) and Lipe and Salterio (2002)

Johnson et al (2005), Aidemark (2001) and Kaplan and Norton (1996a)

Table 7.1: Commonly adopted balanced scorecard metrics (Cohen et a l, 2005.)

Following the identification of metrics commonly adopted in balanced scorecards, the researcher extracts metrics which are of relevance to context of the task selected by the researcher, which is an outsourced IT consultancy. Therefore, for the purpose of this

research, the following metrics were adopted for each perspective of the balanced

Because this preliminary phase of study is important in obtaining a good grasp of measures that are commonly presented on balanced scorecards, the purpose of this phase of the study can therefore be best described as an exploratory qualitative assessment. Data obtained through the observation or interviewing individuals to promote the understanding of the phenomena at the exploratory stages of a study, is often referred to as qualitative in nature (Sekaran, 2003).

7.4.2 Type of Investigation

This initial phase of the study may also be classified as a correlational study as the intention is not to establish a causal connection but to merely identify the information features that are associated with the arrangement and presentation of information features on a balanced scorecard. Sekaran (2003) stated that this type of investigation helps to

identify the important factors associated with a problem. Sekaran (2003) concluded that more often, the researcher is asked to identify the crucial factors associated with the problem, rather than establish a cause-and-effect relationship given the complex nature of relationships that exist amongst several organisational factors.

7.4.3 Extent of Researcher Interference

Correlational studies are often carried out within the natural environment of the organisation or target audience with minimum interference from the researcher (Sekaran, 2003). This phase of the study was carried out with minimal interference from the researcher.

7.4.4 Study Setting

This phase of the study principally took place on the internet through the aid of a computer by studying and observing several balanced scorecards, it can therefore be said that this phase took place in a non-contrived setting (i.e. the natural environment where balanced scorecards are naturally viewed). Sekaran (2003) noted that correlational studies are invariably conducted in non-contrived settings, whereas most rigorous causal studies are done in contrived lab settings.

7.4.5 Unit of Analysis

Since, the balanced scorecards of different organisations were the sources of information for this study; data collected from each organisation was being treated as an individual data source. Therefore, the unit of analysis for this preliminary study is the balanced scorecard.

Unit of analysis may be broadly classified as individual, dyads and groups (Sekaran, 2003).

7.4.6 Data Collection

Data was collected only once, over a two-week period, from February 2"^ to February 16*, 2010. This can be regarded as a one-shot or cross-sectional survey (Sekaran, 2003).

An unobtrusive data collection method was adopted for the study through careful observation and assessment of the balanced scorecards of different organisations.

Unobtrusive measures can be analysed, as surveys can be; letters, memos, and other

"qualitative" information can be content-analysed, just as interviews and direct observations are. The balanced scorecards assessed cut across government, non-profit and commercial organisations. A few examples are listed in Figure 7.2, 7.3 and 7.4.

R egional Airline B a la n c ed S c o r e c a rd

Credit Card Company Balanced Scorecard

ie«|. tS*pàA*di fttrjtWCK

T

Figure 7.3: An example o f a commercial organisation’s balanced scorecard (Scorecard Institute, 2008).

Mobile Phone Manufacturer Balanced Scorecard

i* * c t ' r —

.. t L . 'T ,» > -«## » ^

• V>M» f •

a GB a ^{t;: , »}

':iL

Figure 7.4: An example o f a mobile phone company’s balanced scorecard (Active

7.5 Conceptual Experimental Design

From the assessment, it was observed that different order of visual presentation of the four perspectives is being adopted across various organisations. The four main information features (balanced scorecard perspectives) previously identified were; Financial perspective, Customer perspective. Internal perspective and Learning & Growth perspective. Similar to Experiment 1, the researcher adopts a top-down visual ordering based on the criticality of display i.e. Critical, Important and Desirable. Therefore, this study would seek to prescribe a user centred design through the use of a conjoint technique. In addition to the visual order of presentation of the perspectives on a balanced scorecard, another important feature of balance scorecards is presentation formats. In practice, information is typically presented either in tabular and graphical representations on balanced scorecards. In this regard, Vessey (1991) concluded that performance on a task will be enhanced when there is a cognitive fit (match) between the information displayed and the required task type; that is, when graphs support spatial tasks and tables support

Critical Critical Critical Critical Table Tables Tables Tables

Important Important Important Important Graphical Graphical Graphical Graphical Desirable Desirable Desirable Desirable

7.6 Methodology II (Quantitative): Phase 2&3- Conjoint Questionnaire Design and