Young Hoon and Mincheol’ w effective instrument for evaluating websites in a comprehensive manner, using the modified balanced scorecard (BSC) evaluation approach, and the analytical hierarchy process (AHP).
BSC is an example of an evaluation instrument that assesses website effectiveness.
The BSC evaluation approach is a multidimensional approach that emphasises the links between the four perspectives (technical, marketing, customer, and internal business); each of these is evaluated through a set of critical success factors (CSFs), which are also based on a fi M 1999). The customer perspective relates to efforts to improve customer satisfaction, and is generally evaluated through four CSFs, such as user-friendliness, website attractiveness, the interactive functions of the website, and user security. The internal business processes relate to internal processes that are essential to achieving customer objectives, and are generally evaluated by three CSFs: ease of website maintenance; information on the organisation; and the profile of the organisation, and so on, for the other two perspectives.
The BSC approach is based on the premise that in any organisation, certain factors are critical to its success (Rockart 979 “w ” (Morrison et al. 2004). The BSC requires businesses to identify their critical success factors, for a strategic business perspective (Self 2004). In doing so, the BSC broadly informs management of what is wrong, and gives them a comprehensive view of the business, by integrating organisational performance factors into effective decisions (Kaplan and Norton 1992). Thus, the BSC approach offers a balance between the organisation's goals and objectives, as well as between the individual and a company's portfolios. The BSC approach is flexible enough to be customised, in order to suit the characteristics of specific industry sectors and gauge management effectiveness (Kaplan and Norton 1992).
Based on a modified BSC approach using the four previously mentioned perspectives, Young Hoon and Mincheol (2010) generated a set of 23 CSFs, which were developed through discussions with experts, and reviewing and examining previous research on business, hospitality, and tourism. This is a critical process for the construction of a standardised measurement tool for website evaluations,
37
because the identified CSFs can also be common evaluation factors for tourism website evaluations. For example, the technical perspective was evaluated by six CSFs: the currency of links; the effective use of hypertext markup language (HTML);
browser capability; spell check; and the stability of the website. Ten critical success factors were used and evaluated for the marketing perspective: branding; tangibility of products and services; market segment and target marketing; positioning;
marketing research and database marketing; relationship marketing; programming;
packaging; partnerships; and marketing evaluation. Four CSFs were used to evaluate the customer perspective of websites: user-friendliness; website attractiveness; the interactive functions of the website; and user security. The internal perspective was evaluated through three CSFs: ease of website maintenance; information on the organisation; and the profile of the organisation.
After identifying the main CSFs, Young Hoon and Mincheol (2010) developed the final website evaluation instrument, through the analytical hierarchy process (AHP).
AHP is a decision-making method that was developed by Saaty (1980) to calculate each factor, in order to compare and rank them, together and individually. AHP enables the prioritisation of each of the identified CSFs, and ranks them by their level of importance, with a consistency of measurement. It is a systematic measurement concerned with dominant priorities from pairwise comparisons of homogeneous or nonhomogeneous elements, through clustering elements. AHP compares and ranks each category in a hierarchical structure, including the goal (final object), the top criteria (the four modified BSC perspectives), and the sub-criteria (CSFs). A strength of AHP is its systematic process; thus, a theoretical foundation followed by its viability, usability, and applicability for a decision-making tool. AHP is composed of three main steps: (a) decomposition (structuring the decision problem); (b) comparative judgement (judgement of each criterion at hierarchical levels, through pairwise comparison); and (c) determination of priority ’ w H w Young Hoon and Mincheol skipped the first step of AHP, because the modified BSC was used for the first step, and the hierarchical level already existed. As the BSC approach was used to design a standardised evaluation index, AHP was employed to determine the weight of each CSF.
The results of this evaluation study showed that customer perspectives had the highest weight, followed by marketing perspectives, technical perspectives, and internal perspectives. For the technical perspective, the stability of websites was
38
found to be the most important factor, followed by download time, the currency of links, browser compatibility, spell check, and the effective use of HTML. The results of the marketing perspective showed that the most significant factor was tangibility of products and services, whilst programming was rated as the least significant factor for the marketing perspective. For the internal perspective, ease of website maintenance was indicated as the most important factor, followed by information on the organisation, and the profile of the organisation. From the customer perspective, which was the most preferred perspective amongst the four perspectives, user security was the superior factor, followed by user-friendliness, website attractiveness, and the interactive functions of the website.
Young Hoon and M ’ (2010) evaluation methodology developed a multidimensional and weighted scale, using the modified BSC approach and AHP.
By using these two methods, their study tried to develop standardised matrices for website evaluations, using a systematic process. The importance of each factor, with respect to contributing to the preferred component, was examined, and at the end, pairwise comparisons were conducted to obtain the relative importance of each CSF. Objectively, AHP assists in ranking the four perspectives, along with each of the CSFs. In terms of practical aspects, a website evaluation method was developed systematically and effectively, and has been a validated instrument that provides important factors for evaluating websites. However, although the evaluation methodology has several advantages, it also has several drawbacks.
The Balanced Scorecard (BSC) framework has been frequently revised and extended in subsequent studies, as a valid instrument for website performance evaluations (Feng et al. 2003; Douglas and Mills 2004; So and Morrison 2004; Han and Mills 2006). However, it is a simplified, conceptual model for measuring website effectiveness, and it must be widened to include other dimensions and criteria (Perdue 2001). Morrison, et al. (2004 4 “…
perspectives and measures available, but a general lack of standards and ” B C j complemented, to provide a standardised approach for the evaluation of tourism w w I “ ” w of just one person (one perspective) to conduct an analysis, results in this approach being very subjective in nature. This approach ignores other perspectives, such as those of customers, suppliers and the management of the organisations involved (Chung and Law 2003).
39
Furthermore, although the BSC and AHP are, together, very applicable and rational ways of using multidimensional scales and determining weights through prioritising factors, there was no exact guideline on how experts were chosen for this study, and the results of the study might have been constructed, based on the opinions of the participating experts. Therefore, a more critical process is required for selecting the experts. Research has shown that a more sophisticated model should be employed, in order for a more in-depth assessment to occur (Self 2004). Young Hoon and Mincheol “… B C w literature review and modified again, it might not have constructed the process of the proposed model perfectly”.