MEASURING PERCEIVED WEBSITE USABILITY

MEASURING PERCEIVED WEBSITE USABILITY

ABSTRACT. The objective of this research was to develop a short, reliable, and valid perceived website usability measurement scale. A sample of 350 participants was used to collect the necessary data. Exploratory and confirmatory factor analyses were performed to purify the proposed scale. Analysis indicated that the proposed multi-dimensional usability scale is reliable and shows evidence of construct and predictive validity. Academic and managerial implications were discussed.

KEYWORDS. Website usability, ease-of-navigation, speed, interactivity, user attitude

Jianfeng Wang, Ph.D., is an Assistant Professor of Marketing, Department of Business & Economics, Mansfield University of Pennsylvania, Mansfield, PA 16933

(e-mail:pwang@mansfield.edu)

Sylvain Senecal, Ph.D., is an Associate Professor of Marketing, Department of Marketing, HEC Montreal, Montreal (Quebec), Canada H3T 2A7 (e-mail: sylvain.senecal@hec.ca)

MEASURING PERCEIVED WEBSITE USABILITY

INTRODUCTION

The experience consumers have on a website is increasingly becoming an important topic both for academia (Agarwal and Karahanna, 2000; Novak, Hoffman & Yung, 2000) and for organizations using websites to market their products and services. The website design is an important determinant of visitors’ online purchases and revisit intentions (Hill, 2001; Klein, 1998). Moreover, Nielsen (2000, p.10) argues that “users experience usability of a site before they have committed to using it and before they have spent any money on potential purchases.” Thus, developing sites that are easy to use and that meet organizational needs is critical for organizations. One construct that may be useful in evaluating websites and consequently developing better websites is usability.

The objective of this research is to develop a short, reliable, and valid perceived usability measurement scale. The aim is to develop a parsimonious scale that can be used across websites. Thus, the measurement scale could be used for benchmarking purposes within an organization and/or across organizations. For instance, an organization could measure consumers’ perception of its website usability and of their competitors’ websites in order to benchmark their website with the competition. The development of a usability measurement scale that shows evidence of reliability and construct validity would also be useful to researchers in order to investigate the relationship between perceived usability and other relevant constructs such as attitude toward the website and intention to revisit the website (Cook & Campbell, 1979; Straub, 1989).

Usability and Functionality

The notion of usability is a key theme in the human-computer interaction (HCI) literature. Research in the HCI tradition has long asserted that the study of human factors is crucial to the successful design and implementation of technological devices. The overarching goal of a majority of the HCI work has been to propose techniques, methods, and guidelines for designing better and more “usable” artifacts. Drawing upon cognitive frameworks of human-computer interaction grounded in psychology, prior research developed user models that delineated the cognitive structures driving user behavior (Card, Moran, & Newell, 1983).

The quality of a website can be assessed in different ways. To date, studies of websites have focused on website functionality and on website usability. A system is said to be functional when it provides functions needed by users to perform their tasks (Goodwin, 1987). A website can be evaluated based on the presence or absence of certain functions or on the performance of those functions. However, past research has found that users’ acceptance of a system is contingent not only on its functionality but also on its usability (Davis, 1986; Goodwin, 1987).

The concept of usability can be defined as “how well and how easily a user, without formal training, can interact with an information system of a website” (Benbunan-Fich, 2001). Bernard et al. (1981) suggested that a “truly usable system must be compatible not only with the characteristics of human perception and action, but, most critically, with users’ cognitive skills in communication, understanding, memory, and problem solving.” A usability evaluation consequently assesses the ease of use of a website functions and how well they enable users to perform their tasks efficiently. Thus, usability is a more inclusive construct than functionality. Usability Metrics

A variety of alternative approaches to usability evaluation have been proposed in prior work. Melody et al. (2001) identify five distinct approaches: testing, inspection, inquiry, analytical modeling, and simulation. Among these approaches, one common characteristic of usability evaluation methods is their dependence on subjective assessments in the form of user judgments. Thus, usability is not intrinsically objective in nature, but rather is closely intertwined with an evaluator’s personal interpretation of the artifact and his or her interaction with it (Agarwal & Venkatesh, 2002). Although self-reported measures are commonly used, research shows that perceived ease of use of a system is strongly correlated to subjective system usage measures, but weakly correlated to objective system usage measures (Straub, Limayem, and Karahanna-Evaristo 1995; Barnett at al. 2006).

Research has been ongoing in identifying approaches to improve online usability (Boling, 1995; Levi & Conrad, 1996; Nantel & Senecal, 2007; Palmer, 2002; Pitkow & Kehoe, 1996). Studies often focus on the download delay, success in finding a page or completing a task, or organization of the information gathered during a Web session (Pitkow & Kehoe, 1996; Nantel & Senecal, 2007). For instance, Nantel and Senecal (2007) suggest that there is a positive relationship between the time users spend waiting for webpages to download and the probability that they will complete their task on the website. Other research is based on Microsoft Usability Guidelines (MUG). Five major categories are proposed as relevant while designing websites for business: content (relevance, media use, depth/breadth, current information), ease of use (goals, structure, feedback), promotion, made-for-the medium (community, personalization, refinement), and emotion (challenge, plot, character strength, pace) (Agarwal & Venkatesh, 2002; Venkatesh & Ramesh, 2006; Venkatesh & Agarwal, 2006).

To date, the literature has conceptualized usability as either a unidimensional construct or a multidimensional construct composed of two dimensions (Table 1). Except for Palmer (2002), most research has not explored usability as a construct composed of more than two dimensions. Based on the current literature, we suggest that usability is composed of at least three dimensions: ease-of-use navigation, speed, and interactivity. Table 1 provides a summary of the research on the three main dimensions used to assess the usability construct.

TABLE 1. Usability Metrics Used in Prior Research

Ease-of-Navigation

Speed Interactivity

Agarwal & Venkatesh (2002) Y Y

Barnes & Vidgen (2001) Y Y

Lewis (1995) Y

Loiacono, Watson, & Goodhue (2002) Y Y

Nielsen (1999) Y

Palmer (2002) Y Y Y

Raquel (2001) Y Y

Tilson, Dong, Martin, & Kiele (1998) Y Y

Venkatesh & Ramesh (2006) Y Y

Venkatesh & Agarwal (2006) Y Y

Of the various factors that contribute to usability of a website, ease of navigation has been deemed important by a majority of researchers (See Table 1). Ease of navigation relates to the level of time and effort required to accomplish specific tasks (Venkatesh, 2000). Good navigation design helps users acquire more of the information they are seeking and makes the information easier to find. Thus, a key challenge in building a usable website is to develop a good navigational structure and appropriate hyperlinks. Ease-of-navigation is analogous in

essence to the ease of use in IT research (Davis, Bagozzi, & Warshaw, 1989), but it is specific to website navigation.

According to several authors download delay is also an important design criterion on the Internet (See Table 1). Speed is important since it enables users to attain their goals without too much wait. Dalleart and Kahn (1999) argued that consumers were able to separate the evaluation of waiting experiences from the evaluation of the website. However, they also show that when there is uncertainty about the waiting (as with the majority of downloads), the negative feeling generated by the waiting experience carries over to the evaluation of the website. They suggest that waiting for the homepage to download was less damaging to the website evaluation than having to wait during the interaction with the website. Their study revealed that delays shorter than expected led to better evaluations of the website. In addition, Bucklin and Sismeiro (2003) suggest that there is a negative relationship between downloading time for a web page and the probability of requesting an additional web page within a website. It has to be noted that since the focus of this study is on elements that a website can control, the objective measure of download delay will not be taken into account; only user perception will be assessed.

When users choose to use a technology, they are also choosing to interact with that technology (Orlikowski, 2000). A key capability of the Internet is its capacity to support greater interaction for users (Palmer, 2002). Interactivity can be defined as a characteristic of a computer-mediated communication in the marketplace that increases with the bidirectionality, timeliness, mutual controllability, and responsiveness of communication as perceived by consumers and firms (Yadav & Varadarajan 2005). For instance, interactivity can be used to make the website personalizable. Venkatesh & Ramesh (2006) argue that the ability to customize websites is an important design characteristic because it helps users save time and provides

information that is of greatest interest to them. Thus, as suggested by several authors (See Table 1), we suggest that website interactivity is also an underlying dimension of website usability.

Research Model and Hypotheses

Our conceptual framework is presented in Figure 1. Based on the literature review, we propose that ease-of-navigation, speed, and interactivity are three underlying dimensions of the usability construct. They are first-order factors that share a common variance that reflects a single second-order factor labeled as website usability. Thus, the following hypothesis is posited. H1: Website usability is a single second-order factor with three first-order factors, namely ease-of-navigation, speed, and interactivity.

H1a: Ease-of-navigation is an underlying factor of Website usability. H1b: Speed is an underlying factor of Website usability.

H1c: Interactivity is an underlying factor of Website usability.

FIGURE 1. Conceptual Framework

As suggested in the HCI literature, technological artifacts that are more usable are likely to change user’s cognitions, thus engender positive attitudes. Usable systems not only meet the instrumental goals of users, but also alleviate the cognitive effort associated with use (Nielsen,

Website Usability Attitude toward the Website Ease-of-Navigation Speed Interactivity

2000). Based on empirical results from the Technology Acceptance Model (Davis, 1989; Davis, Bagozzi, & Warshaw, 1989), a usable website should generate a positive attitude toward it (See Figure 1). Thus, a positive correlation should exist between the usability construct and attitude toward the website. A positive relationship would provide evidence of predictive validity of our measurement scale.

H2: There is a positive correlation between consumers’ perceived website usability and their attitude toward the website.

METHODOLOGY

Data

The sample was composed of three hundred and fifty undergraduate business students. Each student was asked to go to a specific transactional website (www.eddiebauer.com) and perform the following simple tasks: 1) Find and read about Eddie Bauer’s Children Privacy Policy; 2) Search and select a sweater that he/she would like to buy, add it to shopping cart, but do not checkout; 3) Find the Eddie Bauer 3-in-1 car seat for the children, add to the shopping cart, but do not checkout; and 4) Remove the sweater and the car seat from the shopping cart, and exit the website. Then, they were asked to complete a paper-pencil questionnaire. The questionnaire was used to assess their perception of the website’s usability and also their attitude toward the website.

Measurement Scales

Items adapted from previous research on website ease-of-navigation (Loiacono, Watson, & Goodhue, 2002; Lewis 1995; Nielsen, 1999), speed (Nielsen 1999; Palmer 2002; Loiacono,

Watson, & Goodhue, 2002), and interactivity (Agarwal & Venkatesh, 2002; Palmer 2002; Tilson, Dong, Martin, & Kiele, 1998; Barnes & Vidgen, 2001) were used to assess each dimension of website usability. The following criteria were used to select scale items: (1) items had to focus on a single dimension, not bridge two or more dimensions, a feature critical for discriminant validity, (2) they had to use, or be adaptable to, a common format for ease of administration (i.e., a seven-point Likert scale). The items used for each dimensions are presented in the Appendix. An adapted version of attitude toward the website scale (Chen & Wells, 1999) was used to assess participants’ attitude toward the website (See Appendix for specific items).

RESULTS

The objective of the analysis was to examine the measurement scale reliability and initial construct validity of the three-dimensional website usability measurement scale. First, descriptive statistics and initial reliability estimates were computed. Second, an exploratory factor analysis was performed to test the proposed structure of the measurement scale and to purify the scale by eliminating items if necessary. Third, a confirmatory factor analysis was performed with the remaining items to verify the scale and test that usability is a second-order construct with a more robust test. Finally, a regression analysis was performed to test the relationship between perceived usability and attitude toward the website.

Descriptive Statistics and Reliability Estimates

Table 2 gives univariate statistics and correlations among the website usability items. In general, participants reported a fairly strong sense of website usability. More importantly, in general correlations between items from the same dimension (See triangles in Table 2) were

higher than correlations between items from two different dimensions. The only exception was item IRC4 which showed higher correlation with items not in the interactivity dimension. The Cronbach’s alpha coefficients are 0.88, 0.94, and 0.80 for navigation, speed and interactivity respectively. Meanwhile, the Cronbach’s alpha coefficient for attitude measurement scale was 0.77.

TABLE 2. Univariate Statistics and Pearson Correlations among Usability Items*

Items** Mean*** S.D. 1 2 3 4 5 6 7 8 9 10 11 1. NAV1 5.57 1.15 2. NAV2 5.63 1.11 0.68 3. NAV3 5.34 1.24 0.67 0.69 4. NAV4 5.68 1.13 0.63 0.67 0.60 5. S1 5.67 1.29 0.50 0.49 0.44 0.45 6. S2 5.54 1.20 0.47 0.56 0.48 0.48 0.79 7. S3 5.51 1.33 0.46 0.50 0.46 0.44 0.77 0.76 8. S4 5.37 1.27 0.48 0.48 0.39 0.48 0.78 0.78 0.83 9. IRC1 4.97 1.30 0.35 0.5 0.45 0.36 0.43 0.47 0.40 0.39 10. IRC2 4.72 1.27 0.26 0.35 0.40 0.23 0.26 0.32 0.30 0.29 0.46 11. IRC3 4.95 1.19 0.26 0.30 0.37 0.23 0.26 0.31 0.32 0.30 0.50 0.63 12. IRC4 5.25 1.25 0.51 0.58 0.54 0.52 0.49 0.54 0.55 0.56 0.43 0.48 0.45 * Significance: p < 0.001.

** See the Appendix for items

Exploratory Factor Analysis

Items were then submitted to an exploratory factor analysis using principal components as means of extraction and varimax as method of rotation. Three factors, displayed in Table 3, emerged. Speed explained 53% of the variance, Ease-of-Navigation 12%, and Interactivity 10%. As recommended for exploratory work by Nunnally (1967), only loadings above 0.60 are displayed. As suspected following the correlation analysis (Table 2), IRC4 did not meet the 0.60 threshold and was removed. All three factors had eigenvalues greater than one and individually explained a significant portion of the usability variance.

TABLE 3. Exploratory Factor Analysis Results

Items (Abbreviated) Factor Loadings

NAV1 NAV2 NAV3 NAV4 S1 S2 S3 S4 IRC1 IRC2 IRC3 IRC4 0.86 0.82 0.86 0.88 0.81 0.80 0.78 0.80 0.62 0.85 0.86 0.54

Factor Labels Speed

Ease-of-Navigation Interactivity Eigenvalues

Percentage Variance Explained

6.33 52.8 1.46 12.2 1.22 10.2 Hypothesis Testing

A second-order confirmatory factor analysis (CFA) was conducted to assess the discriminant validity of the usability items and the contribution of the three dimensions to the

overall construct of usability (Hypothesis 1). We used Structural Equation Modeling (SEM) (Lisrel 8.72 software) to perform the CFA.

Following Sethi and King (1994), iterative modifications were made for each of the constructs by observing modification indices and coefficients to improve key model fit statistics. Further, as recommended by Joreskog and Sorbom (1989), only one item was altered at a time to avoid over-modification of the model. This iterative process continued until all model parameters and key fit indices met recommended criteria. Following this procedure, NAV3 was removed from navigation construct, S4 was removed from the speed construct, and IRC1 was removed from the interaction construct1.

The adequacy of the measurement model for website usability is evaluated based on model-data fit and the magnitude of first-order factor loadings on the second-order website usability factor. Two types of model-data fit indices are used to evaluate the goodness of fit of the model: absolute fit and incremental fit indices (Hair et al. 1998). First, two measures of absolute fit (which determine the degree to which the overall model predicts the observation correlation matrix) were used: chi-square statistic and the root mean square error of approximation (RMSEA). To show a good fit, the chi-square statistic needs to be non significant (i.e., no difference between actual and predicted matrices) and RMSEA values below 0.50 suggest good model fit and values between 0.50 and 0.80 suggest acceptable model fit. Second, two measures of incremental fit (which compares the proposed model to some baseline model) were used: the adjusted goodness-of-fit index (AGFI) and the non-normed fit index (NNFI). NNFI and AGFI indices greater than 0.90 suggest adequate model fit and indices greater than

0.95 suggest good model fit. Finally, loadings on the second-order factor above 0.60 are considered acceptable (Bagozzi and Yi 1988).

As illustrated in Figure 2, an excellent fit was obtained (X2= 26.18, p = 0.7128; RMSEA = 0.040, AGFI = 0.96, NNFI = 0.99). Each of the items loaded strongly on the appropriate factor, and the three factors were significantly correlated with each other. Hypotheses 1a, 1b, and 1c were supported since the paths between first-order factors and website usability were significant (p< 0.05).

FIGURE 2. Results of Second-Order Confirmatory Factor Analysis

Finally, in order to test Hypothesis 2, a linear regression was performed using website usability as independent variable and attitude as dependent variable. Results provide support for Hypothesis 2 since the coefficient (Std Beta = 0.79, and t = 19.43) was positive and significant (Table 4).

Website Usability

Ease-of-Navigation Speed Interactivity

NAV1 NAV2 NAV4 S1 S2 S3 IRC2 IRC3 0.84 0.81 0.62 0.79 0.86 0.78 0.89 0.89 0.86 0.81 0.77 0.37 0.26 0.39 0.22 0.21 0.25 0.34 0.40 Chi-Square = 26.18, df = 17, p-value = 0.7128, RMSEA = 0.040, AGFI = 0.96, NNFI = 0.99

TABLE 4. Regression Results for Predicted Path Relationships

Dependent variable Independent variables Std Beta R square t p

Attitude Usability 0.79 0.53 19.43 0.00

In addition, a final regression was performed to test the direct effects of each first-order factor on the attitude toward the website. As expected, all three factors had positive relationships with the attitude toward the site. Ease-of-navigation had the largest effect on attitude, followed by interactivity, and speed (Std Betas = 0.499, 0.305, and 0.115 respectively; all p-values < 0.05).

DISCUSSION

Summary of Findings

With the explosive growth in consumer electronic commerce (Hoffman & Novak, 2000) and Internet-enabled organizations (Straub & Watson, 2001), appropriate metrics that not only evaluates website quality but also provide managers with insights into potential problems areas is urgently needed (Agarwal & Venkatesh, 2002). This research developed and validated a multidimensional measure of website usability in a retailing context. Results suggested that the proposed website usability measurement scale has satisfactory psychometric characteristics. First, results suggested that each of the dimensions of the scale (ease-of-navigation, speed, and interactivity) is reliable. Second, results from factor analyses provided evidence of construct validity. Finally, based on the Technology Acceptance Model, the scale also showed evidence of predictive validity by being positively correlated with participants’ attitude toward the website.

To our knowledge, this research is the first to propose a reliable, valid, and parsimonious perceived website usability measurement scale. This scale is easy to administer (8 items) and it also provides specific information to researchers by being multi-dimensional. Future research should be performed to test the generalizability of the proposed measurement scale to other online contexts. For instance, it would be interesting to use the measurement scale to assess the usability of informational websites destined at consumers or business websites destined at professional buyers instead of consumers.

Implications for Practice

A website should be designed so that users can easily accomplish the task they want to accomplish, or find the information they need. Since many users are unable to complete the task they wanted to accomplish on a website (Kalczynski, Senecal, & Nantel, 2006), it is quite important for managers to be able to investigate and find which characteristics of the website are appreciated and those that are not. This scale will be useful in pinpointing specific usability dimensions of a website (ease-of-navigation, speed, and interactivity) that need to be improved. For instance, a website could be perceived by users as fast and easy to navigate, but as lacking interactivity. Thus, managers could envision solutions such as given opportunities for users to customize their experience on the website.

Limitations

The main goal of this study was to develop a short, reliable, and valid perceived website usability measurement. The first limitation of this research is its limited generalizability. Only one website was used to develop and test our proposed usability measurement scale and only one

segment of Internet users (i.e., undergraduate students) participated in our data collection. As mentioned, additional research should be conducted to validate the proposed usability scale using other websites and types of Internet users. The second limitation of this research is that we cannot be certain that the respondents experienced all the interactivity functions available on the website investigated, thus their answers to the interactivity items may be underestimated. Finally, the proposed measurement scale is based on subjective measures of usability. It would also be possible to assess website usability with objective measures such as scenario completion time, successful scenario completion rate, and time spent recovering errors (Whiteside, Bennett, & Holtzblatt, 1988). In addition, the end users connection speed can also be a factor while measuring speed dimension. Future research should measure what Internet access was available to respondents (T-1 line, dial-up, etc.) while addressing speed. Thus, similarly to the work of Straub, Limayem, and Karahanna-Evaristo (1995) and Barnett et al. (2006) it would be interesting to compare objective and subjective usability measures in future research.

APPENDIX

Construct Items Item Descriptions Sources

Ease-of-Navigation

NAV1 On this website, it is simple to accomplish the task I want to accomplish.

Nielsen (1999) NAV2 I find the website easy to use. Loiacono,Watson,&

Goodhue (2002) NAV3 It is easy to find the information I need. Lewis (1995) NAV4 It was easy to learn to use the website. Lewis (1995)

Speed

S1 The speed in which the computer provided information was fast enough.

Palmer (2002) S2 The rate at which the information was

displayed was fast enough.

Palmer (2002) S3 The website loads quickly. Loiacono,Watson,&

Goodhue (2002) S4 The pages download quickly on this website. Nielsen (1999)

Interactivity

IRC1 The website offers customization. Palmer (2002); Barnes & Vidgen (2001)

IRC2 The website can treat you as a unique person and respond to your specific needs.

Agarwal and Venkatesh (2002)

IRC3 The website provides content tailored to the individual.

Barnes and Vidgen (2001)

IRC4 The website provides adequate feedback to assess my progression when I perform a task.

Tilson, Dong, Martin, & Kiele (1998)

Attitude

ATT1 This website makes it easy for me to build a relationship with this company.

Chen & Wells (1999) ATT2 I am satisfied with the service provided by

this website.

Chen & Wells (1999) ATT3 I feel comfortable in surfing this website. Chen & Wells (1999)

REFERENCES

Agarwal, R. & Karahanna E. (2000). Time Flies When You’re Having Fun: Cognitive Absorption and Beliefs about Information Technology Usage. MIS Quarterly, 24(4), 665-694.

Agarwal, R., & Venkatesh, V. (2002). Assessing a firm’s Web presence: a heuristic evaluation procedure for the measurement of usability. Information System Research, 13, 168-186. Alty, J. L. (1992). Can we measure usability? In the proceedings of the Advanced Information

Systems. 1992 Conference, 95-106. London: Learned Information.

Bagozzi, R. P., & Yi., Yi. (1988). On the evaluation of structural equation models. Journal of Academy of Marketing Science, 16(1), 74-94.

Barnes, S. J., & Vidgen, R. (2001). An evaluation of cyber-bookshops: the WebQual method. International Journal of Electronic Commerce, 6(1), 11-30.

Barnett, T., Kellermanns, F.W., Pearson, A.W., and Pearson, R.A. (2006). Measuring Information System Usage: Replication and Extensions, Journal of Computer Information Systems, 47(2), 76-85.

Benbunan-Fich, R. (2001). Using protocol analysis to evaluate the usability of a commercial we site. Information and Management, 39, 151-163.

Bentler, P. M. (1980). Multivariate analysis with latent variables: causal modeling. Annual Review of Psychology, 31, 419-456.

Bentler, P. M., & Bonnet, D. G. (1980). Significance tests and Goodness of Fit in the analysis of covariance structures. Psychological Bulletin, 88, 588-606.

Bernard, P. J., Hammond, N. V., Morton, J., & Long, J. B. (1981). Consistency and compatibility in human-computer dialogue. International Journal of Man-Machines Studies, 15(1), 87-134.

Boling, E. (1995). Usability testing for Web sites. Learning for the global community. Seventh Annual Hypermedia Conf. Bloomington, IN.

Card, S. K., Moran, T. P., & Newell, A. (1983). The psychology of human-computer interaction. Erlbaum, Hillsdale, NJ.

Chen, Q., & Wells, W. D. (1999). Attitude toward the Site. Journal of Advertising Research, 39 (5), 27-37.

Cook, T., & Campbell, D. (1979). Quasi-experimentation: design and analysis issues for field settings. Houghton Mifflin, Boston, MA.

Davis, F. D. (1986). A technology acceptance model for empirically testing new end-user information systems: theory and results. Doctoral dissertation, MIT.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-339.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: a comparison of two theoretical models. Management Science, 35(8), 982-1003.

Dellaert, B., & Kahn, B. E. (1999). How Tolerable is Delay? Consumers’ Evaluations of Internet Web Sites after Waiting. Journal of Interactive Marketing, 13 (1), 41-54.

Straub, D., Limayem, M., and Karahanna-Evaristo, E. (1995). Measuring system usage: implications for IS theory testing. Management Science, 41(8), 1328-1342.

Goodwin, N. C. (1987). Functionality and usability. Communications of the ACM, 30(3), 229-233.

Hair, J.F., Anderson, R.E., Tatham, R.L., & Black, W.C. (1998). Multivariate Data Analysis. Upper Saddle River, NJ: Prentice Hall.

Hoffman, D. I., & Novak, T. P. (2000). How to acquire customers on the Web. Harvard Business Review, 78, 179-184.

Hill, A. (2001), Top 5 Reasons Your Customers Abandon Their Shopping Carts. Ziff Davis Smart Business for the New Economy, 14 (3), 80-85.

Joreskog, K. G., & Sorbom, D. (1989). Lisrel 7 user’s reference guide. Scientific Software Inc., Chicago, IL.

Kalczynski, P.; Senecal, S., & Nantel, J. (2006). Predicting Online Task Completion with Clickstream Complexity Measures: A Graph-Based Approach, International Journal of Electronic Commerce, 10(3), 123-143.

Klein, L. R. (1998). Evaluating the potential of interactive media through a new lens: Search versus experience goods. Journal of Business Research, 41, 195-203.

Levi, M., & Conrad, F. (1996). A heuristic evaluation of a World Wide Web prototype. Interactions, 3(4), 1-12.

Lewis, J. R. (1995). IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57-78.

Loiacono, E. T., Watson, R. T., & Goodhue, D. L. (2002). A 12-factor quality super-model from a user’s perspective. American Marketing Association: Winter Marketing Educators’ Conference 13 (Austin, Texas: American Marketing Association, 2002): 432-438.

Melody, Y. I., & Marti, A. H. (2001). The state of the art in automating usability evaluation of user interfaces. ACM Computing Survey, 33(4), 470-516.

Nantel, J. & Senecal, S. (2007, forthcoming). The Effect of Counterproductive Time on Online Task Completion, in M. Craig-Lees, G. Gregory & T. Davis (eds.). Advances in Consumer Research Asia Pacific, 7.

Nielsen, J. (1997). The need for speed. www.uscit.com/alertbox/9703a.html.

Nielsen, J. (1999). User interface directions for the web. Communications of the ACM, 42(1), 65-72).

Nielsen, J. (2000). Designing web usability. New Riders Publishing, Indianapolis, IN.

Novak, T.P., Hoffman, D. L., & Yung, Y. F. (2000). Measuring the Customer Experience in Online Environments: A Structural Modeling Approach. Marketing Science, 19(1), 22-44. Nunnally, J. (1967). Psychometric theory. McMraw-Hill, New York.

Orlikowski, W.J. (2000). Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations. Organization Science, 11( 4), 404-428.

Palmer, J. W. (2002). Web site usability, design, and performance metrics. Information Systems Research, 13(2), 151-167.

Pitkow, J., & Kehoe, C. (1996) Emerging trends in the WWW user population. Communications ACM, 39(6), 106-108.

Raquel B. F. (2001). Using protocol analysis to evaluate the usability of a commercial web site. Information & Management, 39, 151-163.

Sethi, V., & King, W. R. (1994). Development of measures to assess the extent to which an information technology application provides competitive advantage. Management Science, 40(12), 1601-1672.

Sismeiro, C. & Bucklin, R.E. (2004). Modeling Purchase Behaviour at an E-Commerce Web Site: A Task Completion Approach. Journal of Marketing Research, 41, 306-323.

Steiger, J.H., & Lind, J.C. (1980). Statistically-based tests for the number of common factors. Paper presented at the annual Spring Meeting of the Psychometric Society in Iowa City. May 30.

Straub, D. (1989). Validating instruments in MIS research. MIS Quarterly, 13(2), 147-169. Straub, D., & Watson, R. (2001). Transformational issues in researching IS and net-enabled

organizations. Information Systems Research, 12, 337-345.

Tilson, R., Dong, J., Martin, S., & Kiele, E. (1998). Factors and principles affecting the usability of four E-commerce sites. Proceedings of the 4th Conference on Human Factors and the Web, Basking Ridge, NJ.

Venkatesh, V. (2000). Determinants of perceived ease of use: integrating perceived behavioral control, computer anxiety and enjoyment into the Technology Acceptance Model. Information Systems Research, 11(4), 342-365.

Venkatesh, V., & Agarwal, R. (2006). Turning visitors into customers: A usability-centric perspective on purchase behavior in electronic channels. Management Science, 52(3), 367-382.

Venkatesh, V., & Ramesh, V. (2006). Web and wireless site usability: understanding differences and Modeling use. MIS Quarterly, 30(1), 181-206.

Yadav, M.S. & Varadarajan, R. (2005), Interactivity in the Electronic Marketplace: An Exposition of the Concept and Implications for Research. Journal of the Academy of Marketing Science, 33(4), 585-603.

Whiteside J., Bennett, J., & Holtzblatt, K. (1988). Usability engineering: our experience and evolution. M. Helander (ed.), Handbook of Human Computer Interaction, Elsevier Science Publishers, Amsterdam, 791-817.