CiteSeerX — Validating the end-user computing satisfaction survey instrument

(1)

4 International Journal of Technology and Human Interaction, 2(4), 4-4, October-December 2006

ABSTRACT

This study assesses the psychometric properties of a Spanish translation of Doll and Torkzadeh’s End-User Computing Satisfaction (EUCS) survey instrument. The study provides evidence that the EUCS Spanish version can be used as a valid and reliable measure of computing satisfaction among computer users in Mexico. The study also adds support to the use of the EUCS instru- ment in the investigation of the perceptions of computer users in countries other than the United States (U.S.) and in languages other than English.

Keywords: end-user computing; end-user computing satisfaction validation; EUCS; global IS; IS success; Mexico; Spanish survey; user satisfaction

Validating the End-User

Computing Satisfaction Survey Instrument in Mexico

George E. Heilman, Winston-Salem State University, USA Jorge Brusa, Texas A&M International University, USA

INTRODUCTION

For many years, information systems (IS) researchers have been interested in the evaluation of user perceptions about the “success” of an information system. For example, Zmud (1979) provided an extensive review of studies regarding the impact of individual user differences on IS success (categorized as user performance, management of IS (MIS) usage and user satisfaction). Ives and Olsen (1984) also performed a lengthy review of research on the effect of user involvement on two classes

of IS success outcome variables: system acceptance (defined to include system usage, behavioral impact and information satisfaction) and system quality. Delone and McLean (1992) noted that while these reviews made valuable contributions to the understanding of success, both were more concerned with the investigation of independent variables than with the dependent variable—success. Delone and McLean reviewed 180 conceptual and empirical articles from the “formative period”

of IS (primarily 1981-1988) and organized the

IDEA GROUP PUBLISHING

701 E. Chocolate Avenue, Suite 200, Hershey PA 17033-1240, USA Tel: 717/533-8845; Fax 717/533-8661; URL-http://www.idea-group.com

ITJ3351

(2)

research into one of six success taxa: system quality, information quality, individual impact, organizational impact, use and user satisfaction.

They found that user satisfaction is the most widely used measure of IS success, and suggest that satisfaction is the preferred measure when system use is mandatory.

An important instrument frequently used to assess user satisfaction is the EUCS survey developed by Doll and Torkzadeh (1988). The EUCS survey consists of a single second-order factor (EUCS) composed of five first-order factors (Content, Accuracy, Format, Ease of Use and Timeliness) measured by 12 questions. Doll and Torkzadeh (1988) validated their survey instrument using a multi-step process and found that the instrument could be used across a variety of applications, hardware platforms, development modes and job positions. Shortly after the initial reporting of the EUCS survey, Etezadi- Amoli and Farhoomand (1991) raised some methodological and theoretical concerns about the instrument. However, extensive testing has established the instrument’s reliability, content validity, construct validity, internal validity, statistical conclusion validity and multigroup invariance. Examples of these tests include the studies of Adams, Nelson and Todd (1992) for voice and e-mail applications; Hendrickson, Glorfeld and Cronan (1994) for mainframe and PC applications; Simon, Grover, Teng and Whitcomb (1996) for computer-related training methods; McHaney and Cronan (1998, 2001) and McHaney, Hightower and White (1999) for computer simulation; Dowing (1999) for interactive telephone voice mail systems; Kim and McHaney (2000) for CASE tools; Aladwani (2002) for assessment of users’ overall satisfaction; Somers, Nelson and Karimi (2003) for enterprise resource planning systems; Doll, Deng, Raghunathan, Torkzadeh and Xia (2004) for decision support, database and transaction processing systems; and Abdinnour-Helm, Chaparro and Farmer (2005) for Web sites.

The EUCS survey also has been used successfully in different cultural and linguistic contexts including studies in Israel (Igbaria &

Zviran, 1996), Kuwait (Aladwani, 2002), New

Zealand (Igbaria, Zinatelli & Cavaye, 1998) and Taiwan (Igbaria, 1992; Igbaria & Zviran, 1996;

McHaney, Hightower & Pearson, 2002). Studies that have used versions of EUCS in languages other than English include translations in Chinese (McHaney et al., 2002) and Hebrew (Igbaria, 1992; Igbaria & Zviran, 1996). Little work, however, has been done in the creation of Spanish versions of any standard IS research instruments nor in the use of the EUCS instrument in Latin America. The primary goal of this investigation is to address this gap in the IS literature and examine the robustness of the EUCS survey when applied to Latin American subjects.

More precisely, this study will examine the validity and reliability of a Spanish translation of the EUCS survey instrument administered to Mexican respondents. Most end-user computing research is conducted in the US, and there is a belief that these results can be generalized to other countries (Shayo, Guthrie & Igbaria, 1999). Hofstede (1980) notes, however, that cultural characteristics might influence the satisfaction process and produce results different from those observed in America using English surveys. Therefore, the belief that instruments like EUCS can be generalized across countries many be “ill advised given the differences in culture, socio-work roles, levels of IT sophisti- cation and access to technology” (Shayo et al., 1999, p. 12). Before instruments like EUCS can be applied confidently in new cultural, coun- try or linguistic contexts, it is recommended that its universality be established through an investigation of the instrument’s validity, psychometric stability and robustness (Davis, 1989, 1994; McHaney et al., 2002; Shayo et al., 1999; Somers et al., 2003).

According to Boudreau et al., (1982) and Davis, Bagozzi and Warshaw (1989), testing of established instruments in different environments leads to a common framework of measures that helps integrate various streams of research, cumulate knowledge and assure comparability across studies. Klenke (1992) notes the importance of cross-validating measurement models and retesting surveys with different samples. Boudreau, Gefen and Straub

(3)

(2001) also emphasize the importance of the validation process. They conclude that without a solid validation of the instruments used in the collection of data, the scientific basis of quantitative research is threatened, because the conclusions and inferences reached with these instruments could be wrong or misleading.

Recognizing the importance of the validation process and the gap in the IS literature, this study will examine the validity and reliability of a Spanish translation of the EUCS survey instrument administered to Mexican subjects. Based on previous literature, we hypothesize that the EUCS survey could be a reliable instrument to measure users’ perceptions of the “success”

of an information system. More precisely, we hypothesize that the 12 questions of the survey will validly and reliably measure a user’s satisfaction with the content, accuracy, format, ease of use and timeliness of an information system and, therefore, the instrument will provide de- pendable information about user perceptions regarding the success of a system.

To test this hypothesis, this study asked computer users who live and work in northern Mexico to complete a Spanish translation of the EUCS survey. The survey leads respondents through a series of 12 questions that provides a general assessment of their level of satisfaction with the computer systems they use at work. A confirmatory factor analysis is used to determine if the collected data supports the hypothesis regarding the validity and reliability

of the survey instrument. The results of this investigation indicate that the Spanish translation of the EUCS is a valid and reliable survey when applied to Spanish-speaking computer users in northern Mexico, and researchers can use the survey instrument confidently in this environment.

THE SURVEY INSTRUMENT

The EUCS questionnaire consists of 12 questions representing a single second-order factor (EUCS) and five sub-factors (Content, Accuracy, Format, Ease of Use and Timeliness).

The Content sub-factor is measured by four questions, while the Accuracy, Format, Ease of Use and Timeliness sub-factors are measured by two questions each. Figure 1 shows a structural representation of the EUCS model.

The EUCS questionnaire was translated to Spanish by an individual fluent in both Spanish and English and reviewed by two additional individuals fluent in both English and Span- ish. The original English version of the EUCS questions and their Spanish translations are shown in Appendix A.

Responses to the questions are measured by a five-point Likert-type scale, where 1 =

“almost never,” 2 = “some of the time,” 3 =

“about half the time,” 4 = “most of the time”

and 5 = “almost always.” In the Spanish version of the survey, 1 = “casi nunca,” 2 = “algunas veces,” 3 = “la mitad de las veces,” 4 = “muchas veces” and 5 = “casi siempre.”

Figure 1. Structural model of the EUCS measure

EUCS C1

C3 C4 C2

A1 A2 F1 F1

E1 E1 T1 T1

Content

Timeliness Ease of Use Format Accuracy

(4)

THE SAMPLE

Exactly 1,200 copies of the translated questionnaire were distributed among employees at a variety of randomly selected private and public organizations in northern Mexico. No incentives were given to the organizations or the employees to complete the survey. A total of 302 surveys (25.1%) were returned.

Of the 302 returned surveys, 237 (78.5%) contained complete information and were usable. Regarding gender, 130 respondents (54.9%) were male and 107 (45.1%) were female. In terms of the organizations with which respondents were affiliated, 129 (54.4%) worked in private companies, 16 (6.8%) worked in public companies, 2 (0.8%) worked in local government, 85 (35.9%) worked in universities, 2 (0.8%) worked in high schools and 3 (1.3%) did not specify. Regarding age, 143 respondents (60.4%) were between the ages of 20 and 30;

74 (31.2%) were between 31 and 40; 18 (7.6%) were between 41 and 50; and 2 (0.8%) were older than 50.

CONSTRUCT VALIDATION

Validity refers to the extent to which an instrument measures what it is intended to

measure. Construct validation establishes that a measure appropriately operationalizes its un- derlying construct. Doll and Torkzadeh (1988) originally proposed that the EUCS instrument represented a five-factor structure. Subsequent research, however, indicates these are actually five sub-factors under a single second-order EUCS factor, as shown in Figure 1 (Chin &

Newsted, 1995; Doll et al., 1994).

Confirmatory factor analysis was used to determine if the data collected from Mexi- can subjects using the Spanish version of the EUCS instrument supports the hypothesized factor structure of the EUCS construct shown in Figure 1. Lisrel 8 was used to test the fit of the model to the collected data. Table 1 presents the correlation matrix on the 12 questionnaire items used in the analysis.

For purposes of scaling and statistical identification, the factor loading of one indicator in each sub-factor is set to 1 and the variance of the second-order UECS factor is set to 1 (Byrne, 1998). The following sections provide a textual description of analysis results. Figure 2 presents a diagrammatic summary of the results, showing factor loadings with significance levels in parentheses.

C1 C2 C3 C4 A1 A2 F1 F2 E1 E2 T1 T2

C1 1.000 C2 0.721 1.000

C3 0.705 .753 1.000

C4 0.646 0.738 0.717 1.000 A1 0.753 0.747 0.671 0.651 1.000 A2 0.643 0.782 0.739 0.732 0.713 1.000 F1 0.755 0.723 0.749 0.690 0.690 0.672 1.000 F2 0.703 0.799 0.756 0.723 0.693 0.779 0.741 1.000 E1 0.638 0.766 0.705 0.666 0.668 0.651 0.678 0.748 1.000 E2 0.674 0.767 0.746 0.712 0.672 0.680 0.755 0.773 0.874 1.000

T1 0.653 0.694 .680 0.686 0.716 0.709 0.651 0.674 0.682 0.666 1.000

T2 0.671 0.737 0.718 0.793 0.724 0.752 0.686 0.701 0.716 0.743 0.770 1.000

Table 1. EUCS correlation matrix

(5)

International Journal of Technology and Human Interaction, 2(4), 4-4, October-December 2006

Reliability

Reliability refers to the degree to which scores are free from measurement errors. It is a necessary but not sufficient condition for instrument validity. One method commonly used to assess internal-consistency reliability is coefficient alpha, which is based on the notion of splitting a measure into as many parts as the number of items. Alpha, then, is the average of all possible split-half reliability coefficients for the measure (Pedazur & Schmelkin, 1991). Coef- ficient alphas greater than .70 indicate reliable constructs (Fornell & Larker, 1981). The alphas for the EUCS sub-factors are: Content = .91, Accuracy = .83, Format = .85, Timeliness = .87, and Ease of Use = .93. Coefficient alpha for the overall instrument is .97, which is well above the recommended threshold and compares favorably with the .92 reported by Doll and Torkzadeh (1988) in their initial study. The conclusion is that the second-order EUCS construct and its five first-order sub-factors are reliable.

Convergent Validity

Convergent validity refers to the con- vergence among different methods designed to measure the same construct (Pedhazur &

Schmelkin, 1991). One technique for evaluating convergent validity views each item in a construct as a different approach to measuring the construct. If t-tests for the loadings of all

the indicators measuring a single construct are statistically significant, all indicators are effectively measuring the same construct, and the construct exhibits convergent validity (Anderson & Gerbing, 1988).

Table 2 shows the indicator loadings for each construct, along with their corresponding t-values. Indicant loadings that were fixed during model analysis have a loading of 1. T- values greater than 3.29 are significant at the .001 level. The loadings for all freed indicants are significant at the .001 level. The conclusion is that EUCS sub-factors exhibit convergent validity.

Discriminant Validity

Discriminant validity implies that one construct can be empirically differentiated from other constructs that may be similar (Kerlinger, 1986). Discrimination may be demonstrated with a chi-square difference test among all possible pairs of constructs, in this case the five sub-factors that make up EUCS (Ahire, Golhar

& Waller, 1996).

Two confirmatory factor analyses (CFAs) are run for each selected pair of sub-factors. In the first CFA, correlation is allowed between the sub-factors. In the second CFA, the correlation between the pair is fixed to 1.00, creating a difference with 1 degree of freedom between the models. If the chi-squares from the two

\ Indicant

Construct \ 1 2 3 4

Content 0.915

(16.81) 0.961

(18.52) 0.941

(17.76) 1.000

*

Accuracy 0.967

(16.62) 1.000

*

Format 0.958

(17.91) 1.000

* Timeliness 0.939

(18.24) 1.000

* Ease of Use 0.962

(25.07) 1.000

*

Table 2. Analysis of convergent validity

(6)

tests are statistically significantly different, the constructs exhibit discriminant validity.

The chi-square critical values for 1 degree of freedom are 3.84 at the .05 significance level, 6.63 at the .01 significance level and 7.88 at the .005 significance level. Table 3 presents the results of the difference tests, showing the differences in chi-squared values between pairs and their corresponding p-values. All differences were significant at the .05 level or lower. The conclusion is that EUCS sub-factors exhibit discriminant validity.

Structural Analysis

Table 4 presents the goodness of fit indices for the EUCS structural model, along with guidelines for evaluating the fit values (Browne

& Cudek, 1993; Hair, Anderson, Tatham &

Black, 1992; Pedhazur & Schmelkin, 1991;

Sharma, 1996). Though always reported, the chi-square test is not considered to be practically meaningful and is typically discounted in favor of other methods for evaluating fit of the model to the data (Bearden et al., 1982). All indices except chi-square yield acceptable values. The

Accuracy Format Timeliness Ease of Use

Content 5.80 (p<.05) 5.55 (p<.05) 6.74 (p<.01) 6.01 (p<.05) Accuracy 6.00 (p<.05) 7.06 (p<.01) 11.24 (p<.005)

Format 9.54 (p<.005) 5.36 (p<.05)

Timeliness 7.53 (p<.01)

Table 3. Analysis of discrminant validity

Goodness of Fit Indicator Value Recommended

Value Conclusion chi-square (49 d.f.) 153.99 (p < .01) p > .05 Poor

normed chi-square (chi-square/d.f.) 3.14 < 5 Good

GFI .91 > .90 Good

AGFI .85 > .80 Good

NFI .95 > .90 Good

NNFI .95 > .90 Good

CFI .97 > .90 Good

RMR .027 < .20 Good

Table 4. Analysis of model fit

Content Accuracy Format Timeliness Ease of Use

EUCS .897

(17.62) .850

(16.20) .874

(16.94) .850

(16.19) .852

(16.27)

Table 5. Analysis of structural loadings on EUCS

(7)

conclusion is that the model provides a good fit for the data.

Since the model fit is acceptable, the loadings of the sub-factors on the second-order factor EUCS can be evaluated. Table 5 presents the structural loadings and their corresponding t-values.

All sub-factor loadings are significant at the .001 level. These findings indicate that our model is a valid representation of the EUCS construct, and supports prior research character- izing EUCS as a second-order factor with five first-order sub-factors.

DISCUSSION

The assessment of IS “success” has been a long-standing and ongoing concern among IS researchers. One of the most commonly used qualitative success measures found in the IS research literature is user perceptions of satisfaction with a system. An important and frequently used survey instrument designed to evaluate relative levels of user satisfaction is the EUCS questionnaire developed by Doll and Torkzadeh (1988).

The EUCS construct is a second-order factor composed of five first-order sub-factors (Content, Accuracy, Format, Ease of Use, and Timeliness). While the second-order EUCS factor gives an overall assessment of the general level of user satisfaction with a system, an analysis of the sub-factors may help investiga- tors identify specific system features that could be tuned to improve overall satisfaction. For example, the overall satisfaction score may be relatively high (> 3) but the Accuracy score may be very much lower than the scores of the other sub-factors. By improving the system’s Accuracy, it could be expected that the level of overall user satisfaction would rise.

Just because a survey instrument such as EUCS has been shown to successfully measure satisfaction in a given task environment with specific operational (hardware/software) param- eters does not mean that the instrument can be used successfully in every environment. For this reason, IS researchers are interested in testing and retesting established instruments, such as EUCS, with different samples in different environments. This cross-validation process helps Figure 2. Structural model of the EUCS measure

End-User Computing Satisfaction C1

C3 C4 C2

A1 A2 F2 F1

E1 E2 T1 T2

Content

Timeliness Ease of Use

Format Accuracy .915

1 .958

(p<.001).850 .852 (p<.001) 1

.874 (p<.001)

1 .939 1 .967 .941 .961

1

.962

(p<.001).850 .897 (p<.001)

(8)

to identify a common framework of measures that can be used to integrate various streams of research, cumulate knowledge, provide comparability across studies and assure that conclusions and inferences reached with these instruments are not wrong or misleading.

The purpose of this study is to extend the generalizability of the EUCS instrument by assessing the psychometric properties of a Spanish translation of the EUCS survey administered to subjects living and working in Mexico. An assessment of the survey instrument’s reliability using coefficient alpha supports the conclusion that the second-order EUCS construct and each of its five first-order sub-factors are reliable. An assessment of t-tests on indicant loadings supports the conclusion that EUCS sub-factors exhibit convergent validity. An assessment of chi-square difference tests among construct sub-factors supports the conclusion that the EUCS sub-factors exhibit discriminant validity. An assessment of the fit indices supports the conclusion that the proposed EUCS model provides a good fit to the Mexican data.

And finally, the significant loadings of the five sub-factors on the EUCS variable provides support for a model with second-order model construct (EUCS) composed of five first-order sub-factors.

The findings indicate that the Spanish translation of the EUCS survey is a valid and reliable instrument that can be used confidently by researchers in investigations involving Spanish-speaking computer users in northern Mexico. These results add support to the greater generalizability of the EUCS instrument and its robustness as a valid measure of computing satisfaction and a surrogate for system success in a variety of cultural and linguistic settings.

The results, however, do not imply that EUCS will be valid in all cultural and linguistic contexts. Even this particular translation may not be valid in all Spanish-speaking countries.

For example, the Microsoft Word language function lists 20 variations for Spanish translations. Our translation uses “Mexican” Spanish and was administered only in northern Mexico.

Generalizability of EUCS will be enhanced by expanding validation testing throughout Mexico and into other Spanish-speaking countries.

CONCLUSION

The purpose of this study is to evaluate the reliability and validity of a Spanish version of the EUCS survey instrument. The EUCS questionnaire is commonly used as a measure of the relative level of user satisfaction with a system and as a surrogate for measuring system success. It has been proven reliable and valid in a number of previous studies when applied to specific applications as well as broader cat- egories of information systems.

Before translated versions of a standard- ized English survey instrument such as EUCS can be used confidently outside the US or in cross-cultural research, it is important to ensure that the translations retain their psychometric properties. Chinese and Hebrew translations of the EUCS have already been validated. This study indicates that a Spanish translation of the EUCS also retains its psychometric properties when using data collected from a sample of Mexican computer users. Additionally, the study adds support to the representation of EUCS as a second-order construct with five sub-factors:

Content, Accuracy, Format, Ease of Use and Timeliness.

The study supports the greater generalizability of the EUCS instrument and its robustness in a variety of cultural and linguistic contexts. The findings indicate that a Spanish translation of the EUCS survey instrument can be used in investigations of Spanish-speaking computer users in northern Mexico, and adds support to the EUCS as a valid and reliable measure of satisfaction that may be used to evaluate system success internationally.

ACKNOwLEDGMENT

The authors would like to thank the review- ers for their comments and Associate Editor Anthony Bryant for his help in improving the quality of the paper.

(9)

REFERENCES

Abdinnour-Helm, S. F., Chaparro, B. S., &

Farmer, S. A. (2005). Using the End-User Computing Satisfaction (EUCS) instrument to measure satisfaction with a Web site. Decision Sciences, 26(2), 349-364.

Adams, D. A., Nelson, R. R., & Todd, P. A.

(1992). Perceived usefulness, ease of use, and usage of information technol- ogy: A replication. MIS Quarterly, 16(2), 227-247.

Ahire, S. L., Golhar D. Y., & Waller, M. A.

(1996). Development and validation of TQM implementation constructs. Deci- sion Sciences, 27(1), 23-56.

Aladwani, A. M. (2002). Organizational ac- tions, computer attitudes, and end-user satisfaction in public organizations: An empirical study. Journal of End User Computing, 14(1), 42-49.

Anderson, J. C., & Gerbing, D. W. (1988).

Structural equation modeling in practice:

A review and recommended two-step Approach. Psychological Bulletin, 103, 411-423.

Bearden, W. O., Sharma S., & Teel, J. E. (1982).

Sample size effects of chi-square and other statistics used in Evaluating causal models. Journal of Marketing Research, 19, 425-430.

Boudreau, M., Gefen, D., & Straub, D.W.

(2001). Validation in information systems research: A state-of-the-art assessment.

MIS Quarterly, 25(1), 1-16.

Browne, M. W. & Cudeck, R. (1993). Alternate ways of assessing model fit. In K. A. Bol- len & J. S. Long (Eds.), Testing structural equation models (pp. 139-154). Newbury Park: Sage Publications.

Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Mahwah: Lawrence Erlbaum Associates.

Chin, W. W., & Newsted, R. N. (1995). The importance of specification in causal modeling: The case of end-user comput- ing satisfaction. Information Systems

Research, 6(1), 73-81.

Davis, F. D., Bagozzi, R. P. & Warshaw, P. R.

(1989). User acceptance of information technology: A comparison of two theoreti- cal models. Management Science, 35(8), 982-1003.

Davis, F. W. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340.

Delone, W. H., & McLean, E. R. (1992).

Information success: The quest for the dependent variable. Information Systems Research, 3(1), 60-95.

Doll, W. J., Deng, X., Raghunathan, T. S., Torkzadeh, G., & Xia, W. (2004). The meaning and measurement of user satisfaction: A multigroup invariance analysis of the end-user computing satisfaction instrument. Journal of Management In- formation Systems, 21(1), 228-262.

Doll, W. J. & Torkzadeh, G. (1988). The measurement of end-user computing satisfac- tion. MIS Quarterly, 12(2), 259-274.

Doll, W. J., Xia, W., & Torkzadeh, G. (1994). A confirmatory factor analysis of the end- user computing satisfaction instrument.

MIS Quarterly, 18(4), 453-461.

Dowing, C. E. (1999). System usage behav- iour as a proxy for user satisfaction: An empirical investigation. Information &

Management, 35, 203-216.

Etezadi-Amoli, J., & Farhoomand, A.F. (1991).

On end-user computing satisfaction. MIS Quarterly, 15(1), 1-4.

Fornell, C., & Larker, D. F. (1981). Evaluat- ing structural equation models with unobserved variables and measurement error. Journal of Marketing Research, 18, 39-51.

Hair, J. F. Jr., Anderson, R. E., Tatham, R. L.,

& Black, W. C. (1992). Multivariate data analysis with readings (3^rd ed.). New York: Macmillan Publishing Company.

Hendrickson, A. R., Glorfeld, L. W., & Cronan, T. P. (1994). On the repeated test-retest reliability of the end-user computing satis- faction instrument: A comment. Decision

(10)

Sciences, 25(4), 665-667.

Hofstede, G. (1980). Motivation, leadership and organization: Do American theories apply abroad. Organizational Dynamics, 9(1), 42-63.

Igbaria, M. (1992). An examination of micro- computer usage in Taiwan. Information

& Management, 22(1), 9-28.

Igbaria, M., Zinatelli, N., & Cavaye, A. L. M.

(1998). Analysis of information technology success in small firms in New Zealand.

International Journal of Information Management, 18(2), 103-119.

Igbaria, M., & Zviran, M. (1996). Comparison of end-user computing characteristics in the U.S., Israel and Taiwan. Information

& Management, 30(1), 1-13.

Ives, B. & Olson, M. H. (1984). User involvement and MIS success: A review of research. Management Science, 30(5), 586-603.

Kerlinger, F. N. (1986). Foundations of be- havioral research (3rd ed.). Fort Worth:

Harcourt Brace Jovanovich College Publishers.

Kim S., & McHaney, R. (2000). Validation of the end-user computing satisfaction instrument in case tool environments.

Journal of Computer Information Sys- tems, 41(1), 49-55.

Klenke, K. (1992). Construct measurement in management information systems: A review and critique of user satisfaction and user involvement instruments. INFOR, 30(4), 235-348.

McHaney, R., & Cronan, T. P. (1998). Computer simulation success: On the use of the end- user computing satisfaction instrument.

Decision Sciences, 29(2), 525-536.

McHaney, R., & Cronan, T. P. (2001). A comparison of surrogate success measures

in on-going representational decision support systems: An extension of simu- lation technology. Journal of End User Computing, 13(2), 15-25.

McHaney, R., Hightower, R., & Pearson, J.

(2002). A validation of the end-user computing satisfaction instrument in Taiwan. Information & Management, 39(6), 503-511.

McHaney, R., Hightower, R., & White, D.

(1999). EUCS test-retest reliability in representational model decision support systems. Information & Management, 36(2), 109-119.

Pedhazur, E. J. & Schmelkin, L. P. (1991).

Measurement, design, and analysis: An integrated approach. Hillside: Lawrence Erlbaum Associates.

Sharma, S. (1996). Applied Multivariate Techniques. New York, NY: John Wiley and Sons.

Shayo, C., Guthrie, R. & Igbaria, M. (1999).

Exploring the measurement of end user computing success. Journal of End User Computing, 11(1), 5-14.

Simon, S. J., Grover, V., Teng, J. T. C., &

Whitcomb, K. (1996). The relationship of information training methods and cognitive ability to end-user satisfaction, comprehension, and skill transfer:

A longitudinal field study. Information Systems Research, 7(2), 466-490.

Somers, T. M., Nelson, K., & Karimi, J. (2003).

Confirmatory factor Analysis of the end- user computing satisfaction instrument:

Replication within an ERP domain. Deci- sion Sciences, 34(3), 595-621.

Zmud, R. W. (1979). Individual differences and MIS success: A review of the empirical literature. Management Science, 25(10), 966-979.

(11)

George Heilman is assistant professor of management information systems at Winston-Salem State University. He has a PhD in computer information systems from the University of Arkansas.

His research interests include technology in multicultural environments, technology in entrepre- neurial organizations and small business computerization.

Jorge Brusa is assistant professor at Texas A&M International University. He has an undergradu- ate degree from the Universidad del Litoral (Argentina), a master’s degree from the University of Arkansas at Little Rock, and doctoral degrees from the Universidad de Belgrano (Argentina) and the University of Arkansas. His research interests include the study of new technologies in international markets and the effect of technology in financial markets. His studies have been published by the Journal of Business, Finance and Accounting, the Quarterly Review of Finance and Accounting, the International Review of Financial Studies, and the Journal of Global Infor- mation Technology Management.

APPENDIx A. EUCS QUESTIONS

Content

C1 Does the System provide the precise information you need?

¿El sistema provee la información que usted necesita?

C2 Does the information content meet your needs?

¿Es la información provista por el sistema lo que usted necesita?

C3 Does the system provide reports that seem to be just about exactly what you need?

¿Los reportes del sistema son lo que usted necesita?

C4 Does the system provide sufficient information?

¿Piensa usted que el sistema provee suficiente información?

Accuracy

A1 Is the system accurate?

¿Es el sistema preciso?

A2 Are you satisfied with the accuracy of the system?

¿Esta usted satisfecho con la precisión del sistema?

Format

F1 Do you think the output is presented in a useful format?

¿Es el resultado del sistema presentado en una forma útil?

F2 Is the information clear?

¿Es la información del sistema clara?

Ease of Use

E1 Is the system user friendly?

¿Para los usuarios, es el sistema fácil de usar?

E2 Is the system easy to use?

¿Es el sistema fácil de usar?

Timeliness

T1 Do you get the information you need in time?

¿Recibe la información que necesita en tiempo?

T2 Does the system provide up-to-date information?

¿Provee el sistema información actualizada?

(12)

Discussion on Paper 5:

Localized End-User Computing Satisfaction Survey is Indeed Valid and Reliable for Spanish- Speaking Users, But Could Be More “Amigable”

Eduardo H. Clark, Freescale Semiconductor, Inc., USA

I appreciate the opportunity to write a com- mentary regarding the Heilman-Brusa paper. I focus my comments on what I understand best as a native Spanish-speaker, former localization coordinator and professional communicator since the mid-1980s.

The translation of the survey questions on the paper is reasonably accurate and easily understood. In fact, the words chosen are quite portable across the whole Spanish-speaking spectrum, even though the intended audience was just that of Northern Mexico. However, there is some room for improvement: The translation is quite literal and could have benefited from a more Spanish-like syntax that would sound more natural to native Spanish speakers. Also, I did find an error in the translation—although it is minor. The translation reads “en tiempo,”

which means “in time.” It should have been “a tiempo,” which is the correct translation of “on time,” as the English version reads.

In the English version of the survey, you find two closely related questions about the usability of the system. One has to do with the ease of use per se and the other with user friendliness.

The way I understand usability, ease of use has to do with simplicity, or lack of complicated procedures; and user friendliness has to do with intuitive use of computer interface resources, such as menus, settings and so forth. Luckily,

the translator did not attempt to translate user friendly (amigable al usuario) in a literal way, because there is no such expression commonly used in Spanish regarding computer interfaces.

Translating idioms and jargon frequently results in trouble and miscommunication.

A computer-based system is generally regarded as user friendly if you do not need a manual to execute simple tasks. The Macintosh OS X operating system comes to mind as an example of user-friendliness; command-line UNIX is an example of the opposite. An example regarding ease of use is a Web browser; it has the simple function of finding Internet-based resources and assisting with interpretation and navigation.

In the translated version of the survey, the difference between ease of use and user friendliness is even more subtle than in the English version. The user-friendly related question mentions the user (usuario). The ease-of-use question does not, even though, regardless of language, ease of use and user-friendliness mean something only in the context of the user’s experience with the system. These minor issues aside, the paper is a successful attempt to port English-based computer system experience research to that of non-English-speaking user experiences in terms that these users can understand in their own language and context.

(13)

6 International Journal of Technology and Human Interaction, 2(4), -6, October-December 2006

Eduardo H. Clark holds a Bachelor of Science degree in industrial engineering and a Master of Science degree in public administration. He is currently employed by Freescale Semiconductor in Austin, Texas, where he is a senior staff member of the technical publications team of the Wire- less and Mobile Systems Group. He is a senior member of the IEEE, immediate past-president of the IEEE Professional Communication Society (PCS), and board member of PCS since 1999.

Mr. Clark is co-author of the English-to-Spanish Computer and Internet Dictionary, 2004, ISBN 1581124996 and presented the paper “Producing Multilingual Online Documentation Using Contract Developers” at the IEEE PCS International Conference in New Orleans, Louisiana (IPCC 99). He was the coordinator of documentation localization for VTEL Corporation and Pervasive Software, Inc. between 1997 and 1999.