An Iterative Usability Evaluation Procedure for Interactive Online Courses
by Laurie P. Dringus
and it should enable students to orga- nize and manage their course activities.
The design of an online course inter- face should be simple and transparent.
Dialogue design should support basic us- ability heuristics such as ease of use, ease of learning the system, aesthetic appeal, clear system feedback and error messag- es to allow quick recovery from system or user input errors (Shneiderman, 1992, Nielsen, 1993). Essentially, the inter- face, if designed effectively, should en- able students to work independently, in- crease their competence of being in the online environment, and give students control over their own learning experi- ence.
The opportunity to easily design graph- ical interfaces using hypertext and mul- timedia has resulted in an explosion of innovative applications surfacing on the Internet (Hughes, 1994). What seems to occur as educators enthusiastically bring online courses to the World Wide Web is that emphasis is placed on the aesthetic appeal of the course home page, its sub- menus and links, as well as the infusion of a variety of media such as hypertext and/or hypermedia, without a systemat- ic regard to evaluating the usability of the interface from the students’ percep- tions. How do students (i.e., navigators of the interface) perceive the online course interface as an extensible learning envi- ronment?
Assuming that improvements and changes to the course interface presenta- tion will evolve over time, it is critical that usability evaluation becomes an it- erative part of the online course design process. This article discusses usability ABSTRACT—The Internet and World
Wide Web (W3) have afforded distance learners simple links to access informa- tion. Faculty can easily bring their courses online to students by building menu sys- tems using hypertext markup language (HTML). Since online course design must include attention to effective user inter- face design, usability evaluation becomes a major component of the design pro- cess. This article focuses on the impor- tance of providing iterative usability eval- uation of online courses and strategies to ensure effective online course inter- face presentation. Some of the goals for evaluating effective online course pre- sentation are: ease of use, aesthetic ap- peal, clear feedback, error handling, con- trols for parallel and serial group com- munication, and transparency of the user interface. In addition, a procedure is de- scribed for ensuring iterative usability evaluation throughout the online course lifecycle that will help instructors rou- tinely inspect and improve the interfac- es of their online courses.
Introduction
The interface or the presentation struc- ture of an online course should provide clear organization of course information.
For example, a menu structure could cen- tralize course syllabi, lecture slides, and class announcements. The interface should present a structure for presenting student activities that require communi- cation exchange and interactivity among students. The interface should enable ef- ficient navigation for students to extract, manipulate, and share new information,
evaluation as an essential component for evaluating distance education environ- ments. A procedure is described for en- suring iterative usability evaluation throughout the online course lifecycle that will help instructors routinely in- spect and improve their online course interfaces.
Usability Evaluation of Online Courses
Educators need to continually assess their assumptions about distance educa- tion, namely that distance education pro- vides convenience, more learner control, and better quality than conventional ed- ucation (Verduin and Clark, 1991). Rou- tine summative or formative evaluations on distance education environments will further validate and perhaps increase an understanding of these assumptions about distance education. For example, routine student evaluations enable students to provide feedback in regard to their per- ceptions about course content, the de- gree to which course objectives are met, and instructor performance. Student per- ceptions about the online course presen- tation or interface also should be includ- ed in online course evaluation. Would distance learners perceive content of on- line courses differently based on the vi- sual presentation of the course? For ex- ample, if students rated overall course content favorably, but rated course struc- ture as below average, will the instructor attribute the low course structure rating to poor planning or will the instructor attribute the low rating to a navigation problem in the online course interface?
These are subtle and important issues that, unfortunately, are not typically eval- uated by students or faculty.
The dynamics of visual presentation and physical environment in an online course environment are dramatically dif- ferent from the dynamics that tradition- ally occur in a face-to-face learning envi- ronment. How does an instructor know that the course interface provides full potential for students to access and uti- lize the information that is being pre- sented? What process does the instruc- tor need to follow to assess the course
interface in terms of the interface pro- viding students with a rich and effective learning environment? One method used to assess these issues is performing a us- ability evaluation of the interface with students.
Usability evaluation can be performed formatively or summatively with simple or complex methods and is described in detail by Nielsen (1993). User evaluation studies are becoming more prevalent in the software industry. One method, heu- ristic evaluation, is a process of looking for what is good or bad about an inter- face (Nielsen, 1993). Nielsen proposed that the goal of heuristic evaluation is to find the usability problems in a user in- terface design and treat the evaluation process as a normal part of the iterative design process. Generally, heuristic eval- uation is performed by having each indi- vidual inspect the interface alone under controlled conditions. It is common prac- tice in usability evaluation to combine inspection methods with subjective eval- uation procedures such as the use of stan- dardized questionnaires to gauge users’
subjective satisfaction of the interface.
When a composite number of subjects have completed their inspections and evaluations, the data is then compiled into a formal record for future reference (Nielsen, 1993).
Design Heuristics Applied to Online Course Presentation
The emphasis on usability evaluation is related to the work of Shneiderman (1992) and Nielsen (1993). Principles for good design are being used extensively by user interface designers as a basis for systematic inspection of user interfaces.
Usability heuristics summarized by Shnei- derman (1992, pp. 72-73) and Nielsen (1993, p. 20) can be applied to evaluat- ing online course interfaces. They in- clude:
• Strive for consistency. Terminology, commands, and consistent sequences of actions should be required in simi- lar situations. Online course menus should maintain a consistent flow for navigation. Command actions for nav-
igating back and forth from submenus should be similar and recognizable from screen to screen.
• Minimize user memory load. Simple displays, consolidated multiple screens, and reliance on recognition memory can reduce short-term memory load.
Online course hyperlinks to local and global locations should be easily lo- cated and easy to navigate. For menus, use meaningful letters or words for menu items that they represent. Pro- vide roadmaps for simple and com- plex information links.
• Provide informative feedback. Mod- est to substantial system feedback is necessary for a range of actions. If online course hyperlinks are currently not operational, the system should ap- propriately inform users that the link is not currently available and they can exit or return to a previous link.
• Provide clearly marked exits. A user should be able to easily exit from a current system state either by choice or via necessity. Provide consistent display of appropriate exit commands on online course screens.
• Provide shortcuts. Reduce the num- ber of interactions and increase the pace of interaction as the user desires.
Organize course links and hyperlinks into logically similar groups. Provide an index of links that could reduce the need for navigation of multiple menu layers where links are originally positioned.
• Prevent errors. Try to prevent serious system errors from occurring. Care- fully design and test the online course interface for navigation efficiency and the degree to which escape and undo functions are recognizable. Be sure that error messages are clearly writ- ten, displayed, and suggest a solution.
• Provide help and documentation.
Short and concise information that leads the user to complete the task may be necessary via screen display or hardcopy. If users cannot intuitively follow the course interface or portions of it, then provide them with online help screens and hardcopy documen-
tation to effectively learn and use the system. Do not make assumptions about students navigating your course interface based on what you know about their computer experience level.
Other important considerations that can be applied to online course presentation include:
• Effect transparency of the interface.
Online course interfaces should pro- mote visibility of information access- es and course activities without inter- fering with the student’s desire to get the job done.
• Provide controls for parallel and serial group communication. Online course interfaces should provide instructors and students with links to tools that allow them to communicate under the following interaction scenarios: stu- dent-to-teacher, teacher-to-student, and student-to-student. The interface should assist the instructor in promot- ing a balance between individual and group experiences by providing on- line mechanisms that are needed to support individual needs (e.g., struc- tured assignment activities) and group needs (e.g. parallel communication, idea generation and sharing, and project management).
• Provide ease of use and ease of learn- ing the system. Avoid an overempha- sis on complex hyperlinks and multi- ple menu pages. Provide simple menus that are organized by breadth instead of depth. Breadth is the number of menu items, depth is the number of menu layers. Provide online and hard- copy tutorials for first time use.
• Achieve aesthetic appeal of the inter- face. Follow usability guidelines by user interface experts (Shneiderman, 1992, Nielsen, 1993) for designing text or graphical screens. Do not crowd online course interfaces with overuse of fancy fonts and multimedia links.
Do not mistake increased functional- ity for aesthetic appeal.
These usability heuristics and consid- erations and their direct application to online course interface presentation pro- vide instructors with some sample guide- lines for designing course interfaces.
These guidelines can be converted into measurable goals that can be evaluated when following a procedure for iterative usability evaluation of online courses.
Procedure For Iterative Usability Evaluation of Online Courses
The following suggested procedure pre- sents ideas for conducting formative us- ability evaluations. However, portions of the procedure can be extracted for summative usability evaluations. Instruc- tors will need to determine the point in the online course design and delivery when assessment is appropriate and the type of assessment (formative or sum- mative) to incorporate.
The suggested procedure is purposely brief; the extent of a formal procedure can be complex. This section contains a simple snapshot of some ideas of how usability evaluation can be integrated into the online course lifecycle. The proce- dure for assessing usability is meant to be iterative throughout the stages of de- velopment, evaluation, and testing.
Development
1. Develop the course interface with an understanding of the diversity of stu- dents and user experience levels.
Shneiderman(1992) emphasized the importance of assessing user profiles in terms of age, gender, physical abil- ities, education, training, motivation, goals, and personality. Also, conduct an assessment of computer experi- ence. How long has the student used computers? What types of software applications is the student familiar with? How long has the student been exposed to your institution’s distance education environment?
2. Perform a task analysis (Nielsen, 1993) of the actions and tasks students will need to do and want to accomplish with the online interface. Seek out the goals of the system and the steps necessary to perform them. Analyze and prioritize these tasks into a list
of functions that the interface should include.
3. Design the course interface by effect- ing the principles and heuristics of good design as outlined by Shneider- man (1992) and Nielsen(1993).
Evaluation
4. Recruit five to ten students to evalu- ate your course interface based on the usability heuristics presented in the section of this paper “Design Heu- ristics Applied to Online Course Pre- sentation.” Provide students with a survey that asks them to rate on a 1 to 10 scale the extent to which these principles were met. In addition, pro- vide survey items or interview ques- tions that ask students for their over- all perceptions of the course in terms of ease of use, ease of learning, aes- thetic appeal, error handling, etc.
Here are some sample questions to prompt students’ reactions about the interface: What errors typically oc- curred as you worked through the course interface? How did you dis- cover and correct errors? Describe notable successes and failures. What kinds of problems did you have with this interface? What are your ideas for improvement? What things would you like to see most changed or least changed to the interface?
5. Integrate questions relating to the organization of information, naviga- tion of course menus, and percep- tions of communicating with the in- structor and other students into student course surveys. Share the data with your administrators and faculty peers. This procedure is espe- cially good for summative course eval- uations.
Testing
6. For formative or summative evalua- tions: Conduct a usability test of your course interface. Establish user test goals and a set of tasks that five to ten students can complete at a designated time and location. Pro- vide a general description of each task to be performed, the steps in each task, and, perhaps, distinguish inter-
action with other tasks. A simple task list could be presented as fol- lows:
• Locate the course syllabus submenu
• Read today’s course announcements
• Click on the “Conferences”
hyperlink
• Select and review at least two course lecture files
• Review all submenus from the home page
• Send your instructor an e-mail message from the Activities menu There are different strategies for struc- turing the test for your students. All students can either work together at the same time, a pair of students can be test- ed at one time, or one student can be tested at one time. During testing, if pos- sible, observe each student completing the usability test and write down their notable actions and reactions to the course interface as they complete the tasks. Look for trouble areas in navigation, error re- covery, exiting submenus, selecting hy- perlinks, etc. Some common measures for usability testing of interfaces include:
subjective comments, task completion time, speed of navigation through the system, problems with titles and screen formats, the number of errors the user made, and the number or extent of re- quests for help either from online or doc- umentation sources.
7. Use the data collected from students to make direct changes to your course interface. A few iterations of making changes to the course interface, as well as user evaluation and testing, may occur during the course lifecy- cle. The process may also continue for subsequent course terms until the instructor is confident that the on- line course interface is fully functional from a usability standpoint.
Summary
This paper has addressed the impor- tance of providing iterative usability eval- uation of online courses, heuristics to ensure effective online course interface presentation, and a procedure for imple- menting iterative usability evaluation
throughout the online course lifecycle.
As more innovative applications surface on the Internet, educators need to sys- tematically evaluate the usability of their online course interfaces from their stu- dents’ perspectives. By following a rou- tine process for iterative usability evalu- ation of online course interfaces, instructors will improve their course in- terfaces based on meaningful data and, in turn, strengthen the distance learn- er’s opportunity for having a rich and an effective learning environment.
References
Hughes, K. (1994). Entering the World- Wide Web: A guide to cyberspace. (Ver- sion 6.1) Palo Alto, CA: Enterprise In- tegration Technologies.
Nielsen, J. (1993). Usability Engineering.
San Diego, CA: Academic Press.
Shneiderman, B. (1992). Designing the User Interface: Strategies for Effective Human-Computer Interaction. Second Edition. Reading, MA: Addison-Wes- ley.
Verduin, J.R. and Clark, T. (1991). Dis- tance Education: The Foundations of Ef- fective Practice. San Francisco, CA: Jos- sey-Bass.
About the Author
Laurie P. Dringus, Ph.D. is an Assis- tant Professor in the School of Comput- er and Information Sciences at Nova Southeastern University, Ft. Lauderdale, Florida. She earned her Ph.D. in Infor- mation Systems from Nova Southeastern University in 1991. She teaches gradu- ate courses in human-computer interac- tion and decision support systems. Sev- eral of her courses are accessible via the Internet. Her research interests focus on design and evaluation of human-com- puter interfaces, usability engineering and evaluation, computer-mediated commu- nications technology and user perfor- mance, group support systems, and col- laborative learning in distance education environments. E-mail Address:
