Human Computer
Interface
Evaluation
K. Bouzoubaa
Introduction
w Why and when to evaluate?
Analyse des besoins
Conception 3. Approche participative Prototypages 2. Modèle Cognitif de l’utilisateur Tests d’acceptation Tests du système Espace IHM 1. Introduction 4. Types d’Interaction 5. Interaction guidelines 6. Web Interaction guidelines
Conception Logicielle Codage Espace logiciel 7. Conception 8. Evaluation Réalisation et tests logiciels
Software Engineering Evaluation
« Software Quality Assurance »
w Software performance
w Hardware performance and cost
w Maintenance
w Safety
w Reliability
w Interoperability
HCI Evaluation Objectives
w
Why evaluate ...?
w
To answer
usability
questions such as:
– Would the operator be able to quickly answer emergency calls?
– Was this ticket machine sufficiently simplified to allow a novice user to easily use it on the first try?
– Is the user familiar with the terminology used in the interface?
– If the user invokes this command by mistake, will it be easy for him to correct or leave the system?
Types of Evaluation
Formative
Evaluation
Made during
design process and
development process
With a incomplete version of the software
Summative
Evaluation
Made at the end
With a final version of the software
Evaluation Techniques
Analytical (HE)
G.O.M.S.
« Goals, Operators, Methods and Selection rules »
Cognitive Walkthrough
è Heuristic Evaluation
o From previous lectures
o Nielsen 10 principles o Others …
Empirical
(User Testing)
è Shneiderman Observation Acceptability tests QuestionnairesHeuristics From Earlier Lectures
w User-centered design
– Know your users
– Understand their tasks
w Memory
– Use chunking to simplify information presentation – Minimize working memory
w Color guidelines
– Don’t depend solely on color distinctions (color blindness) – Avoid red on blue text (chromatic aberration)
– Avoid small blue details
w Norman’s principles of direct manipulation
– Affordance – Visibility – Feedback
Heuristic Evaluation
w
Performed by expert(s)
w
Steps
–
Inspect UI thorough
–
Compare UI against heuristics
–
List usability problems
• Explain & justify each problem with
heuristics
HE: Nielsen 10 principles
1. Match the real world2. Consistency & standards
3. Help & documentation
4. User control & freedom
5. Visibility of system status
6. Flexibility & efficiency
7. Error prevention
8. Recognition, not recall
9. Error reporting, diagnosis, and recovery
1.
Match the Real World
“Speak the user language”
w Technical jargon avoided (Type = écrire
or Type = int, float)
w Use common words, not jargon
– But use domain-specific terms where appropriate
w Don’t put limits on user defined names
(DOS: 12345678.ext)
w Allow aliases/synonyms in command
languages (remove, del, erase)
– Only 7-18% = probability 2 users name the same object with the same name
w Metaphors are useful but may mislead
– Example: Trash in Lotus organizer (not natural)
2. Consistency and Standards
w
Principle of Least Surprise
–
Similar things should look and act similar
–
Different things should look different
–
Example:
share price, stock price, stock quotew
Other properties
–
Size, location, color, wording, ordering, …
3. Help and Documentation
w Users don’t read manuals
– Prefer to spend time working toward their task goals, not learning about your system
w But manuals and online help are vital
– Usually when user is frustrated or in crisis
w Help should be:
– Searchable
– Context-sensitive
– Task-oriented
– Concrete
4. User Control and Freedom
w Provide undo
w Long operations should be cancelable
5. Visibility of System Status
“Feedback”
w Keep user informed of system state
– Cursor change
– Selection highlight
– Status bar
– Don’t overdo it…
– Example: Studios MGM (indication du temps d’attente, allées en forme de labyrinthe, téléviseurs et illusion de progression)
w Response time
– < 0.1 s: seems instantaneous
– 0.1-1 s: user notices, but no feedback needed
– 1-5 s: display busy cursor
6. Flexibility and Efficiency
w
Provide easily-learned shortcuts for frequent
operations
– Keyboard accelerators – Command abbreviations – Bookmarks – History Fast withdraw 100 DH 200 DH … 2000 DH7. Error Prevention
w Selection is less error-prone than typing
w Some techniques:
– Text field (type and format: DD/MM/YYYY)
– Combo Box
– Anticipate {} in Word Processor
8. Recognition, Not Recall
“Minimize Memory Load”
w
Use menus, not command languages
w
Use combo boxes, not textboxes
w
Use generic commands where possible
(
Open, Save, Copy Paste)
9. Error Reporting, Diagnosis, Recovery
w
Be precise; restate user’s input
–
Not “Cannot open file”, but “Cannot open
file named paper.doc”
w
Give constructive help
–
why error occurred and how to fix it
w
Be polite and non blaming
–
Not “fatal error”, not “illegal”
w
Hide technical details (stack trace) until
10. Aesthetic and Minimalist Design
w
“
Less is More”
10. Aesthetic and Minimalist Design
w
Good graphic design
–
Few, well-chosen colors and fonts
–
Group with whitespace
–
Align controls sensibly
w
Use concise language
HE: Nielsen 10 principles
Meet expectations
1. Match the real world
2. Consistency & standards
3. Help & documentation
User is the boss
4. User control & freedom
5. Visibility of system status
6. Flexibility & efficiency
Handle errors
7. Error prevention
8. Recognition, not recall
9. Error reporting, diagnosis, and recovery
Keep it simple
Example
Example
1. Shopping cart icon is not balanced with its background white space: minimalist design
2. Good: user is greeted by name: Visibility of system status
3. Red is used both for help messages and for error messages: Consistency, Match real world
4. “There is a problem with your order”, but no explanation or suggestions for resolution: Error reporting
5. ExtPrice and UnitPrice are strange labels: Match real world
Example
7. "Click here“ is unnecessary: Aesthetic & minimalist design
8. No “Continue shopping" button: User control & freedom 9. Recalculate is very close to Clear Cart: Error prevention
10. “Check Out” button doesn’t look like other buttons:
Consistency
11. Uses “Cart Title” and “Cart Name” for the same concept: Consistency
12. Must recall and type in cart title to load: Recognition not recall, Error prevention, Flexibility & efficiency
Heuristic Evaluation
Advantages
o Low costso Intuitive technique
o Simple to apply
o Usable very early in
the design
Disadvantages
o Focuses on problems not on solutions o Same design à still different problemsHow To Do Heuristic Evaluation
w
Justify every problem with a heuristic
– Too many choices on the home page “Minimalist design”
w
List every problem
– Even if an interface element has multiple problems
w
Go through the interface at least twice
– Once to get the feel of the system
– Again to focus on particular interface elements
w
Don’t limit yourself to the 10 Nielsen’s
heuristics
– We’ve seen others : affordance, visibility, color principles, etc.
Writing Good Heuristic Evaluations
w Heuristic evaluations must communicate well to
developers and managers
w Include positive comments as well as criticisms
– “Good: Toolbar icons are simple, with good contrast and few colors (Minimalist design)”
w Be tactful
– Not “the menu organization is a complete mess”
– Better “menus are not organized by function”
w Be specific
– Not “text is unreachable”
– Better “text is too small, and has poor contrast (black text on dark green background”
Suggested Report Format
w
What to include
– Problem – Heuristic – Description – Severity– Recommendation (if any)
Evaluation Techniques
Analytical
G.O.M.S.
« Goals, Operators, Methods and Selection rules »
Cognitive Walkthrough
è Heuristic Evaluation
o Norman’s principles
ü Affordance ü Visibility ü Feedback o Nielsen 10 principles o Others …
Empirical
(User Testing)
è Shneiderman Observation Acceptability tests QuestionnairesUsability Lab
Salle de test Sa lle d 'o b se rv ati o n Sa lle d 'o b se rv ati o n Équipe de développeurs Spécialistes des facteurs humains Caméra dirigéevers la documentation Caméra dirigée
sur l'utilisateur Utilisateur 2 ou observateur Caméra dirigée sur l'ordinateur Murs à vue unidirectionnelle Enregistrement des évènements Vue de chaque caméra Station de l'expérimentateur Écran géant (duplication de l'écran usager) Contrôle de l'édition et du mixage
User Testing
User Testing
è
Observation
è
Acceptability tests
User Testing
•
Observation
♦ Video Saving ♦ Loud thinking ♦ Passive Observation ♦ Ethnographic studiesUser Testing
•
Acceptability Tests
♦ Assumption: measurable usability objectives
(Shneiderman, 1998) Learning time Performance Error Rate Retention time Subjective Satisfaction
User Testing
•
Acceptability Tests…
♦ Example 1
Les sujets seront 35 secrétaires n’ayant aucune expérience de
traitement de texte mais une bonne expérience de frappe au clavier (35 à 50 mots/min). Une période de 45 min sera consacrée à de l’entraînement avant l’expérience. Au moins 30 des 35 secrétaires devraient être capables de compléter correctement en 30 min 80% des tâches de frappe et d’édition.
User Testing
•
Acceptability Tests
…
♦
Example 2
Après 4 demi-journées d’utilisation régulière du système, 25 des 35 secrétaires devraient être capables de compléter en 20 min les tâches de niveau plus avancé du second « benchmark » en faisant moins de 6 erreurs.
User Testing
•
Questionnaires
QUIS: The Questionnaire for User Interaction Satisfaction
http://www.lap.umd.edu/QUIS/index.html
w The purpose of the questionnaire is to:
– guide in the design or redesign of systems
– give managers a tool for assessing potential areas of system improvement
– provide researchers with a validated instrument for conducting comparative evaluations, and
Hints for better evaluation
w
Use multiple evaluators
–
Different evaluators find different problems
–
The more the better
–
Nielsen recommends 3-5 evaluators
w
Alternate HE with UT
–
Each method finds different problems
Analyse des
besoins Conception Conception Logicielle Codage Réalisation et tests logiciels Tests d’acceptationTests du système
Espace Logiciel (Informaticien)
Prototype
Papier NumériquePrototype IHM / NoyauSéparation
Espace IHM (Informaticien + User)
Évaluation Utilisabilité Guidelines IUG + Web Évaluation Utilisabilité + W3C So ftw ar e d ev . c yc le
Techniques for Web Content
Accessibility Guidelines
Introduction
w Persons attempting to access the Web may:
– not be able to see, hear, move, or may not be able to process some types of information.
– not have, or be able to use, a mouse or even a keyboard
– have a text-only screen, a small screen, or a slow Internet connection
– not speak or understand fluently the language in which the document is written
– be in a situation where their eyes, ears, or hands are busy or interfered with.
– have an early version of a browser, a voice browser, or an operating system other than Windows
w Apply Web Content Accessibility Guidelines (WCAG) will:
41
Introduction
w Apply Web Content Accessibility Guidelines (WCAG) will:
42
Introduction
w Apply Web Content Accessibility Guidelines (WCAG) will:
About WCAG
w
W3C Guidelines:
– 14 guidelines.
– Each guideline has multiple checkpoints.
– Each checkpoint has a priority level (1 - 3).
– Each checkpoint has techniques for implementation.
w
WAVE toolbar:
Guideline 1:
Equivalent text to auditory and visual content
Avoid using images as bullets:
Deprecated example:<IMG src="star.gif" alt="* ">
Use alt when using image as the content of a link:
Images available Images not available
if providing link text, put a space in alt:
Example: <A href="welcome.html"><IMG src="welcome.html" alt=" " > Welcome </A>
Use short text equivalent for images:
When image contains important information use long description:
Image containing important Information longdesc supported by firefox
<IMG src="97sales.gif" alt="Sales for 1997“ longdesc="sales97.html">
For user agent that does not support longdesc attribute:
Sales for 1997 [D]
<IMG src="97sales.gif" alt="Sales for 1997" longdesc="sales.html"><A href="sales.html" title="Description of 1997 sales">[D]</A>
Guideline 1:
Describe frame relationships by using longdesc if title alone is not enough:
<html>
<head><title>Frame’s title</title></head> <frameset rows="20%,40%">
<frame src="top_left.html" title="top_left" longdesc="longdescleft.html"> <frame src="top_right.html" title="top_right" longdesc="longdescright.html"> </frameset>
<!-- Write for browsers that do not support Frame by providing an alternative page -- >
< NOFRAMES> This site requires a frames capable browser. If you can see this message you do not have one. To view this
document without frames you may click <a href=“main.html">here.</a>
</ NOFRAMES>
</html>
Guideline 1:
Alternative presentation of script using the NOSCRIPT element:
<SCRIPT type="text/javascript">document.write("Hello World ! - with JS");</SCRIPT>
<NOSCRIPT> Hello World! - without JS</NOSCRIPT> Or:
< NOSCRIPT >This page uses Javascript. Your browser either doesn't support Javascript or you have it turned off. To see this page as it is meant to appear please use a Javascript enabled browser.</ NOSCRIPT >
Guideline 1:
Guideline 2:
Don’t rely on color alone
All information conveyed with color also available without color:
with color without color
Color contrast:
WAVE Tool:
web accessibility evaluation tool
w
used to aid in the web accessibility
evaluation process
w
To validate a public page:
WAVE toolbar
w The WAVE Firefox toolbar provides a mechanism for running
WAVE reports directly within Firefox.
w The toolbar reports runs entirely within your web browser, no
information is sent to the WAVE server. This ensures 100% private and secure accessibility reporting.
