User Interface Design

(1)

20. Februar 06 IfGi Universität Münster User Interface Design A. Krüger 1

User Interface Design

Winter term 2005/2006

Thursdays, 14-16 c.t., Raum 228

Prof. Dr. Antonio Krüger

Institut für Geoinformatik

Universität Münster

(2)

(3)

The aims



Discuss the benefits & challenges of different types of

observation.



Describe how to observe as an on-looker, a participant,

& an ethnographer.



Discuss how to collect, analyze & present observational

data.



Examine think-aloud, diary studies & logging.



Provide you with experience in doing observation and

critiquing observation studies.

(4)

What and when to observe

(similar to “what and when to evaluate” from last

week)

 Goals & questions determine the paradigms and techniques used.

 Observation is valuable any time during design.

 Quick & dirty observations early in design

 Observation can be done in the field (i.e., field studies) and in controlled environments (i.e., usability studies)

 Observers can be:

• outsiders looking on

• participants, i.e., participant observers • ethnographers

(5)

Frameworks to guide

observation



The person

. Who?



The place.

Where?



The thing.

What?



The Goetz and LeCompte

(1984) framework:

• Who

is present? What is their role?

• What

is happening?

• When

does the activity occur?

• Where

is it happening?

• Why

is it happening?

(6)

The Robinson (1993) framework

 Space. What is the physical space like?

 Actors. Who is involved?

 Activities. What are they doing?

 Objects. What objects are present?

 Acts. What are individuals doing?

 Events. What kind of event is it?

 Goals. What do they to accomplish?

(7)

You need to consider

 Goals & questions

 Which framework & techniques  How to collect data

 Which equipment to use  How to gain acceptance

 How to handle sensitive issues

 Whether and how to involve informants  How to analyze the data

(8)

Observing as an outsider

 As in usability testing

 More objective than participant observation

 In usability lab equipment is in place

 Recording is continuous

 Analysis & observation almost simultaneous

 Care needed to avoid drowning in data

 Analysis can be coarse or fine grained

(9)

Participant observation &

ethnography

 Debate about differences

 Participant observation is key component of ethnography

 Must get co-operation of people observed

 Informants are useful

 Data analysis is continuous

 Interpretivist technique

 Questions get refined as understanding grows

(10)

Data collection techniques



Notes & still camera



Audio & still camera



Video



Tracking users:

• Diaries

• Interaction logging

(11)

Data analysis

· Qualitative

data

-

interpreted

& used to

tell the ‘story’ about what was observed.

· Qualitative data

-

categorized

using

techniques such as content analysis.

· Quantitative data

- collected from

interaction & video logs. Presented as

values, tables, charts, graphs and treated

statistically.

(12)

Interpretive data analysis

· Look for

key events

that drive activity

• Critical incident analysis

• getting stuck, comments, puzzled looks, etc.

· Look for

patterns

of behavior

· Test data sources against each other -

triangulate

· Report findings in a convincing and honest way

· Produce ‘rich’ or ‘thick descriptions’

· Include quotes, pictures, and anecdotes

(13)

Key points

· Observe from outside or as a participant

· Analyzing video and data logs can be time-consuming.

· In participant observation collections of comments,

incidents, and artifacts are made. Ethnography is a

philosophy with a set of techniques that include

participant observation and interviews.

· Ethnographers immerse themselves in the culture that

they study.

(14)

(15)

The aims

 Discuss the role of interviews & questionnaires in evaluation.

 Teach basic questionnaire design.

 Describe how do interviews, heuristic evaluation & walkthroughs.

 Describe how to collect, analyze & present data.

(16)

Interviews



Unstructured

- are not directed by a script.

Rich but not replicable.



Structured

- are tightly scripted, often like

a questionnaire. Replicable but may lack

richness.



Semi-structured

- guided by a script but

interesting issues can be explored in

more depth. Can provide a good balance

between richness and replicability.

(17)

Basics of interviewing

 Remember the DECIDE framework

 Goals and questions guide all interviews

 Two types of questions:

• ‘closed questions’ have a predetermined answer format, e.g., ‘yes’ or ‘no’

• ‘open questions’ do not have a predetermined format

(18)

Things to avoid when preparing

interview questions

· Long questions

· Compound sentences - split into two

· Jargon & language that the interviewee may not

understand

· Leading questions that make assumptions e.g.,

why do you like …?

(19)

Components of an interview

 Introduction - introduce yourself, explain the goals of the interview, reassure about the ethical issues, ask to record, present an informed consent form.

 Warm-up - make first questions easy & non-threatening.

 Main body – present questions in a logical order

 A cool-off period - include a few easy questions to defuse tension at the end

 Closure - thank interviewee, signal the end, e.g, switch recorder off.

(20)

The interview process

 Use the DECIDE framework for guidance

 Dress in a similar way to participants

 Check recording equipment in advance

 Devise a system for coding names of participants to preserve confidentiality.

 Be pleasant

(21)

Probes and prompts

 Probes - devices for getting more information. e.g., ‘would you like to add anything?’

 Prompts - devices to help interviewee, e.g., help with remembering a name

 Remember that probing and prompting should not create bias.

(22)

Group interviews



Also known as ‘focus groups’



Typically 3-10 participants



Provide a diverse range of opinions



Need to be managed to:

• ensure everyone contributes

• discussion isn’t dominated by one person

• the agenda of topics is covered

(23)

Analyzing interview data



Depends on the type of interview



Structured interviews can be analyzed

like questionnaires



Unstructured interviews generate data

like that from participant observation



It is best to analyze unstructured

interviews as soon as possible to identify

topics and themes from the data

(24)

Questionnaires

 Questions can be closed or open

 Closed questions are easiest to analyze, and may be done by computer

 Can be administered to large populations

 Paper, email & the web used for dissemination

 Advantage of electronic questionnaires is that data goes into a data base & is easy to analyze

 Sampling can be a problem when the size of a population is unknown as is common online

(25)

Questionnaire style

 Varies according to goal so use the DECIDE framework for guidance  Questionnaire format can include:

• ‘yes’, ‘no’ checkboxes

• checkboxes that offer many options • Likert rating scales

• semantic scales

• open-ended responses

 Likert scales have a range of points • 3, 5, 7 & 9 point scales are common • Example from c’t: -- - 0 + ++

(26)

Likert Scales: Example 1

I prefer lighter colors to darker ones

Site is aesthetically pleasing

Information is easy to find

The page contains useful information Navigation language is clear+ understandable Strongly Disagre e Disagree Neutral Agree Strongly Agree Question

(27)

Likert Scales: Example 2

10. I can tell that my coworkers respect me.

9. I feel like I make a useful contribution at work.

8. I am confident that I can handle my job without constant assistance.

7. I am proud of my relationship with my supervisor at work.

6. I know I'll be able to cope with work for as long as I want.

5. I can tell that other people at work are glad to have me there.

4. When I feel uncomfortable at work, I know how to handle it.

3. I am proud of my ability to cope with difficulties at work.

2. On the whole, I get along well with others at work.

1. I feel good about my work on the job.

Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e Strongly Agree Somewhat Agree Somewhat Disagree Strongly Disagre e

(28)

Semantic Differential Scales

Lazy Energetic Active Passive Wise Foolish Light Heavy Reliable Unreliable Intelligent Stupid Weak Strong Rugged Delicate Biassed Fair Unhelpful Helpful Cruel Kind Dishones t Honest Dirty Clean 7 6 5 4 3 2 1



Balance of

positive/negative

attributes



Medium values

might be

preferred



What they say ==

what they do ??

(29)

Developing a questionnaire

 Provide a clear statement of purpose & guarantee participants anonymity  Plan questions - if developing a web-based questionnaire, design off-line

first

 Decide on whether phrases will all be positive, all negative or mixed

 Pilot test questions - are they clear, is there sufficient space for responses  Decide how data will be analyzed & consult a statistician if necessary

(30)

Encouraging a good response

 Make sure purpose of study is clear  Promise anonymity

 Ensure questionnaire is well designed

 Offer a short version for those who do not have time to complete a long questionnaire

 If mailed, include an answer envelope  Follow-up with emails, phone calls, letters  Provide an incentive

(31)

Advantages of online

questionnaires

· Responses are usually

received quickly

· No copying and postage costs

· Data can be collected in database for

analysis

· Time required for data analysis is reduced

(32)

Problems with online

questionnaires

· Sampling is problematic

if population size

is unknown

· Preventing individuals from responding

more than once

· Individuals have also been known to

(33)

Questionnaire data analysis &

presentation



Present results clearly – e.g., tables



Simple statistics can say a lot, e.g., mean,

median, mode, standard deviation



Percentages are useful but: give

population size



Bar graphs show categorical data well



More advanced statistics can be used if

needed

(34)

Asking experts



Experts use their knowledge of users &

technology to review software usability



Expert critiques (crits) can be formal or

informal reports



Heuristic evaluation is a review guided by

a set of heuristics



Walkthroughs involve stepping through a

pre-planned scenario noting potential

(35)

Heuristic evaluation

 Developed by Jacob Nielsen in the early 1990s

 Based on heuristics distilled from an empirical analysis of 249 usability problems

 These heuristics have been revised for current technology, e.g., HOMERUN for web

 Heuristics still needed for mobile devices, wearables, virtual worlds, etc.

(36)

Nielsen’s heuristics

 Visibility of system status

 Match between system and real world  User control and freedom

 Consistency and standards

 Help users recognize, diagnose, recover from errors  Error prevention

 Recognition rather than recall  Flexibility and efficiency of use  Aesthetic and minimalist design  Help and documentation

(37)

Discount evaluation



Heuristic evaluation is referred to as

discount evaluation when 5 evaluators

are used.



Empirical evidence suggests that on

average 5 evaluators identify 75-80% of

usability problems.

(38)

3 stages for doing heuristic

evaluation



Briefing session to tell experts what to do



Evaluation period of 1-2 hours in which:

• Each expert works separately

• Take one pass to get a feel for the product

• Take a second pass to focus on specific features



Debriefing session in which experts work

together to prioritize problems

(39)

Advantages and problems



Few ethical & practical issues to consider



Can be difficult & expensive to find experts



Best experts have knowledge of

application domain & users



Biggest problems

• important problems may get missed

(40)

Cognitive walkthroughs



Focus on ease of learning



Designer presents an aspect of the

design & usage scenarios



One or more experts walk through the

design prototype with the scenario



Expert is told the assumptions about user

population, context of use, task details

(41)

The 3 questions



Will the

correct action

be sufficiently

evident

to the user?



Will the user notice that the

correct action

is

available

?



Will the user associate and

interpret

the

response

from the action

correctly

?

As the experts work through the scenario

they note problems

(42)

Pluralistic walkthrough

 Variation on the cognitive walkthrough theme

 Performed by a carefully managed team

 The panel of experts begins by working separately

 Then there is managed discussion that leads to agreed decisions

(43)

Key points

 Structured, unstructured, semi-structured interviews, focus groups & questionnaires

 Closed questions are easiest to analyze & can be replicated

 Open questions are richer

 Check boxes, Likert & semantic scales

 Expert evaluation: heuristic & walkthroughs

 Relatively inexpensive because no users

 Heuristic evaluation relatively easy to learn

(44)

(45)

The aims

· Describe how to do user testing.

· Discuss the differences between user testing, usability testing and research experiments.

· Discuss the role of user testing in usability testing. · Discuss how to design simple experiments.

· Describe GOMS, the keystroke level model, Fitts’ law and discuss when these techniques are useful.

(46)

Experiments, user testing &

usability testing



Experiments test hypotheses to discover new

knowledge by investigating the relationship between two

or more things – i.e., variables.



User testing is applied experimentation in which

developers check that the system being developed is

usable by the intended user population for their tasks.



Usability testing uses a combination of techniques,

(47)

User testing is not research

User testing

 Aim: improve products

 Few participants

 Results inform design

 Not perfectly replicable

 Controlled conditions

 Procedure planned

 Results reported to developers

Research experiments

 Aim: discover knowledge

 Many participants

 Results validated statistically

 Replicable

 Strongly controlled conditions

 Experimental design

 Scientific paper reports results to community

(48)

User testing



Goals & questions focus on how well users perform

tasks with the product



Comparison of products or prototypes common



Major part of usability testing



Focus is on time to complete task & number & type of

errors



Informed by video & interaction logging



User satisfaction questionnaires provide data about

users’ opinions

(49)

Testing conditions



Usability lab or other controlled space



Major emphasis on

• selecting representative users • developing representative tasks



5-10 users typically selected



Tasks usually last no more than 30 minutes



The test conditions should be the same for every

participant

(50)

Type of data

(Wilson & Wixon, ‘97)

· Time to complete a task

· Time to complete a task after a specified time away

from the product

· Number and type of errors per task

· Number of errors per unit of time

· Number of navigations to online help or manuals

· Number of users making a particular error

(51)

Usability engineering orientation

· Current level of performance

· Minimum acceptable level of performance

(52)

How many participants is

enough for user testing?



The number is largely a practical issue



Depends on:

• schedule for testing

• availability of participants

• cost of running tests



Typical 5-10 participants



Some experts argue that testing should

continue until no new insights are gained

(53)

Experiments



Predict the relationship between two or

more variables



Independent variable is manipulated by

the researcher



Dependent variable depends on the

independent variable



Typical experimental designs have one or

two independent variable

(54)

Experimental designs



Different participants - single group of

participants is allocated randomly to the

experimental conditions



Same participants - all participants

appear in every condition



Matched participants - participants are

matched in tuples, e.g., based on

(55)

Example



Hypotheses: “Will the time to read a screen of

text be different if 12-point Helvetica is used

instead of 12-point Times-Roman?”



Condition 1: users read text with Helvetica



Condition 2: users read text with Times Roman



Control condition: read text on paper



Extend design with variable

user-expertise

(additional conditions: expert/beginner)



What are the independent and dependent

variables

(56)

(57)

Evaluation of experiments



Are the results

statistically

significant?



Use the

t-test

to

analyze the ration of

means and group

(58)

T-test

(59)

Predictive models



Provide a way of evaluating products or designs without

directly involving users



Psychological models of users are used to test designs



Less expensive than user testing



Usefulness limited to systems with predictable tasks

-e.g., telephone answering systems, mobiles, etc.

(60)

GOMS

(Card et al., 1983)



Goals - the state the user wants to achieve e.g., find a

website



Operators - the cognitive processes & physical actions

performed to attain those goals, e.g., decide which

search engine to use



Methods - the procedures for accomplishing the goals,

e.g., drag mouse over field, type in keywords, press the

go button



Selection rules - determine which method to select

when there is more than one available

(61)

Keystroke level model



GOMS has also been developed further

into a quantitative model - the keystroke

level model.



This model allows predictions to be made

about how long it takes an expert user to

perform a task.

(62)

(63)

Response times for keystroke

level operators

(64)

Problems of GOMS/Keystroke

model



Doesn’t take into account slack times and critical

situations that may slow down certain strokes.



Example: Usage of system while talking to a person in

parallel.



Further influences that are not taken into account:

fatigue, learning effects, workload, etc..



Models are just good to provide an estimate, they can’t

substitute user testing

(65)

Fitts’ Law

(Paul Fitts 1954)



The law predicts that the time to point at an object using

a device is a function of the distance from the target

object & the object’s size.



The further away & the smaller the object, the longer the

time to locate it and point.



Useful for evaluating systems for which the time to

locate an object is important such

(66)

Key points

· User testing is a central part of usability testing · Testing is done in controlled conditions

· User testing is an adapted form of experimentation

· Experiments aim to test hypotheses by manipulating certain variables while keeping others constant

· The experimenter controls the independent variable(s) but not the dependent variable(s)

· There are three types of experimental design: different-participants, same-participants, & matched participants

· GOMS, Keystroke level model, & Fitts’ Law predict expert, error-free performance

· Predictive models are used to evaluate systems with predictable tasks such as telephones