Interaction Devices

Interaction Devices

Human Computer Interaction

CIS 6930/4930

Interaction Performance

►

60s vs. Today

 Performance

► Hz -> GHz

 Memory

► k -> GB

 Storage

► k -> TB

 Input

► punch cards

► Keyboards, Pens, tablets, mobile

phones, mice, digital cameras, web cams

 Output

► 10 character/sec

► Megapixel displays, color laser,

surround sound, force feedback, VR

Interaction Performance

► Future?

 Gestural input

 Two-handed input

 3D I/O

 Others: voice, wearable, whole body, eye trackers, data gloves, haptics, force feedback

 Engineering research!

 Entire companies created around one single technology

► Current trend:

 Multimodal (using car

navigation via buttons or voice)

Keyboard and Keypads

►

QWERTY keyboards been

around for a long time

 (1870s – Christopher Sholes)

 Cons: Not easy to learn

 Pros: Familiarity

 Stats:

► Beginners: 1 keystroke per

sec

► Average office worker: 5

keystrokes (50 wpm)

► Experts: 15 keystrokes per

sec (150 wpm)

►

Is it possible to do better?

Keyboard and Keypads

► Look at the piano for possible

inspiration

► Court reporter keyboards (one

keypress = multiple letters or a word)

 300 wpm, requires extensive training and use

► Keyboard properties that matter

 Size

► large - imposing for novices,

appears more complex

► mobile devices  Adjustable

► Reduces RSI, better

performance and comfort

Keyboard Layouts

► QWERTY

 Frequently used pairs far apart

 Fewer typewriter jams

 Electronic approaches don’t jam.. why use it?

► DVOARK (1920s)

 150 wpm->200 wpm

 Reducing errors

 Takes about one week to switch

 Stops most from trying

► ABCDE – style

 Easier for non-typists

 Studies show no improvement vs. QWERTY

► Number pads

 What’s in the top row?

 Look at phones (slight faster), then look at calculators, keypads

► Those for disabled

 Split keyboards

 KeyBowl’s orbiTouch (screenshot)

 Eyetrackers, mice

Keys

► Current keyboards have been

extensively tested

 Size

 Shape

 Required force

 Spacing

► Speed vs. error rates for

majority of users

► Distinctive click gives audio

feedback

 Why membrane keyboards are slow (Atari 400?)

► Environment hazards might

necessitate

Keys Guidelines

► Special keys should be denoted ► State keys (such as caps, etc.)

should have easily noted states

► Special curves or dots for home

keys for touch typists

► Inverted T Cursor movement

keys are important (though cross is easier for novices)

► Auto-repeat feature

 Improves performance, but only if repeat is customizable (motor impaired, young, old)

► Two thinking points:

 Why are home keys fastest to type?

 Why are certain keys larger? (Enter, Shift, Space bar)

Keypads for small devices

► PDAs, Cellphones, Game consoles ► Fold out keyboards

► Virtual keyboard

► Cloth keyboards (ElekSen) ► Haptic feedback?

► Mobile phones

 Combine static keys with dynamic soft

keys

 Multi-tap a key to get to a character

 Study: Predictive techniques greatly improve performance

 Ex. LetterWise = 20 wpm vs 15 wpm multitap

► Draw keyboard on screen and tap w/

pen

 Speed: 20 to 30 wpm (Sears ’93)

► Handwriting recognition (still hard)

Pointing Devices

► Direct manipulation needs some pointing device ► Factors:

 Size of device

 Accuracy

 Dimensionality

► Interaction Tasks:

 Select – users choose from a set of items. menu selection, from a list

 Position – users choose a point in 1D, 2D, 3D (ex. paint)space.

 used to create a drawing, to place a new window.

 Orientation – Control orientation or provide direct 3D orientation input

 Path – Multiple poses are recorded

► ex. to draw a line

 Quantify – control widgets that affect variables

 Text – move text

► Faster and gives less error than keyboard ► Two types (Box 9.1)

 Direct control – device is on the screen surface (Lightpen,touchscreen, stylus)

Direct-control pointing

► First device – lightpen

 Point to a place on screen and press a button

 Pros:

► Easy to understand and use

► Very fast for some operations (e.g.

drawing)

 Cons:

► Hand gets tired fast!

► Hand and pen blocks view of screen ► Fragile

► Evolved into the touchscreen

 Pros: Very robust, no moving parts

 Cons: Depending on app, accuracy could be an issue

► 1600x1600 res with acoustic wave

 Must be careful about software design for selection (land-on strategy).

► If you don’t show a cursor of where you

are selecting, users get confused

Direct-control pointing

►

Primarily for novice

users or large user

base

►

Case study: Disney

World

►

Need to consider those

Indirect-Control Pointing

► Pros:

 Reduces hand-fatigue

 Reduces obscuration problems

► Cons:

 Increases cognitive load

 Spatial ability comes more into play

► Mouse

 Pros:

► Familiarity

► Wide availability ► Low cost

► Easy to use ► Accurate  Cons:

► Time to grab mouse ► Desk space

► Encumbrance (wire), dirt

Indirect-Control Pointing

►

Trackball



Pros:

►Small physical footprint ►Good for kiosks

►

Joystick

 Easy to use, lots of buttons

 Good for tracking (guide or follow an on screen object)

 Does it map well to your app?

►

Touchpoint

 Pressure-sensitive ‘nubbin’ on laptops

Indirect-Control Pointing

►

Touchpad



Laptop mouse device



Lack of moving parts,

and low profile



Accuracy, esp. those w/

motor disabilities

►

Graphics Tablet



Screen shot



comfort



good for cad, artists

Comparing pointing devices

►

Direct pointing

 Study: Faster but less accurate than indirect (Haller ’84)

►

Lots of studies confirm mouse is best for most tasks for

speed and accuracy

►

Trackpoint < Trackballs & Touchpads < Mouse

►

Short distances – cursor keys are better

►

Disabled prefer joysticks and trackballs

 If force application is a problem, then touch sensitive is preferred

 Vision impaired have problems with most pointing devices

► Use multimodal approach or customizable cursors ► Read Vanderheiden ’04 for a case study

►

Designers should smooth out trajectories

Fitts’s Law

► Paul Fitts (1954) developed a model of human hand

movement

► Used to predict time to point at an object

► What are the factors to determine the time to point to

an object?

 D – distance to target

 W – size of target

► Just from your own experience, is this function linear?

 No, since if Target A is D distance and Target B is 2D distance, it doesn’t take twice as long

 What about target size? Not linear there either

► _{MT = a + b log}

2(D/W + 1)

 a = time to start/stop in seconds (empirically measured per device)

 b = inherent speed of the device (empirically measured per device)

 Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm

► _{Ans: 300 + 200 log}

2(14/2 + 1) = 900 ms

Fitts’s Law

►

MT = a + b log

2

(D/W + 1)



a = time to start/stop in seconds (empirically measured

per device)



b = inherent speed of the device (empirically measured

per device)



Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm

►_{Ans: 300 + 200 log}

2(14/2 + 1) = 900 ms

 Question: If I wanted to half the pointing time (on average), how much do I change the size?

►

Proven to provide good timings for most age groups

►

Newer versions taken into account

 Direction (we are faster horizontally than vertically)

 Device weight

 Target shape

 Arm position (resting or midair)

Very Successfully Studied

► Applies to

 Feet, eye gaze, head mounted sights

 Many types of input devices

 Physical environments (underwater!)

 User populations (even retarded and drugged)

 Drag & Drop and Point & Click

► Limitations

 Dimensionality

 Software accelerated pointer motion

 Training

 Trajectory Tasks (Accot-Zhai Steering Law)

 Decision Making (Hick’s Law)

► Results (what does it say about)

 Buttons and widget size?

 Edges?

 Popup vs. pull-down menus

 Pie vs. Linear menus

 iPhone/web pages (real borders) vs. monitor+mouse (virtual borders)

► Interesting readings:

 http://particletree.com/features/visualizing-fittss-law/

 http://www.asktog.com/columns/022DesignedToGiveFitts.html

Precision Pointing Movement Time

►

Study: Sears and Shneiderman ’91

 Broke down task into gross and fine components for small targets

 _{PPMT = a + b log2(D/W+1) + c log2(d/W)}

► c – speed for short distance movement ► d – minor distance

 Notice how the overall time changes with a smaller target.

►

Other factors

 Age (Pg. 369)

►

Research: How can we design devices that produce smaller

constants for the predictive equation

 Two handed

8.3.6 Nonstandard interaction and Devices

► Themes:

 Make device more diverse

► Users ► Task

 Improve match between task and device

 Improve affordance

 Refine input

 Feedback strategies

► Foot controls

 Already used in music where hands might be busy

 Cars

 Foot mouse was twice as slow as hand mouse

8.3.6 Nonstandard interaction and Devices

►

Eye-tracking

 Accuracy 1-2 degrees

 selections are by constant stare for 200-600 ms

 How do you distinguish w/ a selection and a gaze?

 Combine w/ manual input

►

Multiple degree of freedom

devices

 Logitech Spaceball and SpaceMouse

 Ascension Bird

8.3.6 Nonstandard interaction and Devices

►

Boom Chameleon

 Pros: Natural, good spatial understanding

 Cons: limited applications, hard to interact (very

passive)

►

DataGlove

 Pinch glove

 Gesture recognition

 American Sign Language, musical director

 Pros: Natural

8.3.6 Nonstandard interaction and Devices

► Haptic Feedback

 Why is resistance useful?

 SensAble Technology’s Phantom

 Cons: limited applications

 Sound and vibration are easier and can be a good approximation

► Rumble pack ► Two-Handed input

 Different hands have different precision

 Non-dominant hand selects fill, the other selects objects

► Ubiquitous Computing and Tangible

User Interface

 Active Badges allows you to move about the house w/ your profile

 Which sensors could you use?

 Elderly, disabled

 Research: Smart House

 Myron Kruger – novel user

►

Paper/Whiteboards

 Video capture of annotations

 Record notes (special tracked pens Logitech digital pen)

►

Handheld Devices

 PDA

 Universal remote

 Help disabled

► Read LCD screens ► Rooms in building ► Maps

 Interesting body-context-sensitive.

► Ex. hold PDA by ear = phone call

answer.

►

Miscellaneous



Shapetape – reports 3D

shape.

►Tracks limbs

►

Engineer for specific

app (like a gun trigger

connected to serial

port)



Pros: good affordance



Cons: Limited general

use, time

Speech and Auditory Interfaces

►

There’s the dream

►

Then there’s reality

►

Practical apps don’t really require freeform

discussions with a computer



Goals:

►Low cognitive load ►Low error rates

►

Smaller goals:



Speech Store and Forward (voice mail)



Speech Generation

Speech and Auditory Interfaces

►

Bandwidth is much lower than visual displays

►

Ephemeral nature of speech (tone, etc.)

►

Difficulty in parsing/searching (Box 9.2)

►

Types

 Discrete-word recognition

 Continuous speech

 Voice information

 Speech generation

 Non-speech auditory

►

If you want to do research here, lots of research in the

Discrete-Word Recognition

►

Individual words spoken by a specific person

►

Command and control

►

90-98% for 100-10000 word vocabularies

►

Training

 Speaker speaks the vocabulary

 Speaker-independent

►

Still requires

 Low noise operating environment

 Microphones

 Vocabulary choice

 Clear voice (language disabled are hampered, stressed)

Discrete-Word Recognition

► Helps:

 Disabled

 Elderly

 Cognitive challenged

 User is visually distracted

 Mobility or space restrictions

► Apps:

 Telephone-based info

► Study: much slower for cursor movement than mouse or keyboard

(Christian ’00)

► Study: choosing actions (such as drawing actions) improved

performance by 21% (Pausch ’91) and word processing (Karl ’93)

 However acoustic memory requires high cognitive load (> than hand/eye)

Continuous Speech Recognition

► Dictation

► Error rates and error repair are still poor

► Higher cognitive load, could lower overall quality ► Why is it hard?

 Recognize boundaries (normal speech blurs them)

 Context sensitivity

 “How to wreck a nice beach”

► Much training

► Specialized vocabularies (like medical or legal) ► Apps:

 Dictate reports, notes, letters

 Communication skills practice (virtual patient)

 Automatic retrieval/transcription of audio content (like radio, CC)

Voice Information Systems

► Use human voice as a source of info

► Apps:

 Tourist info

 Museum audio tours

 Voice menus (Interactive Voice Response IVR systems)

► Use speech recognition to also cut through menus

 If menus are too long, users get frustrated

 Cheaper than hiring 24 hr/day reps

► Voice mail systems

 Interface isn’t the best

► Get email in your car

 Also helps with non-tech savvy like the elderly

► Potentially aides with

 Learning (engage more senses)

 Cognitive load (hypothesize each sense has a limited ‘bandwidth’)

Speech Generation

►

Play back speech (games)

►

Combine text (navigation systems)

►

Careful evaluation!



Speech isn’t always great

►Door is ajar – now just a tone ►Use flash

►Supermarket scanners



Often times a simple tone is better



Why? Cognitive load

Speech Generation

► Ex: Text-to-Speech (TTS)

► Latest TTS uses multiple syllabi to make generated speech sound

better

 Robotic speech could be desirable to get attention

 All depends on app

 Thus don’t assume one way is the best, you should user test

► Apps: TTS for blind, JAWS

► Web-based voice apps: VoiceXML and SALT (tagged web pages).

 Good for disabled, and also for mobile devices

► Use if

 Message is short

 Requires dynamic responses

 Events in time

► Good when visual displays aren’t that useful. When?

Non-speech Auditory Interface

►

Audio tones that provide information

►

Major Research Area



Sonification – converting information into audio



Audiolization



Auditory Interfaces

►

Browsers produced a click when you clicked on a

link



Increases confidence



Can do tasks without visual cognitive load



Helps figure out when things are wrong

Non-speech Auditory Interface

► Terms:

 Auditory icons – familiar sounds (record real world sound and play it in your app)

 Earcons – new learned sounds (door ajar)

► Role in video games is huge

 Emotions, Tension, set mood

► To create 3D sound

 Need to do more than stereo

 Take into account Head-related transfer function (HRTF)

► Ear and head shape

► New musical instruments

 Theremin

Displays

►

Primary Source of

feedback

►

Properties:

 Physical Dimension

 Resolution

 Color Depth and correctness

 Brightness, contrast, glare

 Power

 Refresh rate

 Cost

 Reliability

Display

Technology

►

Monochrome displays

(single color)

 Low cost

 Greater intensity range (medical)

►

Color

 Raster Scan CRT

 LCD – thin, bright

 Plasma – very bright, thin

 LED – large public displays

 Electronic Ink – new product w/ tiny capsules of negative black particles and positive white

Large Displays

►

Wall displays



Informational

►Control rooms, military, flight

control rooms, emergency response

►Provides

 System overview

 Increases situational awareness

 Effective team review

►Old: Array of CRTs



Interactive

►Require new interaction

methods (freehand sketch, PDAs)

Large Displays

►

Multiple Desktop Displays



Multiple CRTs or Flat panels for

large desktops



Cheap



Familiar



Spatial divide up tasks



Comparison tasks are easier



Too much info?

►

HMD

►

Eventually -> Every surface a

Mobile device displays

►

Applications

 Personal

► Reprogrammable picture

frames

 Digital family portrait (GaTech)

 Business

► PDAs, cellphones

 Medical

► Monitor patients

 Research: Modality

Translation Services (Trace Center – University of

Wisconsin)

► As you move about it auto

Mobile device displays

►

Actions on mobile devices

 Monitor information and alert (calendar)

 Gather then spread out information (phone)

 Participate in groups and relate to individual

(networked devices)

 Locate services and identify objects (GPS car system)

Mobile device displays

► Guidelines for design

 Bergman ’00, Weiss, ’02

 Industry led research and design case studies (Lindholm ’03)

 Typically short in time usage (except handheld games)

 Optimize for repetitive tasks (rank functions by frequency)

 Research: new ways to organize large amounts of info on a small screen

 Study: Rapid Serial Visual Presentation (RSVP) presents text at a constant speed (33% improvement Oquist ’03)

 _{Searching and web browsing still very poor}

performance

Animation, Image, and Video

►

Content quality has also greatly

increased

►

3D rendering is near life-like

►

Digital Photography is common

►

Scanned documents

►

Video compression

►

Multimedia considerations for the

disabled

►

Printers