Interaction Devices
Human Computer Interaction
CIS 6930/4930
Interaction Performance
►
60s vs. Today
▪ Performance
► Hz -> GHz
▪ Memory
► k -> GB
▪ Storage
► k -> TB
▪ Input
► punch cards
► Keyboards, Pens, tablets, mobile
phones, mice, digital cameras, web cams
▪ Output
► 10 character/sec
► Megapixel displays, color laser,
surround sound, force feedback, VR
Interaction Performance
► Future?
▪ Gestural input
▪ Two-handed input
▪ 3D I/O
▪ Others: voice, wearable, whole body, eye trackers, data gloves, haptics, force feedback
▪ Engineering research!
▪ Entire companies created around one single technology
► Current trend:
▪ Multimodal (using car
navigation via buttons or voice)
Keyboard and Keypads
►
QWERTY keyboards been
around for a long time
▪ (1870s – Christopher Sholes)
▪ Cons: Not easy to learn
▪ Pros: Familiarity
▪ Stats:
► Beginners: 1 keystroke per
sec
► Average office worker: 5
keystrokes (50 wpm)
► Experts: 15 keystrokes per
sec (150 wpm)
►
Is it possible to do better?
Keyboard and Keypads
► Look at the piano for possible
inspiration
► Court reporter keyboards (one
keypress = multiple letters or a word)
▪ 300 wpm, requires extensive training and use
► Keyboard properties that matter
▪ Size
► large - imposing for novices,
appears more complex
► mobile devices
▪ Adjustable
► Reduces RSI, better
performance and comfort ▪ Mobile phone keyboards,
Keyboard Layouts
► QWERTY
▪ Frequently used pairs far apart
▪ Fewer typewriter jams
▪ Electronic approaches don’t jam.. why use it?
► DVOARK (1920s)
▪ 150 wpm->200 wpm
▪ Reducing errors
▪ Takes about one week to switch
▪ Stops most from trying
► ABCDE – style
▪ Easier for non-typists
▪ Studies show no improvement vs. QWERTY
► Number pads
▪ What’s in the top row?
▪ Look at phones (slight faster), then look at calculators, keypads
► Those for disabled
▪ Split keyboards
▪ KeyBowl’s orbiTouch (screenshot)
▪ Eyetrackers, mice
Keys
► Current keyboards have been
extensively tested
▪ Size
▪ Shape
▪ Required force
▪ Spacing
► Speed vs. error rates for
majority of users
► Distinctive click gives audio
feedback
▪ Why membrane keyboards are slow (Atari 400?)
► Environment hazards might
necessitate
Keys Guidelines
► Special keys should be denoted ► State keys (such as caps, etc.)
should have easily noted states
► Special curves or dots for home
keys for touch typists
► Inverted T Cursor movement
keys are important (though cross is easier for novices)
► Auto-repeat feature
▪ Improves performance, but only if repeat is customizable (motor impaired, young, old)
► Two thinking points:
▪ Why are home keys fastest to type?
▪ Why are certain keys larger? (Enter, Shift, Space bar)
Keypads for small devices
► PDAs, Cellphones, Game consoles ► Fold out keyboards
► Virtual keyboard
► Cloth keyboards (ElekSen) ► Haptic feedback?
► Mobile phones
▪ Combine static keys with dynamic soft keys
▪ Multi-tap a key to get to a character
▪ Study: Predictive techniques greatly
improve performance
▪ Ex. LetterWise = 20 wpm vs 15 wpm multitap
► Draw keyboard on screen and tap w/ pen
▪ Speed: 20 to 30 wpm (Sears ’93)
► Handwriting recognition (still hard)
Pointing Devices
► Direct manipulation needs some pointing device ► Factors:
▪ Size of device
▪ Accuracy
▪ Dimensionality
► Interaction Tasks:
▪ Select – users choose from a set of items. menu selection, from a list
▪ Position – users choose a point in 1D, 2D, 3D (ex. paint)space.
▪ used to create a drawing, to place a new window.
▪ Orientation – Control orientation or provide direct 3D orientation input
▪ Path – Multiple poses are recorded
► ex. to draw a line
▪ Quantify – control widgets that affect variables
▪ Text – move text
► Faster and gives less error than keyboard ► Two types (Box 9.1)
▪ Direct control – device is on the screen surface
(Lightpen,touchscreen, stylus)
Direct-control pointing
► First device – lightpen▪ Point to a place on screen and press a button
▪ Pros:
► Easy to understand and use
► Very fast for some operations (e.g.
drawing)
▪ Cons:
► Hand gets tired fast!
► Hand and pen blocks view of screen ► Fragile
► Evolved into the touchscreen
▪ Pros: Very robust, no moving parts
▪ Cons: Depending on app, accuracy
could be an issue
► 1600x1600 res with acoustic wave
▪ Must be careful about software design for selection (land-on strategy).
► If you don’t show a cursor of where you
are selecting, users get confused ▪ User confidence is improved with a
Direct-control pointing
►
Primarily for novice
users or large user
base
►
Case study: Disney
World
►
Need to consider those
Indirect-Control Pointing
► Pros:
▪ Reduces hand-fatigue
▪ Reduces obscuration problems
► Cons:
▪ Increases cognitive load
▪ Spatial ability comes more into play
► Mouse
▪ Pros:
► Familiarity
► Wide availability ► Low cost
► Easy to use ► Accurate
▪ Cons:
► Time to grab mouse ► Desk space
► Encumbrance (wire), dirt
► Long motions aren’t easy or obvious (pick up and replace)
Indirect-Control Pointing
►
Trackball
▪
Pros:
► Small physical footprint ► Good for kiosks
►
Joystick
▪ Easy to use, lots of buttons
▪ Good for tracking (guide or follow an on screen object)
▪ Does it map well to your app?
►
Touchpoint
▪ Pressure-sensitive ‘nubbin’ on laptops
Indirect-Control Pointing
►
Touchpad
▪
Laptop mouse device
▪
Lack of moving parts,
and low profile
▪
Accuracy, esp. those w/
motor disabilities
►
Graphics Tablet
▪
Screen shot
▪
comfort
▪
good for cad, artists
Comparing pointing devices
►
Direct pointing
▪ Study: Faster but less accurate than indirect (Haller ’84)
►
Lots of studies confirm mouse is best for most tasks for
speed and accuracy
►
Trackpoint < Trackballs & Touchpads < Mouse
►
Short distances – cursor keys are better
►
Disabled prefer joysticks and trackballs
▪ If force application is a problem, then touch sensitive is preferred
▪ Vision impaired have problems with most pointing devices
► Use multimodal approach or customizable cursors ► Read Vanderheiden ’04 for a case study
►
Designers should smooth out trajectories
Fitts’s Law
► Paul Fitts (1954) developed a model of human hand
movement
► Used to predict time to point at an object
► What are the factors to determine the time to point to
an object?
▪ D – distance to target
▪ W – size of target
► Just from your own experience, is this function linear?
▪ No, since if Target A is D distance and Target B is 2D
distance, it doesn’t take twice as long
▪ What about target size? Not linear there either
► MT = a + b log2(D/W + 1)
▪ a = time to start/stop in seconds (empirically measured per device)
▪ b = inherent speed of the device (empirically measured per device)
▪ Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm
► Ans: 300 + 200 log
2(14/2 + 1) = 900 ms
Fitts’s Law
►
MT = a + b log
2(D/W + 1)
▪
a = time to start/stop in seconds (empirically measured
per device)
▪
b = inherent speed of the device (empirically measured
per device)
▪
Ex. a = 300 ms, b = 200 ms/bit, D = 14 cm, W = 2 cm
► Ans: 300 + 200 log2(14/2 + 1) = 900 ms
▪ Question: If I wanted to half the pointing time (on average), how much do I change the size?
►
Proven to provide good timings for most age groups
►
Newer versions taken into account
▪ Direction (we are faster horizontally than vertically)
▪ Device weight
▪ Target shape
▪ Arm position (resting or midair)
Very Successfully Studied
► Applies to
▪ Feet, eye gaze, head mounted sights
▪ Many types of input devices
▪ Physical environments (underwater!)
▪ User populations (even retarded and drugged)
▪ Drag & Drop and Point & Click ► Limitations
▪ Dimensionality
▪ Software accelerated pointer motion
▪ Training
▪ Trajectory Tasks (Accot-Zhai Steering Law)
▪ Decision Making (Hick’s Law) ► Results (what does it say about)
▪ Buttons and widget size?
▪ Edges?
▪ Popup vs. pull-down menus
▪ Pie vs. Linear menus
▪ iPhone/web pages (real borders) vs. monitor+mouse (virtual borders) ► Interesting readings:
▪ http://particletree.com/features/visualizing-fittss-law/
▪ http://www.asktog.com/columns/022DesignedToGiveFitts.html
Precision Pointing Movement Time
►
Study: Sears and Shneiderman ’91
▪ Broke down task into gross and fine components for small targets
▪ PPMT = a + b log2(D/W+1) + c log2(d/W)
► c – speed for short distance movement ► d – minor distance
▪ Notice how the overall time changes with a smaller target.
►
Other factors
▪ Age (Pg. 369)
►
Research: How can we design devices that produce smaller
constants for the predictive equation
▪ Two handed
8.3.6 Nonstandard interaction and Devices
► Themes:
▪ Make device more diverse ► Users
► Task
▪ Improve match between task and device
▪ Improve affordance
▪ Refine input
▪ Feedback strategies
► Foot controls
▪ Already used in music where hands might be busy
▪ Cars
▪ Foot mouse was twice as slow as hand mouse
8.3.6 Nonstandard interaction and Devices
►
Eye-tracking
▪ Accuracy 1-2 degrees
▪ selections are by constant stare for 200-600 ms
▪ How do you distinguish w/ a selection and a gaze?
▪ Combine w/ manual input
►
Multiple degree of freedom
devices
▪ Logitech Spaceball and SpaceMouse
▪ Ascension Bird
8.3.6 Nonstandard interaction and Devices
►
Boom Chameleon
▪ Pros: Natural, good spatial
understanding
▪ Cons: limited applications,
hard to interact (very passive)
►
DataGlove
▪ Pinch glove
▪ Gesture recognition
▪ American Sign Language, musical director
▪ Pros: Natural
▪ Cons: Size, hygiene,
8.3.6 Nonstandard interaction and Devices
► Haptic Feedback
▪ Why is resistance useful?
▪ SensAble Technology’s Phantom
▪ Cons: limited applications
▪ Sound and vibration are easier and can be a good approximation
► Rumble pack
► Two-Handed input
▪ Different hands have different precision
▪ Non-dominant hand selects fill, the other selects objects
► Ubiquitous Computing and Tangible
User Interface
▪ Active Badges allows you to move about the house w/ your profile
▪ Which sensors could you use?
▪ Elderly, disabled
▪ Research: Smart House
▪ Myron Kruger – novel user
►
Paper/Whiteboards
▪ Video capture of annotations
▪ Record notes (special tracked pens Logitech digital pen)
►
Handheld Devices
▪ PDA
▪ Universal remote
▪ Help disabled
► Read LCD screens
► Rooms in building
► Maps
▪ Interesting body-context-sensitive.
► Ex. hold PDA by ear = phone call
answer.
►
Miscellaneous
▪
Shapetape – reports 3D
shape.
► Tracks limbs
►
Engineer for specific
app (like a gun trigger
connected to serial
port)
▪
Pros: good affordance
▪
Cons: Limited general
use, time
Speech and Auditory Interfaces
►
There’s the dream
►
Then there’s reality
►
Practical apps don’t really require freeform
discussions with a computer
▪
Goals:
► Low cognitive load ► Low error rates
►
Smaller goals:
▪
Speech Store and Forward (voice mail)
▪
Speech Generation
Speech and Auditory Interfaces
►
Bandwidth is much lower than visual displays
►
Ephemeral nature of speech (tone, etc.)
►
Difficulty in parsing/searching (Box 9.2)
►
Types
▪ Discrete-word recognition
▪ Continuous speech
▪ Voice information
▪ Speech generation
▪ Non-speech auditory
►
If you want to do research here, lots of research in the
Discrete-Word Recognition
►
Individual words spoken by a specific person
►
Command and control
►
90-98% for 100-10000 word vocabularies
►
Training
▪ Speaker speaks the vocabulary
▪ Speaker-independent
►
Still requires
▪ Low noise operating environment
▪ Microphones
▪ Vocabulary choice
▪ Clear voice (language disabled are hampered, stressed)
Discrete-Word Recognition
► Helps:
▪ Disabled
▪ Elderly
▪ Cognitive challenged
▪ User is visually distracted
▪ Mobility or space restrictions
► Apps:
▪ Telephone-based info
► Study: much slower for cursor movement than mouse or keyboard
(Christian ’00)
► Study: choosing actions (such as drawing actions) improved
performance by 21% (Pausch ’91) and word processing (Karl ’93)
▪ However acoustic memory requires high cognitive load (> than hand/eye)
Continuous Speech Recognition
► Dictation
► Error rates and error repair are still poor
► Higher cognitive load, could lower overall quality ► Why is it hard?
▪ Recognize boundaries (normal speech blurs them)
▪ Context sensitivity
▪ “How to wreck a nice beach”
► Much training
► Specialized vocabularies (like medical or legal) ► Apps:
▪ Dictate reports, notes, letters
▪ Communication skills practice (virtual patient)
▪ Automatic retrieval/transcription of audio content (like radio, CC)
Voice Information Systems
► Use human voice as a source of info ► Apps:
▪ Tourist info
▪ Museum audio tours
▪ Voice menus (Interactive Voice Response IVR systems)
► Use speech recognition to also cut through menus
▪ If menus are too long, users get frustrated
▪ Cheaper than hiring 24 hr/day reps
► Voice mail systems
▪ Interface isn’t the best
► Get email in your car
▪ Also helps with non-tech savvy like the elderly
► Potentially aides with
▪ Learning (engage more senses)
▪ Cognitive load (hypothesize each sense has a limited ‘bandwidth’)
Speech Generation
►
Play back speech (games)
►
Combine text (navigation systems)
►
Careful evaluation!
▪
Speech isn’t always great
► Door is ajar – now just a tone ► Use flash
► Supermarket scanners
▪
Often times a simple tone is better
▪
Why? Cognitive load
Speech Generation
► Ex: Text-to-Speech (TTS)
► Latest TTS uses multiple syllabi to make generated speech sound
better
▪ Robotic speech could be desirable to get attention
▪ All depends on app
▪ Thus don’t assume one way is the best, you should user test
► Apps: TTS for blind, JAWS
► Web-based voice apps: VoiceXML and SALT (tagged web pages).
▪ Good for disabled, and also for mobile devices
► Use if
▪ Message is short
▪ Requires dynamic responses
▪ Events in time
► Good when visual displays aren’t that useful. When?
Non-speech Auditory Interface
►
Audio tones that provide information
►
Major Research Area
▪
Sonification – converting information into audio
▪
Audiolization
▪
Auditory Interfaces
►
Browsers produced a click when you clicked on a
link
▪
Increases confidence
▪
Can do tasks without visual cognitive load
▪
Helps figure out when things are wrong
Non-speech Auditory Interface
► Terms:
▪ Auditory icons – familiar sounds (record real world sound and play it in your app)
▪ Earcons – new learned sounds (door ajar)
► Role in video games is huge
▪ Emotions, Tension, set mood
► To create 3D sound
▪ Need to do more than stereo
▪ Take into account Head-related transfer function (HRTF)
► Ear and head shape
► New musical instruments
▪ Theremin
Displays
►
Primary Source of
feedback
►
Properties:
▪ Physical Dimension
▪ Resolution
▪ Color Depth and correctness
▪ Brightness, contrast, glare
▪ Power
▪ Refresh rate
▪ Cost
▪ Reliability
Display
Technology
►
Monochrome displays
(single color)
▪ Low cost
▪ Greater intensity range (medical)
►
Color
▪ Raster Scan CRT
▪ LCD – thin, bright
▪ Plasma – very bright, thin
▪ LED – large public displays
▪ Electronic Ink – new product w/ tiny capsules of negative black particles and positive white
Large Displays
►
Wall displays
▪
Informational
► Control rooms, military, flight
control rooms, emergency response
► Provides
▪ System overview
▪ Increases situational awareness
▪ Effective team review
► Old: Array of CRTs
▪
Interactive
► Require new interaction methods
(freehand sketch, PDAs)
Large Displays
►
Multiple Desktop Displays
▪
Multiple CRTs or Flat panels for
large desktops
▪
Cheap
▪
Familiar
▪
Spatial divide up tasks
▪
Comparison tasks are easier
▪
Too much info?
►
HMD
►
Eventually -> Every surface a
Mobile device displays
►
Applications
▪ Personal
► Reprogrammable picture
frames
▪ Digital family portrait (GaTech)
▪ Business
► PDAs, cellphones
▪ Medical
► Monitor patients
▪ Research: Modality
Translation Services (Trace Center – University of
Wisconsin)
► As you move about it auto
Mobile device displays
►
Actions on mobile devices
▪ Monitor information and alert (calendar)
▪ Gather then spread out information (phone)
▪ Participate in groups and relate to individual
(networked devices)
▪ Locate services and identify objects (GPS car system)
Mobile device displays
► Guidelines for design
▪ Bergman ’00, Weiss, ’02
▪ Industry led research and design case studies (Lindholm ’03)
▪ Typically short in time usage (except handheld games)
▪ Optimize for repetitive tasks (rank functions by frequency)
▪ Research: new ways to organize large
amounts of info on a small screen
▪ Study: Rapid Serial Visual Presentation
(RSVP) presents text at a constant speed (33% improvement Oquist ’03)
▪ Searching and web browsing still very poor performance
Animation, Image, and Video
►
Content quality has also greatly
increased
►
3D rendering is near life-like
►
Digital Photography is common
►
Scanned documents
►
Video compression
►
Multimedia considerations for the
disabled
►
