App analytics: evaluating the distraction potential of in-vehicle device apps

(1)

App analytics: evaluating the

distraction potential of in-vehicle

device apps

Institute of Ergonomics (TU Munich)

Michael Krause, Antonia Conti, Klaus Bengler

Carmeq

Matthias Henning, Christopher Seubert

Daimler

Christian Heinrich

Laudenklos Engineering

Carolin Herrigel

GM Corporation

Daniel Glaser

(2)

Agenda

• Motivation

• Methods

• Results

(3)

COPYRIGHT RIGHTS OF THE CONTENT REMAIN WITH THE INSTITUTE OF ERGONOMICS TU MÜNCHEN

Project Purpose in a Nutshell

• Investigate different subtasks used in mobile device apps.

• Subtask examples: scroll a list, enter a zip code

• Typical subtasks will be evaluated to explore their distraction

potential.

• Main project goal is to be able to judge whether an app is

ready to be tested in a driving simulator (with test

participants) based on its component subtasks.

(4)

Approach & Aim

• Acquire gaze duration and occlusion values (used to quantify driver

distraction) for typical interactions (e.g. scroll a list, type a name, etc.) with

“natural” apps while driving in a mockup. • Aim: to evaluate the possibility and

feasibility of a method for human factors experts to predict the experimental

outcome of a driving simulation

experiment—so clearly unsuitable apps can be pre-excluded from further testing.

(5)

Other Task Analytics

• Task Analysis

– GOMS Model (Goals, Operators, Methods, Selection rules) • Keystroke-level (KLM-GOMS)

– SAE J 2365 Calculation of the Time to Complete In-Vehicle Navigation and Route Guidance Tasks

» Adaptation of KLM-GOMS to In-Vehicle Tasks » 15-Second Rule

• Methods Time Measurement

(6)

Overview

• First Experiment: Acquire data while driving and interacting with

touch screen apps

• Second Experiment: Acquire data while driving and interacting with a

rotary knob

• Predict the outcomes (Single Glance Duration, Total Glance Time and Total Shutter Open Time) of the third experiment with the outcomes the first two experiments

• Third experiment: Acquire data while driving and interacting with touchscreen and rotary knob apps

(7)

(8)

Measurement Methods

• Occlusion

Test person performs a task with glasses on that periodically open (e.g. transparent) and close (e.g. opaque). Task interruptability and

resumability are tested.

Open/closed intervals used: open: 1500; closed: 1000 ms

• Eye tracking

Test person wears Dikablis eye tracking headset. Single glance

durations and total glance times are measured.

(9)

Simulated Driving Task

• AAM scenario adapted to German Autobahn standards

• Participants were instructed and trained to follow a leading vehicle traveling at 80km/h at a distance of approximately 50m

• Seating mockup with SILAB 4.0 (WIVW GmbH, Würzburg)

(10)

Touch Tablet

• Intenso Tab 824, rooted and screen adjusted to display at: 800 x 480. 160ppi (5.8”).

(11)

Rotary & Radio

30th July 2015 Institute of Ergonomics - Technische Universität München 11

• Rotary: Daimler COMAND system with iPhone coupling

(Digital DriveStyle)

(12)

Procedure

• Approximate experiment duration: – Experiment 1 (touch): 3-4 hrs – Experiment 2 (rotary knob): 2 hrs – Experiment 3 (both): 2 hrs

• Procedure:

– Participants were trained with the system(s) under investigation. – Different system tasks performed with occlusion and eye tracking in

randomized order.

– Each task was separately explained and trained, followed by two measurement blocks (occlusion and eye tracking).

– Order of occlusion and eye tracking was also randomized. – Reference tasks were performed: Radio tuning task.

(13)

Example “Slider”

Four subtasks

• Select icon (start app) • System delay

• Swipe and select button • Adjust slider

(14)

Metrics of Interest

Single Glance Duration

Sum of sequential fixations to an Area of Interest (AoI)

Total Glance Time (Eye tracking)

Sum of single glance durations to an Area of Interest (AoI) during a task.

Total Shutter Open Time (Occlusion)

(15)

85

th

_Percentile

The 85th _{Percentile (P85) is often used for gaze metrics.}

P85 tells that 85% of all measurements are below this P85 value.

(16)

Berlin-Munich-Method

• Instead of storing a single (mean) value (Y) for every sub-task, we store a (mean) value (Y1, Y2, Y3,…Yn) for each subject and every subtask - by doing this, we run a "virtual experiment"

Subject_1 Subject_2 Subject_3 …

Sub-task_1 2s 1.5s 3s …

Sub-task_2 4s 3s 3s …

Sub-task… … … … …

Data base

‚Virtual Experiment‘ _{Subject_1 Subject_2} _{Subject_3} _…

Sub-task_7 5s 4s 6s …

Sub-task_9 3s 2.5s 2s …

(17)

Tasks in evaluation experiment 3

Short

Name Device of SubtasksCount Short Description

T1 Touch 8 Enable a checkbox in a configuration submenu of an app and leave application

T2 Touch 2 Enter a calculation into calculator (about 10 input steps)

T3 Touch 5 Record a short voice message/note (one word)

T4 Touch 7 Search radio stream by text search (6 chars)

R1 Rotary 9 Share your location (Glympse) with someone from the contact list

R2 Rotary 10 Reply to a message with a predefined short text

R3 Rotary 3 Switch off the infotainment screen

R4 Rotary 5 Search a radio stream by text search (2 chars)

(18)

Participants

Experi-ment N male/female Agemin – max; M (SD) Lefthanded Eye wareneeded km/year<10,000 km/year>20,000

1 21 11/10 45 – 64; 56 (6) 1 18 3 7

2 21 11/10 46 – 68; 59 (6) 1 17 3 8

3 21 10/11 47 – 65; 55 (4) 2 17 5 7

Subjects from the first two experiments were not allowed to participate in the third experiment.

(19)

(20)

Total Glance Time 85th Percentiles (TGT)

1% 1% 36% 19% -7% 3% 102% -5% 0 5 10 15 20 25 Se co nds estimated measured

(21)

Total Shutter Open Time 85th Percentiles (TSOT)

-8% -6% 27% 6% 7% 6% 94% -12% 0 5 10 15 20 25 T1 T2 T3 T4 R1 R2 R3 R4 Se co nds estimated measured

(22)

Single Glance Duration 85th Percentiles (SGD)

28% 12% 15% 8% -4% _-7% 6% 0% 0,0 0,2 0,4 0,6 0,8 1,0 1,2 1,4 1,6 1,8 2,0 T1 T2 T3 T4 R1 R2 R3 R4 Se co nds estimated measured

(23)

(24)

Discussion / Conclussion

• Results can be used in two ways:

1. The list of subtasks with characteristic values (TGT, TSOT, SGD) could be a low-cost, valuable tool for developers looking to include different subtasks in their app and to

estimate their distraction potential. It can be useful to make them more aware of driver distraction issues, thus, an

educational aspect.

2. Virtual experiment: determining subcomponents of a task is

somewhat subjective, it is often difficult to find an appropriate subtask (assuming it has already been tested).Further work is needed. But might be able to filter out clearly unreasonable and ill-suited apps before subject testing.

(25)

Michael Krause Graduate Research Associate Faculty of Mechanical Engineering Institute of Ergonomics Boltzmannstraße 15 D-85747 Garching Tel +49 89 289-15404 Fax +49 89 289-15389 [email protected] http://www.ergonomie.tum.de