TM
Liberty Mutual Research Institute for Safety
Liberty Mutual Research Institute for Safety
Feasibility of an Augmented
Reality-Based Approach to
Driving Simulation
Matthias Roetting (LMRIS) Thomas B. Sheridan (MIT AgeLab)
Feasibility of an Augmented
Reality-Based Approach to
Driving Simulation
Matthias Roetting (LMRIS) Thomas B. Sheridan (MIT AgeLab)
International Symposium
New Transportation Technology for Older People
September 23-24, 2003 at MIT
International Symposium
New Transportation Technology for Older People
September 23-24, 2003 at MITTM
LM Research Institute
LM Research Institute
Center for Safety Research Center for Safety Research Center for Disability Research Center for Disability ResearchMission
To differentiate Liberty Mutual Group as the market leader by performing and supporting business-relevant research in injury prevention and disability reduction.
Mission
To differentiate Liberty Mutual Group as the market leader by performing and supporting business-relevant research in injury prevention and disability reduction.
Liberty Mutual Research
Institute for Safety
Liberty Mutual Research
Institute for Safety
Strategy
Conduct research that is business-relevant in a peer-reviewed
environment.
Strategy
Conduct research that is business-relevant in a peer-reviewed environment. 100 20 30 40 50 60 70 80 90 100 86 87 88 89 90 91 92 93 94 95 96 97 98 99 '00 '01 '02 Conference papers Published Accepted
TM
Transportation Research
Transportation Research
Distracted Drivers Instrumented Test Track In-Vehicle Technology “Heads-Free” Eye Tracker Impaired Drivers Age of Drivers Feedback to DriversDistracted and Impaired Drivers:
New In-Vehicle Technology as Cause
and Cure
Distracted and Impaired Drivers:
New In-Vehicle Technology as Cause
and Cure
TM
Research of Driver Responses
Research of Driver Responses
AR- Simulation
AR- Simulation
Real Car,
Public Road
Real Car,
Public Road
Real Car,
Closed Track
Real Car,
Closed Track
Vision-Only
Simulator
Vision-Only
Simulator
Moving Base
Simulator
Moving Base
Simulator
TM
Basic Idea and Advantages
Basic Idea and Advantages
Basic Idea
– Subject drives actual vehicle in an obstacle-free area – Virtual visual hazards are displayed
Advantages
– No motion simulation – Perfect vestibular cues – The road scene can be
100% virtual (as in conventional driving simulators)
a combination of real environment and superposed virtual hazards
– There are multiple ways to display road scene and to combine real and virtual images
Basic Idea
– Subject drives actual vehicle in an obstacle-free area
– Virtual visual hazards are displayed
Advantages
– No motion simulation
– Perfect vestibular cues
– The road scene can be
100% virtual (as in conventional driving simulators)
a combination of real environment and superposed virtual hazards
– There are multiple ways to display road scene and to combine real and virtual images
TM
Components
Components
Restricted driving area
Instrumented vehicle
Display to driver (of virtual or combined real-virtual scene)
Means to measure vehicle and head position (to compensate virtual image for vehicle or head motion)
Software and computers
Restricted driving area
Instrumented vehicle
Display to driver (of virtual or combined real-virtual scene)
Means to measure vehicle and head position (to compensate virtual image for vehicle or head motion)
TM
Vehicle Instrumentation
Vehicle Instrumentation
Second brake
Car parameters from electric and electronic system
Hand wheel angle
GPS (WAAS enabled)
Lane Tracker
3-axis accelerometer
Orientation sensor
Head-free eye tracker
2 PC-based video recording systems Power on/off Speed Brake Drive indicator
Reverse (back up)
Turn signals left/right
Running lights
High beam
Horn
Wipers
TM
Optical-See-Through
Optical-See-Through
Scene generator Scene generator Optical combiners Optical combiners Head locations Head locations Head tracker Head tracker Monitors Monitors Real world Real world Graphic images Graphic imagesTM
Video-See-Through
Video-See-Through
Head tracker Head tracker Real world Real world Scene generator Scene generator Head locations Head locations Video compositor Video compositor Graphic images Graphic images Video of real world Video of real worldTM
Video-See-Through
Video-See-Through
Adapted from Azuma, R.T. (1997). A Survey of Augmented Reality. Presence, 6
Real world Real world Scene generator Scene generator Head locations Head locations Video compositor Video compositor Graphic images Graphic images Video of real world Video of real world Marker detection Marker detection Combined video Combined video
TM
Advantages / Disadvantages
Advantages / Disadvantages
+
Optical markers allow accurate superposition+
Insensitive to movement of HMD in relation to head+
Occlusion possible-
Reduced FOV-
Overlay requires more time to compute-
Parallax between camera and eye-
User is “blind” in case of power/system failure+
Simple system+
Undiminished resolution of “real world”-
Occlusion hardly possible-
High calibration demands-
Delay between real and virtual image-
Luminance differences between real and virtual image-
Optical combiner reduces light intensityVideo-See-Through
Optical-See-Through
TM
Head Mounted, Video-S-T
TM
Vehicle Mounted, Video-S-T
TM
AR Scene Generation
AR Scene Generation
Virtual
Obstacle
Virtual
Obstacle
Camera
Camera
Display
Display
Real World Scene
Real World Scene
Real Scene +
Generated Object
Real Scene +
Generated Object
TM
Experiments
Experiments
Experiment 1, purpose:
Compare the degree of control of the car for an
easy and difficult driving task with direct view,
HMD and VMD (without augmentation)
Experiment 2, purpose:
Test the fidelity of augmentation by comparing an
easy and difficult driving task with direct view,
HMD and VMD (hazard objects are real for direct
view and virtual for HMD and VMD)
Experiment 1, purpose:
Compare the degree of control of the car for an
easy and difficult driving task with direct view,
HMD and VMD (without augmentation)
Experiment 2, purpose:
Test the fidelity of augmentation by comparing an
easy and difficult driving task with direct view,
HMD and VMD (hazard objects are real for direct
view and virtual for HMD and VMD)
TM
Potential Applications
Potential Applications
Research on human response as a function of
– safety enhancing systems
– distractions
– impairments
drugs
fatigue
– etc.
Vehicle development (where actual prototype
vehicle can be used)
Driver training
Research on human response as a function of
–
safety enhancing systems
–
distractions
–
impairments
drugs
fatigue
–
etc.
Vehicle development (where actual prototype
vehicle can be used)
TM
Outlook
Outlook
Build system with
optical-see-through
HMD and HUD
Use VMD in
combination with eye
tracker to simulate the
effect of vision defects
and research the
resulting changes in
driving performance
(e.g., color blindness,
glaucoma)
Build system with
optical-see-through
HMD and HUD
Use VMD in
combination with eye
tracker to simulate the
effect of vision defects
and research the
resulting changes in
driving performance
(e.g., color blindness,
glaucoma)
Center for Perceptual Systems Space Variant Imaging Web Site at the UTexas
TM
