Augmented Reality

(1)

- the course 2011

Kaj Grønbæk, Professor Center for Interactive Spaces Department of Computer Science, University of Aarhus

Email: [email protected]

Course Web: https://services.brics.dk/java/courseadmin/AR/

Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY

Plan

› Brief presentation

› About the course

› A little AR history

› Concepts and Survey

› Quick intro to getting started with ARToolkit

(2)

2

3

Augmented reality

› An aspect of ubiquitous/pervasive computing

› Focus on IT-based systems, bridging between physical and

digital worlds

› Involves IT-research, engineering, architecture and industrial

design…

› Based on context-awareness and positioning knowledge

› Based on the research in Center for Interactive spaces

Augmented Reality Kaj Grønbæk

AARHUS UNIVERSITY

4

Form of the Course

› Lectures about prototypical systems and applications

› Aprox. half of the time on Thursdays

› Active participation

› Student discussion of selected papers and tools › Small exercises to be accpted

(3)

5

Course Plan - 2011

› Week 4: Introduction to the course and Augmented Reality

concepts

› Week 5: Augmented Reality Surfaces and CyberCodes

› Week 6: AR Toolkit and examples of applications.

› Week 7: Mobile and Wearable AR

› Week 8: Position and direction based AR – incl. LAYAR

› Week 9: Augmented Paper and Electronic Paper

› Week 10: Computer augmented objects, Physical

Hypermedia, RFID and NFC

› Week 11: Tentatively: Augmented reality gaming or other

option; Conclusion and evaluation of the course.

AARHUS UNIVERSITY

(4)

4

7

SitePack mobile computing device

Augmented reality: feature tracking

› From the WorkSpace project › Integrating location based,

registration, annotation and visualization Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY 8

Visualizing hedges

Nielsen, M.B., Kramp, G., and Grønbæk, K. Mobile Augmented Reality Support for Architects based on Feature Tracking Techniques (2004). In Lecture Notes in Computer Science Volume 3038. Computational Science - ICCS 2004: 4th International Conference, Kraków, Poland, June 6-9, Part III (eds. Marian Bubak, Geert Dick van Albada, Peter M. A. Sloot, et al.). Springer-Verlag Heidelberg. pp. 921 - 928.

(5)

9

Previous EU-project WorkSPACE:

Physical Hypermedia for Architects

Augmented Reality: •  RFID tags

•  Visual ARToolkit tags

Grønbæk, K., Ørbæk, P., Kristensen, J.F., and Eriksen, M.A. Physical Hypermedia: Augmenting Physical Material with Hypermedia Structures (2003). In New Review of Hypermedia and Multimedia (NRHM). Vol 9. Taylor & Francis, Abingdon, UK (pp. 5-34)

AARHUS UNIVERSITY

BattleBoard 3D - Augmented Reality Game

AugmentedMedia Startup company

International Paper:

Andersen, T.L., Kristensen, S., Nielsen, B.W. and Grønbæk, K. Designing Augmented Reality Board Games: The BattleBoard 3D experience. In proceedings of OZCHI 2004, 22-24 November, 2004 at the University of Wollongong, Australia.

(6)

6 11

ARDressCode

International Paper Kjærside, K., Kortbek, K. J., Møller, H.H., and Grønbæk, K. ARDressCode: Augmented Dressing Room with Tag-based Motion Tracking and Real-Time Clothes Simulation, In proceedings of the Central European Multimedia and Virtual Reality Conference 2005 (CEMVRC05), 8-10 June 2005, Prague, Czech Republic .

Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY 12

Aarhus Festuge

(7)

13

Events within 2 km

Layar version of Aarhus Festival program

AARHUS UNIVERSITY

(8)

8

15

StarCatcher – GPS game

AARHUS UNIVERSITTY

(9)

17

Readings

› Azuma: Recent Advances in Augmented Reality

› Mackay: CACM 93 intro

› Mackay: Augmented Reality: Linking Real and Virtual Worlds.

A new paradigm for interacting with computers.

AARHUS UNIVERSITY

Augmented Reality versus

Virtual Reality

AR (ubicomp)

› Physical world as the basis › Bring computer capabilities into

the physical world › Embodied virtuality

VR

› Simulated artificial world is the basis

(10)

10 21

A little history

› Xerox around 90 › Weiser, Mackay,Wellner › Mackays Ariel

› EuroCODE project with AU

› Columbia University › Feiner › CACM July 93 › Sony › Rekimoto › Medialab › Ishii › DARE konference i 2000 › Helsingør Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY 24

AZUMA ET AL.

(11)

25

What is AR? (Azuma et al.)

”Definition”

› AR system supplement the real world with virtual (computer-generated) objects that appear to coexist in the same space as the real world.

An AR system has the following properties:

› combines real and virtual objects in a real environment; › runs interactively, and in real time; and

› registers (aligns) real and virtual objects with each other.

AARHUS UNIVERSITY

Enabling technologies - Displays

Types of displays

› Head worn displays (HWD)

› optical see-through, video see-through, virtual retinal display

› Handheld displays › Projection displays Problems/Challenges › Quality of display › Size and weight

(12)

12

27

Enabling technologies - Tracking and sensing › Indoor tracking

› environment prepared with tags

and sensors

› video tracking, magnetic tracking

› combined techniques works best

› Outdoor tracking

› GPS, digital compass, gyroscope

› video-based, fixed land mark,

known features Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY 28

Tracking and sensing: challenges

› Low latency

› registration errors

› Environment preparation

› overhead in tagging

› Calibration

(13)

29

Interaction

› Beyond keyboards, mouse and menus

› Gestures

› Utilize movement sensors

› Tangible interaction

› Physical tools to grab virtual objects › haptic feedback

› Audio/speech interfaces

› Audio controlled interaction

AARHUS UNIVERSITY

Visualization - challenges

› correct for location errors

› information overload in overlays

› filtering

› real time photorealistic rendering

› mediated reality extract real objects and replace with virtual › depth perception

(14)

14

31

Perceptual problems

› Latency

› E.g. Errors in location detection

› Depth perception

› Occlusion problems

› Adaption

› Users adapt their perception to AR views and have problems when they

leave the equipment

› Eye strain

› Projections on the eye, same image, diﬀerent images, stereo

Mobile applications

› Real time overlay

(15)

33

Sports and commercial applications

› Overlays with player info.

computed realtime on top of video

› Overlays of commercials

and ads on real world and tv-broadcasts. Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY

Collaborative applications

› Examples of collaborative AR systems using

› use see-through handheld displays

› seethrough head-worn displays

› A problem/challenge:

› ensuring that the users can establish a shared understanding of the

virtual space, analogous to their understanding of the physical space.

› In Studierstube, the designers attempt to overcome this problem by

rendering virtual representations of the physical pointers, which are visible to all participants

(16)

16

35

Azuma summary/comments

› Very much focus on overlay and 3D techniques inspired from

VR

› Problems/challenges are mainly graphics problems

MACKAY ET AL.

(17)

37

Wendy Mackay

› Focus on Interactive Paper rather than 3D/VR like

augmented reality

› Introduces a taxonomy relating augmented reality

technology to users, objects and environments › Gives three central examples of interactive paper

applications Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY

Mackay’s categories

(18)

18

39

Mackay’s examples

› Ariel – Handling of drawings and paper

› Video Mosaic – Storyboard editor

› Caméléon – augmented flight strips

AARHUS UNIVERSITY

40

(19)

41

Video Mosaic

› Storyboarding tool

› Paper storyboards

linked to video clips

AARHUS UNIVERSITY

Caméléon – Flight strips

› Flight strip represents one

plane

› Corresponds to a spot on the radar › Status information are written on the strips › Acts as a token/depeche to be handed over

(20)

20

Discussion

Q&A

ARTOOLKIT

(21)

45

ARToolKIT

›

Open source tool kit

›

Webcam and a PC/Mac

›

Goggles/Glasses/Monitor

›

Supports

› Make your own tags › Make your own models

› Make your own application logic

›

Implemented on iPhone e.g.

by ARToolWorks

› http://www.artoolworks.com/ http://ftp.hitl.washington.edu/research/ shared_space/download/ Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY

ARToolkit

› Source code in C++

› Callibration and registration applications

› Example applications

(22)

22

47

ARToolKit processing overview

AARHUS UNIVERSITY

48

3D models for the overlay

› Imported in VRML format

› VRML = Virtual Reality Modelling Language

› VRML 2.0 (VRML97) is a standard for exchange of 3D models

including animations

› Is exported from many 3D tools, e.g. 3D Studio MAX, Alias

(23)

AR-toolkit

How to use?

Demo

Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY

ARToolKit refs

›  AR Toolkit Homepage › (http://www.hitl.washington.edu/research/shared_space/) › HitLabNZ › (http://www.hitlabnz.org/) ›  AR Toolkit Manual › (http://www.hitl.washington.edu/artoolkit/documentation)

›  DARTDesigners ARToolKit integrated in Macromedia 2004 MX

› (http://www.gvu.gatech.edu/dart/)

› Sourceforge.net

(24)

24

51

Using ARToolKit

Important files

› Camera Calibration Data

(camera_para.dat) › Markers/Patterns (eg.

Patt.hiro)

› Multi Marker Specification

files (marker.dat) Data/Vrml_data Wrl/bud_B.dat Data/patt.hiro Wrl/bud_B.wrl Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY 52

Demo of mk_patt.exe

› Demonstrates how to register your own markers.

(25)

53

The first exercise

› Form groups and make a little ARToolkit prototype App

› Relate it to an application scenario

› Make a few tags

› Associate some objects

› Run the prototype

› Make some simple changes in the application code

› E.g. detect and react to proximity of tags

› Present in Week 6 Augmented Reality Kaj Grønbæk AARHUS UNIVERSITY

Questions

?