Augmented Reality
Applications at VTT
Prof. Charles Woodward
VTT Technical Research Centre of Finland
Augmented Reality (AR)
Mixes virtual objects with view of real world C.f. Virtual Reality (completely virtual worlds) Properties: real-time, interactive, 3D registration <-> Challenges: tracking, accuracy, speed, mobile
Introduction
Related technologies
Augmented Virtuality Mediated Reality Diminished Reality, … <->Mixed Reality (MR)Reality /
Video image
Virtual Reality /
Computer graphics
Augmented
Reality
VTT Technical Research Centre of Finland
The largest research instute in Northern Europe Provides high-end technology solutions and innovation services
Figures
3167 employees Annual turnover 292 M€Funding
1/3 industry 1/3 government1/3 public sector (EU, Tekes)
Background
VTT
Basic
research
Applied
research
Development
work
AR at VTT
Mixed Reality & 3D Tracking teams
Established in 2000 / 2004, based in Espoo AR/MR core technology & high-end applications Supporting / application teams around VTT
Leading edge technology
First in the world with various AR applications Providing forerunner position for customers Licensing for independent developers Seewww.vtt.fi/multimedia
Networks
Multi-disciplinary VTT wide collaboration Strong industry relations and experience International networks, incl. EIT ICT Labs, Columbia Univ., HIT Lab NZ
Early Applications:
Games & Entertainment
CamBall
SymBall
Vappunaamari.com
TelePet StereoGames
Consumer Applications: Print & Advertising
Nestle bunny on cocoa package Dibidogs in Katso &
AR children’s book
DNV (Det Norske Veritas): Technology Outlook 2020 Veijarit & Vares movie
trailers in Seiska
Sparkly in Elle magazine
Gorba phone on Bit Bang book
Interior Design
VividAR
AR interior design system by VividWorks, based on VTT technology Free download, various furniture retail companies around the world
Incl. Photorealistic rendering for AR
Virtual and Mirror Worlds
Mixed Reality teleconferencing between real people and Second Life avatarsAugmented Reality <-> Augmented Virtuality Implemented in co-operation with Nokia & IBM Corp. (USA)
Industrial Applications: Production Sites
Augmented maintenance, operation, training Plant asset management: Simantics Mobile Robot simulations in real environments
Partners: Wärtsilä, Metso, Valtra, Cargotec, Fortum, ..
Augmented Assembly
Pilot case
Valtra (tractor company) Hydraulic block (200 pcs)
3D puzzle
Data Pipeline
3D CAD STEP DFA CG ARSatellite assembly
Remote training
Manufacturing design
Maintenance of
heavy machinery
Industrial plant
maintenance
Thales Alenia Space NEM Solutions TNO
Tecnatom
Metso Minerals
EU FP7 Project ManuVAR
(Coordinator Boris Krassi, VTT Tampere)ARScaleModel
ARWebCam
Building Applications – Case Digitalo
ARFloorPlan
AR-PDA system 2003
Client/serverMarker based
UMPC stand-alone system 2006
Interactive placement of model Optical flow / feature based tracking“Google Earth on Earth” 2007
Model placed in Google Earth User’s location provided by GPS Automatic scale & perspective Interactive positioningLimitations, e.g.
Google Earth map inaccuracy
Collada format requires conversions via 3DS Max, ArchiCAD etc. Conversions often loose features, bad support for large models etc.
Mobile AR – Early Work
Mobile AR – Pilot Case
New Skanska offices in Helsinki
Devices: laptop PC, UMPC, data glasses (Sep 2009) Photorealistic visualization of architectural model (May 2010) Augmenting 4D plans during construction work (Oct 2010) Comparison with partially ready building (May 2011) Client/server solution -> mobile phones (Aug 2011)
Mobile AR – Real Use Case
AR visualization of architectural plans
New hotel being planned at historical Billnäs Ironworks sitehttp://www.billnas.fi/en/
Mobile AR visualization was presented to group of 20 city council members and civil engineers Devices: laptop PC (free viewing positions) and five N900 phones (predefined “hot spot” locations) Most positive feedback from audience: viewing the models on site helped to understand plans
Mobile AR – City Planning
City development project in Helsinki Jätkäsaari
Incl. KämpTower, tallest building in Helsinki (33 floors)
Mobile AR visualization presented to Helsinki city council members, March 2012 Windows tablets, operated by architects & other users
Mobile AR – Public, Citizens
AR visualization of wind generator park
Wind generator park being planned at Pörtom (Finland) Tour around the site, to show AR plans for local people, farmers Windows Phone 8, features incl. animations, masking by sky line etc. Two further projects under way with VindIn (Sweden)
World Browsing
Challenge
Accurate tracking of mobile device (both location and rotation) for augmenting the camera view with additional info, e.g. mobile advertising
Solution
Use offline collected images and 3D point cloud data of the environment to track the mobile device Optimized for mobile phones (Nokia)
Also, Mobile CityGuide, developed for Sanoma News (local events, Metro newspaper content)
Tourist Attractions
Futuristic History
Augmenting historical photos at real locations
Showing how city used to look in the good old times
Co-operation with Finnish National Museum & others
First pilot taking place in Turku 2014
Mobile Indoors AR
Project DigiSpaces
Mobile AR visualization and interaction with Building Information Models (BIM) & service book
Applications: building's life cycle services, e.g. maintenance & repair
Partners: Skanska, Granlund, Pöyry, Tekla, Solibri, Nokia / Quuppa
Functionality
Navigation to service targets Highlighting targets in camera view Opening targets by clicking in camera view “X-ray vision”, seeing targets behind walls etc.
Tracking
Accurate 3D tracking based on 3D point clouds, obtained from photos
Managing & updating several point clouds Supported by accelerometer, gyro-compass
Core Technology: Tracking
Marker based
Accurate & fast marker pose estimation
Marker fields, defined manually, or autocalibrated Features: optical flow tracking, hiding of markers, etc.
Markerless
Tracking of 2D templates, 3D features, point clouds 6 DOF tracking, using 3D model based initialization ISMAR tracking competition: tied winner (2010), third (2011) First competitor ever with mobile phone solution (2010)
ALVAR Subroutine Library
”A Library for Virtual and Augmented Reality”
Marker based and markerless tracking for AR Download atwww.vtt.fi/multimedia/alvar.html
Modules
Desktop (W indows / Linux … Mac) –Open Sourced ! Mobile (iOS, Android, Symbian, Maemo … Lumia) Web (Flash, Silverlight)
Rendering engine (Windows & mobile devices) Dassault VirTools plug-in
Users
OEM e.g. Vuzix (data glasses), VividWorks (AR interiors) Research partners Columbia Univ. (Goblin XNA / Microsoft) Open Source users ROS (Robot Operating System), … Over 3000 others, < 100 new downloads per month
State-of-the-Art
Point cloud based 3D tracking
Point cloud created e.g. from images of target Properties: accurate, fast, flexible, robust
Licensing by Inglobe Technologies (Italy) / AR-Media
Single solution to handle wide range of applications
Print media <-> 3D objects <-> Indoors <-> Outdoors See exampleshttp://www.youtube.com/user/inglobe
Contact:
Prof. Charles Woodward, VTT
http://www.vtt.fi/multimedia
