06.12.2013 1
Visual and mobile Smart Data
06.12.2013
Copyright © 2013 Augmented Vision - DFKI 1
Didier Stricker
Department Augmented Vision @ DFKI
• Head: Didier Stricker
• Founded in July 2008
• 30 fulltime researchers
• 3 strongly connected research areas
Computer Vision & Video Analytics Body Sensor Networks & Sensor Interpretation Augmented Reality, Visualization & HCI 2
Big Data Taxonomy
3 IDC. IDC’s Worldwide Big Data Taxonomy,2011.
[Online] Available from:
http://www.idc.com/getdoc.jsp?containerId=23 1099 [Accessed 9th July 2013].
Outline
•
Big Visual Data
1. New Media (hyper videos, video archive,…) 2. Security – tracking and investigation
3. E-Commerce
4. 3D scene reconstruction
•
User Physical Activity Monitoring and Geo-Information-System (GIS)
1. Complexity and personalization
2. Streaming and visualization into GIS
4 Scalability of current solutions ?
To develop novel image/video understanding approaches assuming „Big (Visual) Data“?
Scalability of streaming solution GIS and „Big (Sensor) Data“
Outline
•
Big Visual Data
1. New Media (hyper videos, video archive,…) 2. Security – tracking and investigation
3. E-Commerce
4. 3D scene reconstruction
•
User Physical Activity Monitoring and Geo-Information-System (GIS)
1. Complexity and personalization2. Streaming and visualization into GIS
5 Scalability of current solutions ?
To develop novel image/video understanding approaches assuming „Big (Visual) Data“?
Scalability of streaming solution GIS and „Big (Sensor) Data“
Object detection/retrieval
Reference region in one image
?
-7-
Object detection
•
Problem
Full exhaustive search of the best region in the complete image at
different scales. Example: 70.000 regions at 19 scales for an
image 320x240: 15 secondes
-8-
-9-
-10-
Error in pixel to the Ground Truth region
Hypervideos
13
●
Adding informations to objects in videos
Images in E-commerce
•
Images are required for
• E-Commerce• Printing media
• Shop planning
•
Required Functions
• Finding / retrieving images
• Image comparison
• Extracting meta-data
• Ingredient
Scalable solution for extraction of text in natural images
Detected text area
Input image Top and bottom
line
Normalized text area
„Peacocks“
Large scale modeling
18
High-quality camera
•
100 Million Pixels
•
Spherical images
•
High Dynamic Range
Towards Giga-Pixel image-processing
Fritz-Walter-Stadion
Input images
Structure From Motion
Dense pointcloud
Calibrated cameras 76 Sparse points 803,231 Reconstructed points 240,101,306 Measures 215m x 170 m
Pagani et al.:
Dense 3D point cloud generation from multiple high resolution spherical
images
Outline
•
Big Visual Data
•
Large image and 3D reconstruction
•
Video Content Analytics
•
User Activity Monitoring and Geo-Information-System (GIS)
•
Activity recognition: complexity and personalization
•
Streaming and visualization into GIS
21/11/2013
Two different physical activity (PA) categories
12/6/2013
DFKI 28
aerobic or endurance
To promote cardio-vascular health
strength or stretching
To improve or maintain
strength/balance
• Intensity • Duration • Type • Intensity • Technique • Quality• Global activity profile
• Monitoring over active time of the day
• Reduced sensor setup
• Exercise prescription compliance
• Monitoring over one exercise session (< 1 h)
• Full body motion tracking!
(Personalized) physical activity monitoring
Everyday life activity monitoring
•
Estimating intensity of general activities,
15 activities assigned to 3 intensity classes:
–
Light intensity activities (< 3.0 METs)
lie, sit, stand, drive car, iron, fold laundry, clean house, watch TV, computer work
–
Moderate intensity activities (3.0 – 6.0 METs)
walk, cycle, descend stairs, vacuum clean, Nordic walk
–
Vigorous intensity activities (> 6.0 METs)
run, ascend stairs, rope jump, play soccer
ISWC 2013 31
ConfAdaBoost.M1 III
weak learner returns
the confidence of the
classification estimation
the more
confident the weak
learner
is in an instance's correct
classification / misclassification
the more that instance's weight
is reduced / increased
the more confident the weak
learner is in a new instance's
prediction, the more it counts
in the final combined classifier
A new approach
32 ●
Test on the UCI database
Personalization
Basic idea:
• Everyone is different!
• Record your own data for a given activity over only one minute
• Record about 6 out of 15 activities
• Re-arrangement of the classifiers (on
the phone)
• Improve your recognition score
21/11/2013 33
Lying
Walking Running Cycling Nordic
walking
Other Sitting Standing Summary
123
1 2 3
Alerts
Cloud Service
(e.g. SOS)
Visualize data with
GeoVisualizer
Smart phone
to track internal and
TrackMe – Overview
•
Tracks, logs, and uploads (offline and online)
of data from sensors (internal and external)
•
Acts as a data logger, records data, and
calculates (low and high-level) information
even when not connected to the internet /
cloud
•
Retrieves data via Bluetooth from external
sensors
•
Retrieves data from available internal sensors,
e.g. accelerometer, GPS, Gyro, Light, Audio
•
Computing the following low and high-level
information
•
Push data manually to the cloud from within
the TrackMe app or automatically upload for
real-time monitoring
GeoVisualizer – Overview (cont.)
40
•
Developed in the European project SUDPLAN
(
www.sudplan.eu
)
•
Different end-user requirements
(e.g. data format, visualization, color mapping,...)
•
Easy-to-use toolkit
•
Licensed as open-source application under
the LGPL version 3.0
•
Latest stable release 3.2.0
Air-Quality Results
SUDPLAN
Sustainable Urban Development Planner for Climate Change Adaptation
Continuous real-time analysis and
visualisation
Outlook
•
Current focus of research
•
Video / image analysis: large scale application
•
Actvitiy monitoring: long term autonomy
•
Out interest in this workshop
• Learn from experience in other field (bio-informatics, )
06.12.2013 43
http://av.dfki.de/
Thank you for your attention!
DFKI GmbH
Department Augmented Vision Trippstadterstr. 122
D-67663Kaiserslautern
06.12.2013
Copyright © 2013 Augmented Vision - DFKI 43
Didier Stricker