Title Slide
Chahab Nastar, CEO
Vienna, 20 September 2005
Industrial Challenges for
Agenda
CBIR – what is it good for?
Technological challenges of CBIR
Evaluating CBIR
About LTU technologies
Q&A
CBIR... what is it good for?
Historically, (generic) CBIR is a cool technology...
- convergence of information retrieval and computer vision - « user in the loop », link to web search engines
- « query by example » approach
- feature extraction, fast search, relevance feedback Æ a lot of academic papers
... in search of a useful application in the « real » world
- worst-case scenario for entrepreneurship: technology preceding market traction
CBIR’s redemption
a powerful paradigm for object/scene recognition
finding a series of images of the same scene/object
user is in the loop
CBIR in the « real » world
Known world (reference)
B
Unknown world (query)
clone images duplicate images
similar images
User scenarios
Verification:
1:1
- unrelated to information retrieval
Identification:
1:N
- target search
- applications: stolen art, counterfeiting, media monitoring
Investigation:
q:N
- find common ground between images
- applications: child exploitation, IT forensics, intelligence
Classification:
A K∈
LTU’s technology – generalized CBIR
Input
Any digital image format, e-mail attachement, video frame,… Output Similarity Retrieval Clone Detection
Categorization face 100%
indoor 80%
Image Database / Knowledge Base
Visual Feature Extraction
color, shape, texture, arrangements, moments,…
Real-time Analysis
Image “DNA”
1
Technological challenges of CBIR
Metric Problem Application Visual/semantic distance Similarity:A ~ B ? Visual retrieval
Matching distance Matching:
A = B ? « Clone » detection
Hamming distance Classification:
A K ? Image annotation
∈
Technological challenges of CBIR
Metric Problem Application Visual/semantic distance Similarity:A ~ B ? Visual retrieval
Matching distance Matching:
A = B ? « Clone » detection
Hamming distance Classification:
A K ? Image annotation
∈
Clone Detection
Image clones same visual content, different graphical rendering (typically Photoshop manipulation)
cropped
reencoded
original old B&W negatized bad scan
Technological challenges of CBIR
Metric Problem Application Visual/semantic distance Similarity:A ~ B ? Visual retrieval
Matching distance Matching:
A = B ? « Clone » detection
Hamming distance Classification:
A K ? Image annotation
∈
Technological challenges of CBIR
Metric Problem Application Visual/semantic distance Similarity:A ~ B ? Visual retrieval
Matching distance Matching:
A = B ? « Clone » detection
Hamming distance Classification:
A K ? Image annotation
∈
Annotation by supervised learning
Classifier bank A A B B C C D D 11 00 11 11 00 11
Fusion module
Keyword Keyword
Finite
Finite vocabularyvocabulary
Evaluation of CBIR depends upon
Image Content
- Scenes/objects represented - Image Quality
- Domain-specific knowledge
User scenario
- identification
benchmarking database? - investigation
using a series of images - classification
Usage scenario
- Search Acceptable max rank of « retrieved images »? precision or recall?
Evaluation within a customer project
Large database with Ground Truth
- provided by customer or- put together by LTU
User scenario & usage scenario taken into account
- in most mission-critical scenarios, maximizing recall is the main goal
The LTUtech dataset
LTU has collected (mostly over the web) more than 80,000
images of 267 visual categories
A visual category is a set of examples of an object
- animals, CD covers, gloves, watches etc.
An average of 300 images per visual category
The total database size is 1.3 GB
The images vary in quality, size, format
We do not known the copyright of the images
The database is shared with the WP3 members of the
MUSCLE network
Golden Award for Technology Golden Award for Technology
The world’s top 100 innovators
MIT Technology Review
The world’s top 100 innovators
MIT Technology Review
European Information Society
Technologies Prize
European Information Society
Technologies Prize Europe’s Top 100 Companies
The Red Herring
Europe’s Top 100 Companies
The Red Herring
About LTU Technologies
Founded: 1999 from our group’s research atINRIA (imedia research group)
Mission: Bringing image/video indexing/retrieval to the market
Headquartered: Paris & Washington D.C.
Value Prop: Law Enforcement & Intelligence
Providing tools to automatically “mine” information from large, unknown sets of images Data
Collection
Seized HD
Internet
P2P
Database of images w/ case information Database of suspect
Images
Content Management
Key-Clients (1)
Government, Police, International agencies
Investigations
Child protection, Stolen art
Computer forensics
Counterfeiting, ID theft
Intelligence and monitoring
Value Prop: Media & Intellectual Property
Providing efficient access to images and tools for monitoring their usage
1 Search
2 Monitoring
3 Keywording
Manage Distribute
E-Mail Internet
Key-Clients (2)
Media and IP protection
Image bank management
e-commerce, media
Patent and design protection
Search, classification of designs
Online IP protection
Logos, visual assets
Partners
DAM software, text search engines
Summary
Challenges in bringing CBIR to the market
- mission-critical applications
Evaluation is case-by-case
- domain-specificQuestions & Answers
Chahab Nastar CEO
Paris, France
cn AT LTUtech.com
Chahab Nastar CEO
Paris, France
