Audio synthesis: MIDI Digital Audio Coding/Compression. Today: Intellectual property management for digital media. What is Digital Watermarking?

(1)

ENEE408G Lecture ENEE408G Lecture--99 ENEE408G Lecture ENEE408G Lecture--99

Digital Watermarking and Fingerprinting

for Digital Rights Protection of Multimedia

 URL: http://www.ece.umd.edu/class/enee408g/

 Slides included here are based on Spring 2012 offering in the order of introduction, image video speech and audio © Copyrighted 2002 2012

image, video, speech, and audio. © Copyrighted 2002-2012.

 ENEE408G course was developed @ ECE Department, University of Maryland, College Park. Inquiries can be addressed to Profs. Ray Liu ([email protected]) and Min Wu ([email protected]).

ENEE408G Capstone -- Multimedia Signal Processing

Last Lecture



Audio synthesis: MIDI

Digital Audio Coding/Compression

L

iu ©

2002)



Digital Audio Coding/Compression

– Psychoacoustics properties used in perceptual audio coding – MPEG-1 Audio coding

b y M.W u & R. L g 

Today:

S lides (created b

y

– Digital Rights Management of Multimedia via “Watermarking”

P

ENEE408G

S

UMC

P

ENEE408G Capstone -- Multimedia Signal Processing MM Digital Rights [2]

Demands on Info. Security and Protection



Intellectual property management for digital media

– Promising electronic marketplace for digital music and movies

L

iu ©

2002)

g p g

– Advantages of digital: perfect reproduction, easy transmission, … – Napster controversy b y M.W u & R. L



Conventional encryption alone still leaves many problems

unsolved

S

lides (created

b

– Protection from encryption vanishes once data is decrypted

 Still want establish ownership and restrict illegal re-distributions

How to distinguish changes introduced by compression vs malicious

P

ENEE408G

S

– How to distinguish changes introduced by compression vs. malicious tampering?

 Bit-by-bit accuracy is not always desired authenticity criterion for MM

UMC

P

Digital Watermarking/Data Hiding in Multimedia



What is Digital Watermarking?

– Examples: Picture in picture, words in words

L

iu ©

2002)

p p

Silent message, invisible images

– Secondary information in perceptual digital media data

b

y M.W

u

& R.

L



The need of watermarking: robust vs. fragile

– Copyright protection: prove the ownership Fingerprinting: trace the source

S

lides (created

b

– Fingerprinting: trace the source

– Copy protection: prevent illegal copying Data authentication: check authenticity of data

P

ENEE408G

S

– Data authentication: check authenticity of data

 Fragile or semi-fragile watermarking

UMC

(2)

Example on Invisible and Robust Watermark

10011010 …

© Copyright …

py g

Fragile Watermark Example: Document Authentication

ENEE408G Capstone -- Multimedia Signal Processing MM Digital Rights [6]  Embed pre-determined pattern or content features beforehand

 Verify hidden data’s integrity to decide on authenticity

General Framework of Data Hiding

marked media (w/ hidden data) host media © 2002) _{(w/ hidden data)} embed embed data to be data to be hidden hidden ted by M.W u © embed embed hidden hidden compress compress 101101 … 101101 … “Hello, World” “Hello, World” 8 G Slides (crea process / process / attack attack play/ record/… play/ record/… extracted extracted

Player; or Forensic Analyzer Player; or Forensic Analyzer

M CP ENEE40 8 extract extract extracted extracted data data 101101 … 101101 … “Hello, World”

“Hello, World” test media

U

M

Basic Requirements for (Robust) Watermarking



Imperceptibility

(perceptual transparency)

Payload

W



Payload

– the amount of information that can be stored in a watermark

R b

t

b

y R. Liu & M.

W



Robustness



Security – Kerckhoff Principle

lides (created

b

– The method used to encrypt the data is known to an unauthorized party and that the security must lie in the choice of a key.

ENEE408G S

l



Blind and non-blind detection

(aka Oblivious vs Non-oblivious) – Blind detection ~ does not use the original unmarked copy

UMCP

(3)

Data Embedding by Replacing LSBs

8-’04) s earch T a lks ’9 © based on Re s a ted by M.W u © 3 1 Slides (cre a

Replace LSB with Pentagon’s MSB

U

MCP

ENEE6

3

Replace LSB with Pentagon s MSB

U

Data Embedding by Replacing LSBs (cont’d)

8-’04) s earch T a lks ’9 © based on Re s a ted by M.W u © 3 1 Slides (cre a

Replace 6 LSBs with Pentagon’s 6 MSBs

U

MCP

ENEE6

3

Replace 6 LSBs with Pentagon s 6 MSBs

U

A Simple Audio Watermark in Time Domain

=> See Lab Project 4 for details



Put message in the Least-Significant-Bits (LSBs)

– Encode a message into bits

W

 e.g., represent a character string into bits using ASCII code

– Embedder puts in LSBs of audio samples

 Repeat embedding the same bit in a few samples if needed

b

y R. Liu & M.

W

– Detector retrieves embedded bits from LSBs

 Perform majority voting if repeated embedding is used

– Repack bits into message

lides (created

b

p g



Tradeoff between perceptual quality and robustness

– Compare the embedding in 1st_{LSBs, 2}nd_{LSBs, …}

ENEE408G S

l

p g , ,



Security

– Can they see/hear your message?

UMCP

y y g

– Can other people make imperceptible change to alter your message?

A More Robust Watermark in Transform Domain

=> See Design Project 4 for details

Liu ©

2002)



Embedder:

use HAS & embed in perceptually significant freq.

Noise-like seq. Key/Seed by M.W u & R. Watermark y V’(j) = V(j) + a(j) W(j) W(j) e.g., a(j) = 0.05 |V(j)| S lides (created

f

Segmentation (frame size L)

Audio File DCT iDCT Watermarked_{audio file}

added in mid-frequency region V(j) V’(j) P ENEE408G S



Detector: determine the existence of a specific wmk

– Subtract host signal, measure similarity (via correlation), & threshold it

E t t UMC P DCT Audio File in question Extract mid-frequency coefficients + _ watermark detector Detection result Si il it V’’ V W DCT Original audio file A specific watermark Similarity measures: <V’’ V, W>, or < (V’’V) ./ a, W >, or

(4)

Discussions



Why use noise-like sequence as watermark?

– Imperceptibility

L

iu ©

2002)

p p y

– Confidentiality of the embedded data – Robustness against jamming

b y M.W u & R. L 

Imperceptibility

– Frequency domain embedding: can take advantage of known perceptual properties such as masking

lides (created

b

perceptual properties such as masking

– Can apply sophisticated HAS models to improve perceptual quality



Robustness and security

ENEE408G S

l

y

– Use “attacks” to find weaknesses and improve designs – Case study: SDMI public challenge (Fall’00)

UMCP

Digital Fingerprinting and Tracing Traitors



Leak of information poses serious threats to government

operations and commercial markets

©

2005)

– e.g., pirated content or classified document studio The Lord of Alice Bob w1 w2 Sell Sell ted by M.W u © 

Promising countermeasure:

b

tl

b d di it l fi

i t

the Ring Bob

Carl

w3

8

G Slides (crea

robustly embed digital fingerprints

– Insert ID or “fingerprint” (often through robust watermarking) to identify each user

M

CP

ENEE40

8

y

– Purpose: deter information leakage; digital rights management

provide post-delivery protection complementary to encryption

U

M

ENEE408G Capstone -- Multimedia Signal Processing MM Digital Rights [15] – Challenge: imperceptibility, robustness, tracing capability

Fingerprinting Curves

f l l f 103 ₁₀9

Fingerprinted Curve (100 control points)

U

M

( p y )

false alarm of 10-3_{~ 10}-9

( p )

Embedded Fingerprinting for Multimedia

Multimedia Document Customer’s ID: Alice Fingerprinted Copy Fingerprinted Copy Multimedia Document Customer’s ID: Alice Fingerprinted Copy Fingerprinted Copy Embedded Finger-© 2005) embed embed Digital Fingerprint 101101 … 101101 … Distribute _{to Alice} Fingerprinted Copy Fingerprinted Copy embed embed Digital Fingerprint 101101 … 101101 … Distribute _{to Alice} Fingerprinted Copy Fingerprinted Copy Finger-printing ted by M.W u © Alice Alice Bob

Bob Unauthorized Unauthorized

re re--distributiondistribution

Alice Alice

Bob

Bob Unauthorized Unauthorized

re re--distributiondistribution Multi-user Attacks 8 G Slides (crea Collusion Attack Collusion Attack (to remove fingerprints) (to remove fingerprints)

Colluded Copy



Fingerprinted doc for different users

Collusion Attack Collusion Attack (to remove fingerprints) (to remove fingerprints)

Colluded Copy



Fingerprinted doc for different users Attacks M CP ENEE40 8 Extract Extract Fingerprints Fingerprints Suspicious Suspicious Copy Copy 101110 … 101110 … Codebook Alice, Bob, … Identify Identify Traitors Traitors Extract Extract Fingerprints Fingerprints Suspicious Suspicious Copy Copy 101110 … 101110 … Codebook Alice, Bob, … Identify Identify Traitors Traitors Traitor Tracing U M

ENEE408G Capstone -- Multimedia Signal Processing MM Digital Rights [17] Copy

Copy

(5)

Collusion

Collusion--Resistant Fingerprinting of Maps

Resistant Fingerprinting of Maps

2-User Interleaving Attack 5-User Averaging Attack

8 G Slides (crea

. . .

M CP ENEE40 8 U M

• Also survive combination attacks of collusion + print + scan

Example of Anti

Example of Anti--Collusion Fingerprint Code:

Collusion Fingerprint Code:

E b d 16 bit C d f D t ti 3 C ll d O t f 20

Embed 16-bit Code for Detecting 3 Colluders Out of 20

User-1 ( -1,-1, -1, -1,1, 1, 1, 1, …, 1 ) ( -1, 1, 1, 1, 1, 1, …, -1, 1, 1, 1 ) User-4 © 2005) ted by M.W u © 8 G Slides (crea M CP ENEE40 8 Collude by Averaging Uniquely Identify User 1 & 4 Embed fingerprint via

HVS-based spread spectrum embedding in block-DCT domain

U

M

ENEE408G Capstone -- Multimedia Signal Processing MM Digital Rights [19] Extracted fingerprint code ( -1, 0, 0, 0, 1, …, 0, 0, 0,1, 1, 1 )

User 1 & 4 in block DCT domain

Case Study: Tracing Movie Screening Copies



Potential civilian use for digital rights management (DRM)

 Copyright industry – $500+ Billion business ~ 5% U.S. GDP

©

2005) py g y $5 5



Alleged Movie Pirate Arrested (23 January 2004)

– A real case of a successful deployment of 'traitor-tracing'

ted by

M.W

u

©

mechanism in the digital realm

– Use invisible fingerprints to protect screener copies of pre-release movies

8

G Slides (crea _movies

Carmine Caridi Russell friends … Internet w1 Last Samurai M CP ENEE40 8

Hollywood studio traced pirated version

U

M

http://www.msnbc.msn.com/id/4037016/

Summary



Multimedia watermarking for rights management

L

iu ©

2002)



Reading Assignment

– F. Hartung and M. Kutter: “Multimedia Watermarking Techniques”, Proc. of the IEEE, pp.1079-1107, July 1999.

b y M.W u & R. L , pp , y

– M. Wu and B. Liu, “Multimedia Data Hiding”, Chapter 10 on SDMI audio watermark challenge, preprint, 2002 (electronic handout). M Wu W Trappe Z Wang and K J R Liu: "Collusion Resistant

S

lides (created

b

– M. Wu, W. Trappe, Z. Wang, and K.J.R. Liu: "Collusion Resistant Fingerprinting for Multimedia", IEEE Signal Processing Magazine, Special Issue on Digital Rights Management, pp.15-27, March 2004. http://www.ece.umd.edu/~minwu/public paper/Jnl/0403FPcollusion IEEEfinal SPM.pdf

P

ENEE408G

S

p p _p p _ _ p



This week’s Lab session:

UMC

P

(6)

L

iu ©

2002)

For Further Explorations:

b y M.W u & R. L

p

General Formulation of Two Types of Watermarking

S lides (created b P ENEE408G S UMC P

To Explore Further: Type

To Explore Further: Type--I Additive Embedding

I Additive Embedding



Add secondary signal in host media



Representative: spread spectrum embedding

©

2002) _

_{Representative: spread spectrum embedding}

– Add a noise-like signal and detection via correlation – Good tradeoff between imperceptibility and robustness

ted by

M.W

u

©

p p y

– Limited capacity: host signal often appears as major interferer

8 G Slides (crea modulation modulation data to be hidd  X X’ = X +  marked copy 10110100 ... 10110100 ... M CP ENEE40 8 hidden X original source < X’ + noise,  > = <  + (X + noise),  > U M

ENEE408G Capstone -- Multimedia Signal Processing MM Digital Rights [24] < X’ + noise - X,  > = <  + noise,  >

Type

Type--II Relationship Enforcement Embedding

II Relationship Enforcement Embedding



Deterministically enforcing relationship

– Secondary information carried solely in watermarked signal

lookup table mapping

0 1 0 1 … 0 1 1 0 …

8

G Slides (crea



Representative: odd-even embedding

– No need to know host signal (no host interference)

even “0” odd “1” M CP ENEE40 8

– High capacity but limited robustness

– Robustness achieved by quantization or tolerance zone

U

M

E.g. enforcing black pixel# per block to odd/even to hide data in binary image

Type

Type--II Relationship Enforcement (cont’d)

II Relationship Enforcement (cont’d)



General approach:

– Partition host signal space into sub-regions

©

2002)

 each region is labeled with 0 or 1

 marked sig. is from a region close to orig. & labeled w/ the bit to hide

Secondar info carried solel in X’

ted by

M.W

u

©

– Secondary info. carried solely in X’

 difference (X’-X) doesn’t necessarily reflect the embedded data

8 G Slides (crea mapping mapping { b} data to be hidden _X X’= f( b ) marked copy 1 or 0 1 or 0 M CP ENEE40 8 

Advanced embedding:

host sig. U M

ENEE408G Capstone -- Multimedia Signal Processing MM Digital Rights [26] – Combining the two types with techniques suggested by info. theory