Multimedia Data Hiding

(1)

Multimedia Data Hiding

Min Wu

Dept. of Electrical Engineering

Committee: Profs. B.Liu, P.Ramadge, S.Kulkarni

FPO 4/01

Electrical Engineering Dept. Princeton University 2

Introduction

!

Digital Watermarking / Multimedia Data Hiding

– Hide secondary data in digital image/video/audio/3D

!

Uses of hidden data:

– ownership verification, alteration detection

– access control, annotation, side info. delivery

!

Issues and challenges:

– imperceptibility, robustness & security,

capacity

– tradeoff between the conflicting

requirements

Robustness Robustness Capacity Capacity Imperceptibility Imperceptibility FPO 4/01

General Framework

marked media

(w/ hidden data)

embed

data to be

hidden

original

media

compress

process /

attack

extract

play/ record/…

extracted

data

player

101101 …

“Hello, World”

101101 …

“Hello, World”

FPO 4/01

Key Elements of Data Hiding

!

Perceptual model

!

Embedding one bit

!

Multiple bits

!

Uneven embedding capacity

!

Robustness and security

!

What data to embed

Physical Layer

“

how to embed one or multiple bits?”

Upper

Layers

uneven capacity equalization

error correction

security

(2)

FPO 4/01

Thesis Outline

"

Fundamental issues and solutions

#

embedding strategy classification and capacity issues

#

handling uneven capacity

#

modulation/multiplexing techniques for hiding multiple bits

"

Algorithm and system designs

#

binary images

#

image authentication

#

video copy/access control and fingerprinting

#

applications in video communication

"

Attacks and countermeasures

#

“innocent tools” – block replacement and double-capturing

#

countermeasure against rotation/scale/translation

#

robustness & security analysis on SDMI audio watermarking

FPO 4/01

Algorithm and System Designs

"

Demonstrating solutions to fundamental issues

FPO 4/01

Data Hiding in Binary Image

!

A simple yet important class of images

– scanned documents, drawings, signatures

!

Challenges

– little room for “invisible” changes

– uneven distribution of changeable pixels

Clinton electronically signed

Electronic Signatures Act

-Yahoo News 6/30/00 http://www.whitehouse.gov/ media/gif/bil.gifas of 7/00

E-PAD (InterLink Electronics)

FPO 4/01

Example

-

1: “Signature in Signature”

– Annotating digitized signature with content info. of the signed document

(Finkelstein - Princeton U.)

(3)

FPO 4/01

Our Approach

!

Block-based pixel-domain method

– hide a fixed number of bits in each block

– extract hidden data without the use of original copy

!

Three issues

– determine which pixels to flip for invisibility

– embed data in each block using flippable pixels

– handle uneven embedding capacity via shuffling

Robustness is not a major requirement for authentication and annotation applications.

FPO 4/01

Preserve Visual Quality

!

Assign flippability score to each pixel

– determine how noticeable the flipping of a pixel is

– based on smoothness and connectivity

– Hierarchical

!

Sort pixels in each block according to the scores

– flip high-score pixels with high priority

(a) (b)

FPO 4/01

Embedding Mechanism

!

Extracting data without original image

– hard to directly encode data in flippable pixels

#

flippability may change after encoding

!

Our approach

– manipulate flippable pixels to enforce block-based property

#

enforce the total number of black pixels to be odd/even to hide

1 bit / block, or use more general mapping

#

incorporate quantization or tolerance zone for robustness

# of black pixel per blk

2kQ (2k+1)Q (2k+2)Q (2k+3)Q

odd-even mapping

lookup table mapping

0 1 0 1

… 0 1 1 0 …

FPO 4/01

Pixels with high flippability score are shown in the images.

Shuffling for Binary Image

!

Uneven distribution of

flippable pixels

– most are on rugged

boundary

!

Embedding rate

(per block)

– variable:

need side info.

– constant:

require larger blk

!

Random shuffling

equalizes distribution

– embed more bits

– enhance security

00 5 10 15 20 25 30 35 40 45 50 0.05 0.1 0.15 0.2 0.25

embeddble coeff. # per block (signature img)

p o rt ion of b loc k s before shuffle after shuffle

Important !

(4)

FPO 4/01

changeable pixel/coeff.

unchangeable pixel/coeff.

S

balls in total

n = pS

blue balls

Analysis of Shuffling

– Mean follows hypergeometric distribution

. . .

q

balls

N = S/q

blocks

pick w/o replacement

m

r~ # of blocks each having rblue balls out of qballs

FPO 4/01

Electrical Engineering Dept. Princeton University 14 0 5 10 15 20 25 30 35 40 0 0.05 0.1 0.15 0.2 0.25

# of flippable pixels per block (signature img)

p o rt io n of b loc k s ( x 10 0% ) before shuff simulation mean simulation std analytic mean analytic std before shuffle std after shuffle mean after shuffle

Compare Analysis with Simulation for Shuffling

Simulation: 1000 indep. random shuff.

q = 16 x 16 S = 288 x 48 N = S/q = 18 x 3 p = 5.45% std after shuffle mean after shuffle

before shuffle 0.0010 0.0010 5.56x10-3 % 5.81x10-3 % 5.56% m2/N (2ndbin) 0 3.79x10-4 0 % 7.77x10-4 % 1.85% m1/N (1stbin) 0 9.78x10-5 0 % 5.16x10-5 % 20.37% m0/N (0th_bin) simulation analysis simulation analysis FPO 4/01

Example

-

1: “Signature in Signature”

– Annotating digitized signature with content info. of the signed document

(Finkelstein - Princeton U.)

Each block is 320-pixel large, 1bit / blk.

FPO 4/01

Example

-

2: Document Authentication

– Embed pre-determined pattern or content features beforehand

– Verify hidden data’s integrity to decide on authenticity

(f) alter (a) (b) (g) after alteration (e) (c) (d)

(5)

FPO 4/01

Robustness vs. Capacity

!

Blind/non-coherent detection

~ original copy unavailable

!

Single robustness-capacity setting

– over-estimates and/or under-estimates actual noise

– not all embedded data are equally important

Robustness Robustness Capacity Capacity Imperceptibility Imperceptibility

stronger noisenoise weaker

-15 -10 -5 0 5 10 15 20 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 10log10(E 2/σ2) (dB) Cap a c it y C ( b it s /c h . u s e)

Capacity of Type-I (host=10E) and Type-II AW GN ch. (wmk MSE E2 )

Type-I (C-i C-o, blind detection) Type-II (D-i D-o)

-4 -3 -2 -1 0 1 0 0.02 0.04 0.06 0.08 0.1 grayscale/color image/video FPO 4/01

Experimental Results

Video Examples

Level-1: high capacity Level-2: high robustness embed. rate robustness embed. rate robustness Notes 60-frame 352 x 240 flower garden sequence 640 bits (91 char.) 132 bits (18 char.) 660-frame 352 x 240 concatenated sequence 3032 bits MPEG-2 4.5Mbps; frame dropping _{1266 bits}

MPEG-2 1.5Mbps; frame dropping also embed control info.

avg. chunk size = 6 frames

FPO 4/01

Video Example

– 1st & 30th Mpeg4.5Mbps frame of original, marked, and their luminance difference – human visual model for imperceptibility: protect smooth areas and sharp edges

FPO 4/01

Robust Video Data Hiding

!

Embedding domain

– partition video into segments of similar consecutive frames

– for each frame of a segment, embed same data in block-DCT domain

– embed segment index to detect frame jitter

!

Multi-level embedding

– light processing

⇒

data extractable from just a few frames

– severe processing

⇒

extractable by processing more frames

!

Uneven embedding capacity

– within a frame: constant embedding rate per region & shuffling

– between frames: embed # of hidden bit per frame as side info.

!

Modulation/Multiplexing techniques

– “TDMA”, “CDMA”, orthogonal/bi-orthogonal modulation

embed bi & (i mod M)

seg. i seg. i+1 embed bi+1 & (i+1 mod M)

(6)

FPO 4/01

Attacks & Countermeasures

FPO 4/01

Watermark Attacks: What and Why?

!

Attacks: intentionally obliterate watermarks

– remove a robust watermark

– make watermark undetectable

(e.g., miss synchronization)

– uncertainty in detection

(e.g., multiple ownership claims)

– forge a valid (fragile) watermark

– bypass watermark detector

!

Why study attacks?

– identify weaknesses

– propose improvement

– understand pros and

limitation of tech. solution

To win each campaign, a general

should know both his troop and

the opponent’s as well as possible.

----

Sun

Tzu

,

The Art of War,

The Art of War, 500 B.C.

500 B.C.

FPO 4/01

“Innocent Tools” Used by Attackers

!

Recovery of lost blocks

– for resilient multimedia transmission of JPEG/MPEG

– good quality by edge-directed interpolation:

Jung et al; Zeng-Liu

!

Remove robust watermark by block replacement

edge estimation

edge-directed interpolation

FPO 4/01

!

Attack effective on block-DCT based spread-spectrum watermark

marked original (no distortion)

JPEG 10%

after proposed attack

JPEG 10%

w/o distort

Interp.

w/ orig

34.96

138.51

6.30 w/o orig

12.40

19.32

4.52

512x512 lenna Threshold: 3 ~ 6

(7)

FPO 4/01

S

ecure

D

igital

M

usic

I

nitiative Challenge

!

International consortium

~ 180+companies/organizations

– currently pursuing watermark based solution for access and

copy control on digital music

!

Public challenge

( 9/15-10/8/2000 )

– attacks on four robust watermark technologies

!

Non-traditional research values

– reveal real industrial problem and state-of-art technologies

– present an emulated rivalry environment for better

understanding on audio watermarking

– lead to a few research problems

FPO 4/01

SDMI Challenge Setup

•Obtained From SDMI •Job for “Attackers”

•Black Box (unknown)

“Watermark

Found”

Detect

Any

Marked Audio

Embed

Watermark

(special signal)

Sample-1

(original)

Sample-2

(marked)

“Watermark

NOT

Found”

Attack

Detect

Sample-3

(marked)

Sample-4

(attacked)

GOAL

FPO 4/01

Observation on One SDMI Watermark

– Difference between original and marked samples given by SDMI

FPO 4/01

Learning from SDMI Challenge

!

Our successful attacks

– blind attacks:

warping, jittering

– attacks based on studying orig.-marked pairs

#

deliberate filtering / subtraction / randomization

!

Noteworthy issues

– duality between embedding and attacks

– secrecy of embedding can’t rely on orig. being unknown

– double-watermarking used by SDMI

#

robust wmk

⇒

should resist processing/attacks

#

fragile wmk

⇒

indicate audio experience compression

– attacks and countermeasures on forging fragile wmk

(8)

FPO 4/01

Is Watermark Useful?

!

Not an answer to all

– our attacks pointed out weaknesses of specific proposals and demonstrated

general approaches

!

For copy/access control

– hard to get complete solution with technology alone

#

business model, pricing model, etc.

– improved watermark tech. could be

part of the solution

#

make attack non-trivial and keep honest people honest

!

Other applications

– detecting alteration

#

digital camera/camcorder; digitized signature/ binary doc.

– convey side information

#

for performance enhancement or additional funtionality

FPO 4/01

Summary

FPO 4/01

Summary & Conclusion

!

Data hiding in digital multimedia for a variety of

purposes, involving multiple disciplines

!

Tradeoff among many criterions

!

Important to think both as designer and as attacker

!

Data hiding in market

– digital cameras with authentication watermark module

– plug-in for image editors

– video watermark proposals for DVD copy control

– on-going SDMI effort for digital music

– “Digital Rights Management (DRM)” for multimedia data

FPO 4/01

Other Data Hiding Works in Thesis

!

Watermark-based image/video authentication, attack &

countermeasures

– hide auth. data via look-up table in quantized coeff.

(ICIP’98 &’99)

– double capturing attack and countermeasure

(Asilomar’99)

!

Rotation/Scale/Translation resilient watermarking

(w/ NECI)

– add spread-spectrum wmk in log-polar of FFT magnitude

(Trans. IP’01,

SPIE’00)

!

Data hiding for video communication

(w/ P. Yin)

– real-time video transcoding via downsizing

#

send subblock motion for better visual quality

– error concealment

(9)

FPO 4/01

Publication List

Fundamental Issues

1. M. Wu, B. Liu: “Data Hiding in Images and Videos: Part I – Fundamental Issues and Solutions”, draft, to be submitted to IEEE Trans. on Circuits & Systems for Video Technology, Feb. 2001.

2. M. Wu, H. Yu, A. Gelman: “Multi-level Data Hiding for Digital Image and Video”, SPIE’99.

3. M. Wu, B. Liu: “Digital Watermarking Using Shuffling”, IEEE ICIP'99.

Designs

1. C-Y. Lin, M. Wu, Y-M. Lui, J.A. Bloom, M.L. Miller, I.J. Cox: “Rotation, Scale, and Translation Resilient Public Watermarking for Images,” to appear in IEEE Transactions on Image Processing, May 2001.

2. M. Wu, B. Liu: “Data Hiding in Binary Images”, submitted to IEEE Trans. on Multimedia, Apr. 2001.

3. M. Wu, H. Yu, B. Liu: “Data Hiding in Images and Videos: Part II – Designs and Applications”, draft, to be submitted to IEEE Trans. on Circuits & Systems for Video Technology, Feb. 2001.

4. M. Wu, B. Liu: “Data Hiding for Image and Video Authentication”, to be submitted to IEEE Trans. on Image Processing, Jan. 2001

5. M. Wu, E. Tang, B. Liu: “Data Hiding in Digital Binary Image”, IEEE ICME'00.

6. M. Wu, H. Yu: “Video Access Control via Multi-level Data Hiding”, IEEE ICME'00.

7. P. Yin, M. Wu, B. Liu: “Video Transcoding by Reducing Spatial Resolution”, IEEE ICIP’00.

8. C-Y. Lin, M. Wu, J.A. Bloom, M.L. Miller, I.J. Cox, and Y-M. Lui: “Rotation, Scale, and Translation Resilient Public Watermarking for Images,” SPIE’2000.

www.ee.princeton.edu/~minwu/research.html

FPO 4/01

(list of design papers - cont’d)

9. M. Wu, B. Liu: “Watermarking for Image Authentication”, ICIP'98.

10. P. Yin, M. Wu, B. Liu: “Error Concealment for MPEG Video Over Internet”, submitted to ICIP’01.

Attacks & Countermeasures

1. M. Wu, S. Craver, E. Felten, B. Liu: “Analysis of Attacks on SDMI Audio Watermarks”, to appear in IEEE ICASSP'01.

2. S. Craver, P. McGregor, M. Wu, B. Liu, A. Stubblefield, B. Swartzlander, D.S. Wallach, D. Dean, E.W. Felten: “Reading Between the Lines: Lessons from the SDMI Challenge”, to appear in 4thInfo. Hiding Workshop, 2001.

3. M. Wu, B. Liu, “Attacks on Digital Watermarks”, Asilomar’99.

Non-watermark Works on Video

(not included in thesis)

1. M. Wu, R. Joyce, H-S. Wong, L. Guan, S-Y. Kung: “Dynamic Resource Allocation Via Video Content and Short-term Traffic Statistics”, to appear in IEEE Trans. on Multimedia, special issues on multimedia over IP, June 2001.

2. M. Wu, R. Joyce, S-Y. Kung: “Dynamic Resource Allocation Via Video Content and Short-term Traffic Statistics”, ICIP ’00, invited paper.

3. H-S. Wong, M. Wu, R. Joyce, L. Guan, S-Y. Kung: “A Neural Network Approach For Predicting Network Resource Requirement in Video Transmission,” IEEE Pacific Rim Conference on Multimedia (PCM’00).