GM-W3: Two Methods in Saliency Detection

(1)

GM-W3: Two Methods in Saliency Detection

Hongyang Li (oø )

IIAU Lab, Dalian University of Technology [email protected]

Supervisor: Prof. Huchuan Lu (© © © A A A Ç Ç Ç)

September 17, 2013

Hongyang Li (oø ) Two Methods in Saliency Detection

(2)

1 Graph-based Manifold Ranking Model Construction

Results

2 Low Rank Matrix Recovery Model Construction Results

3 Summary

Chuan Yang et al.

Saliency Detection via Graph-Based Manifold Ranking CVPR 2013.

Xiaohui Shen and Ying Wu

A Unified Approach to Salient Object Detection via Low Rank Matrix Recovery

CVPR 2012. Oral Presentation.

(3)

Graph-based Manifold Ranking

Inspired by PageRank and spectral clustering algorithms, the optimization is:

Steps:

1

superpixels ¹ , affinity matrix W, degree matrix D.

2

ranking function, f ^∗ = (D − αW) ⁻¹ y.

3

background queries, S _t , S _b , S _l , S _r ; saliency map S _bq .

4

foreground queries, S _fq = ¯f ^∗ (i ).

1

Website

(4)

Geodesic distance constraints (Fig.3).

Wise choice of queries (Fig.4, Fig.5).

A two-stage mechanism (Fig.5, Fig.6).

(5)

Some Issues

How to find a pervasive parameter?

“These two parameters are empirically chosen, for all the experiments.”

Figure: σ

²

= 0.1, α = 0.99

(6)

How to find a pervasive parameter?

“These two parameters are empirically chosen, for all the experiments.”

Figure: σ

²

= 0.2, α = 0.99

(7)

Some Issues(Cont’)

How to find a pervasive parameter?

“These two parameters are empirically chosen, for all the experiments.”

Figure: σ

²

= 0.1, α = 0.8

(8)

(9)

Some Issues(Cont’)

Figure: Carefully chosen images.

Figure: Stage 1.

Figure: Stage 2.

(10)

Under some circumstances, the proposed algorithm is confused

with the background.

(11)

Low Rank Matrix Recovery for Saliency

Slide Credit: X.Shen.

(12)

Input ² : a feature matrix F ∈ R ^N×D , of the image.

Output: an optimal low-rank matrix L ^∗ , of the background; an optimal sparse-noise matrix S ^∗ , of the salient object.

Basic Model

(L ^∗ , S ^∗ ) = arg min L,S (kLk ∗ + λkSk 1 ) s.t. F = L + S

Notation: k · k

∗

, nuclear norm; k · k

1

, l

1

-norm.

Method Solution

³

robust PCA.

Saliency Measure kS

i

k

1

.

Image Decomposition mean-shift clustering.

Feature Representation color, edge, texture; altogether D = 53 features.

2From: J.Yan et al, Visual saliency detection via sparsity pursuit. ISPL,2010.

3From: J.Wright et al, Robust principal component analysis. NIPS,2009.

(13)

What is a good way to decompose the image?

Slide Credit: X.Shen.

(14)

Slide Credit: X.Shen.

(15)

Feature Representation

Sparse coding? A sparse encoded vector for each image patch, NOT the sparsity of the saliency over the entire image.

Sln-1: learn a transformation from a set of training images.

Sln-2: higher-level prior integration based on human perception.

Unified Model

(L ^∗ , S ^∗ ) = arg min L,S (kLk ∗ + λkSk 1 ) s.t. TFP = L + S

T: a learned feature transformation, i.e., linear metric T ∈ R ^D×D . P: a higher-level prior map, i.e., P = diag (p ₁ , p ₂ , . . . , p _N ), where p i indicates the probability of being salient for the i-th segment.

(16)

(17)

GM-W3: Two Methods in Saliency Detection

GM-W3: Two Methods in Saliency Detection

Hongyang Li (oø )

IIAU Lab, Dalian University of Technology [email protected]

Supervisor: Prof. Huchuan Lu (© © © A A A Ç Ç Ç)

September 17, 2013

1 Graph-based Manifold Ranking Model Construction

Results

2 Low Rank Matrix Recovery Model Construction Results

3 Summary

Chuan Yang et al.

Saliency Detection via Graph-Based Manifold Ranking CVPR 2013.

Xiaohui Shen and Ying Wu

A Unified Approach to Salient Object Detection via Low Rank Matrix Recovery

CVPR 2012. Oral Presentation.

Graph-based Manifold Ranking

Inspired by PageRank and spectral clustering algorithms, the optimization is:

Steps:

superpixels 1 , affinity matrix W, degree matrix D.

ranking function, f ∗ = (D − αW) −1 y.

background queries, S t , S b , S l , S r ; saliency map S bq .

foreground queries, S fq = ¯f ∗ (i ).

Website

Geodesic distance constraints (Fig.3).

Wise choice of queries (Fig.4, Fig.5).

A two-stage mechanism (Fig.5, Fig.6).

Some Issues

How to find a pervasive parameter?

“These two parameters are empirically chosen, for all the experiments.”

Figure: σ

= 0.1, α = 0.99

How to find a pervasive parameter?

“These two parameters are empirically chosen, for all the experiments.”

Figure: σ

= 0.2, α = 0.99

Some Issues(Cont’)

How to find a pervasive parameter?

“These two parameters are empirically chosen, for all the experiments.”

Figure: σ

= 0.1, α = 0.8

Some Issues(Cont’)

Figure: Carefully chosen images.

Figure: Stage 1.

Figure: Stage 2.

Under some circumstances, the proposed algorithm is confused

with the background.

Low Rank Matrix Recovery for Saliency

Slide Credit: X.Shen.

Input 2 : a feature matrix F ∈ R N×D , of the image.

Output: an optimal low-rank matrix L ∗ , of the background; an optimal sparse-noise matrix S ∗ , of the salient object.

Basic Model

(L ∗ , S ∗ ) = arg min L,S (kLk ∗ + λkSk 1 ) s.t. F = L + S

Notation: k · k

, nuclear norm; k · k

, l

-norm.

Method Solution

robust PCA.

Saliency Measure kS

k

.

Image Decomposition mean-shift clustering.

Feature Representation color, edge, texture; altogether D = 53 features.

What is a good way to decompose the image?

Slide Credit: X.Shen.

Slide Credit: X.Shen.

Feature Representation

Sparse coding? A sparse encoded vector for each image patch, NOT the sparsity of the saliency over the entire image.

Sln-1: learn a transformation from a set of training images.

Sln-2: higher-level prior integration based on human perception.

Unified Model

(L ∗ , S ∗ ) = arg min L,S (kLk ∗ + λkSk 1 ) s.t. TFP = L + S

T: a learned feature transformation, i.e., linear metric T ∈ R D×D . P: a higher-level prior map, i.e., P = diag (p 1 , p 2 , . . . , p N ), where p i indicates the probability of being salient for the i-th segment.

Learn a Feature Transformation(Cont’)

Objective Function

T ∗ = arg min T O(T) , K 1 P K

k=1 kTF k Q k k ∗ − γkTk ∗

s.t. kTk 2 = c where F k and Q k are the feature representation and saliency indicator of the k-th training image respectively.

Elaborations on the function

Method: Gradient Descent, T t+1 = T t − α ∂O(T) ∂T More details, please see paper. However,...

superpixels ¹ , affinity matrix W, degree matrix D.

ranking function, f ^∗ = (D − αW) ⁻¹ y.

background queries, S _t , S _b , S _l , S _r ; saliency map S _bq .

foreground queries, S _fq = ¯f ^∗ (i ).

Input ² : a feature matrix F ∈ R ^N×D , of the image.

Output: an optimal low-rank matrix L ^∗ , of the background; an optimal sparse-noise matrix S ^∗ , of the salient object.

(L ^∗ , S ^∗ ) = arg min L,S (kLk ∗ + λkSk 1 ) s.t. F = L + S

(L ^∗ , S ^∗ ) = arg min L,S (kLk ∗ + λkSk 1 ) s.t. TFP = L + S

T: a learned feature transformation, i.e., linear metric T ∈ R ^D×D . P: a higher-level prior map, i.e., P = diag (p ₁ , p ₂ , . . . , p _N ), where p i indicates the probability of being salient for the i-th segment.

T ^∗ = arg min _T O(T) , _K ¹ P K

k=1 kTF _k Q _k k _∗ − γkTk _∗

s.t. kTk 2 = c where F _k and Q _k are the feature representation and saliency indicator of the k-th training image respectively.

Method: Gradient Descent, T t+1 = T t − α ^∂O(T) _∂T More details, please see paper. However,...