Local Polytope

As mentioned in the previous section, minimizing an arbitrary multi-label MRF en- ergy can be written as a linear program over the marginal polytope Ω. However, in general,Ωhas exponentially many faces and vertices, hence making it an intractable problem. In this section, we will approximate the marginal polytope with a simpler polytope that has a polynomial number of faces.

To this end, one may think of a different relaxation for the set of pointsT. Before that, let us first relax the integral constraints on the labellingx∈Γ.

Definition 2.3.5. Let us define the set of all valid real (continuous) labellings as

S = y ∑λ∈Lyi:λ =1, i∈ V yi:λ ≥0,i∈ V, λ∈ L . (2.70)

The set of real labellings and the integral labellings are related as follows.

Lemma 2.3.2. The set of real labellings S is the convex hull of the set of valid integral labellingsΓ,i.e.,S =conv(Γ).

Proof. It can be easily proven that any point y∈ S is a convex combination of points inΓ. Now, sinceS is a convex set,S =conv(Γ).

Similar to Eq. (2.62), by introducing additional variablesyij:λµ, we define the vec-

tor ˜yas

˜

y= [. . . ,yi:λ, . . . ,yij:λµ, . . .], (2.71)

with the same ordering of elements as in Eq. (2.62). Note that the vector ˜y ∈ IRN. Now we are ready to define a convex polytope containing T that has a polynomial number of faces.

Definition 2.3.6. Let the vector ˜y ∈IRNbe as defined in Eq. (2.71). Thelocal polytope

Λis defined as Λ=              ˜ y ∑λ∈Lyi:λ =1, i∈ V yi:λ ≥0,i∈ V, λ∈ L ∑λ∈Lyij:λµ=yj:µ, (i,j)∈ E, µ∈ L ∑µ∈Lyij:λµ=yi:λ, (i,j)∈ E, λ∈ L yij:λµ≥0, (i,j)∈ E,λ,µ∈ L              . (2.72)

The relationship between Ω and Λ can be stated as follows [Wainwright et al., 2005].

Lemma 2.3.3. The local polytopeΛis a convex set containing the marginal polytopeΩ,i.e.,

Figure 2.16: The local polytopeΛ(red) is a convex outer bound of the marginal polytopeΩ. The additional vertices of the local polytope are often referred to as fractional vertices.

Proof. SinceΛis defined using linear constraints, it is convex. Note that any point in T satisfies the constraints defined in Eq. (2.72) and by definitionΩ=conv(T). Since

Λ is a convex set containing T, it must also contain the convex hull of T. Hence,

Ω⊂Λ.

From Lemma 2.3.1, the vertices of the marginal polytope is the setT. Let us now prove that any point inT is a vertex ofΛ. We prove this by showing that there is no line segment in Λsuch that ˜x ∈ T is aninterior point7. Consider ˜x ∈ T and ˜y ∈ Λ, and let

τ= (1−α)x˜ +αy˜ , (2.73) for someα∈[−ε,ε]. Note that, whenα<0, ˜xis an interior point of the line segment connecting ˜y and τ. We show that τ ∈/ Λif α < 0. Choose i ∈ V andλ ∈ Lsuch that xi:λ = 0 and yi:λ > 0. This is possible unless ˜y = x. Let the element in˜ τ

corresponding to indices{i,λ}beτi:λ. Then,

τi:λ = (1−α)xi:λ+αyi:λ =αyi:λ . (2.74)

Here, if α< 0, thenτi:λ < 0. This meansτ ∈/ Λ. Hence, any point ˜x ∈ T must be a

vertex ofΛ.

See Figure 2.16 for an illustration of this Lemma. Now, it is clear that the local polytope has an exponential number of vertices (becauseT has an exponential num- ber of elements) and a polynomial number of faces (because Λ is defined using a polynomial number of inequalities). However, the local polytope may contain addi- tional vertices, which are often referred to as fractional vertices. See Figure 2.17 for an example of a fractional vertex of the local polytope. Since the local polytope is defined using a polynomial number of constraints, the minimization over it can be done in polynomial time8 [Karmarkar, 1984].

7_{If C is a subset of a Euclidean space, then xis aninterior pointof C, if there exists an open ball}

centered atxwhich is completely contained inC[Boyd and Vandenberghe, 2009]. Anopen ballcentered atxis defined as,B(x,r) ={y| ky−xk ≤r}, for somer>0, wherek · kis some norm.

Figure 2.17: An MRF example where a fractional labelling is a vertex of the local polytope. Here, we consider a binary MRF with 3 nodes, and the labelling is denoted with shaded nodes and edges, where, for all the nodes, yi:λ = 0.5 and, for the edges shown in the figure,

yij:λµ=0.5. It can be verified that the labelling shown is a vertex ofΛby following the same

argument as in the proof of Lemma 2.3.3.

In general, Λ is strictly larger than Ω. Hence, the minimization over the local polytope provides alower boundto the optimal energy,

min τ∈Λ hθ,τi ≤min τ∈Ω hθ,τi=min x∈Γ Eθ(x). (2.75)

This lower bound is tight for multi-label submodular MRFs [Werner, 2007] and tree structured MRFs [Wainwright et al., 2005]. Specifically, for such MRFs, the optimal labelling can be obtained by minimizing the LP relaxation over the local polytope. This means the minimum obtained by optimizing overΛis a vertex of the marginal polytope. In fact, the marginal polytope and local polytope depend on the MRF graphG = (V,E)and not on the energy parametersθ. Therefore, for tree structured

MRFs, the relationship betweenΩandΛcan be stated as follows. Theorem 2.3.1. For a tree structured MRF,Ω=Λ.

Proof. The idea is that, for tree structured MRFs, the optimal labelling can be obtained by minimizing over the local polytope [Wainwright et al., 2008],i.e.,

min τ∈Λ hθ,τi=min τ∈Ω hθ,τi=min x∈Γ Eθ(x), (2.76) and letτ∗ be the point where the minimum is attained. Now, ifΩ⊂ΛandΩ6= Λ,

then, one can find a θ such that τ∗ ∈ Λ and τ∗ ∈/ Ω, which is a contradiction.

Therefore,Ω=Λ. See [Wainwright et al., 2008] for more detail.

Note that, for multi-label submodular MRFs defined on general graphs, Λ is a strict outer bound of Ω. However, due to the restriction on the energy parameters

θ, the exact minimum is obtained by optimizing over Λ. In more general cases,

optimization over Λ yields fractional labellings. Such a fractional labelling can be rounded to obtain an integral labelling using simple argmaxrounding. Specifically, lety∗ be the optimal fractional labelling, then the integral labellingx∗ is given by

x_i∗=argmax

λ∈L

y_i∗_:λ ∀i∈ V . (2.77)

In general, this rounding procedure is not optimal. Nevertheless, more sophisticated rounding schemes that provide theoretical bounds on the rounded labelling have also been introduced [Kleinberg and Tardos, 2002; Ravikumar et al., 2008].