The Problem with T-junctions

(a) (b)

Figure 6.16: Enforcing the 2-coloring across rim-edges (best seen in color). (a) A close-up of the 2-colored visual hull. Cone intersection curves are shown in bold (red and blue). Rim edges are shown in green. Dotted curves indicate possible position of rims. (b) T-junctions give rise to self-occlusions on the rim – the part that detaches from the visual hull surface and lies within its volume – we show the surface by its intersection with the sheet, a swept surface dropped down from the rim edges into the volume.

silhouettes. A T-junction arises on the silhouette, when the corresponding rim curve on the surface is occluded by the surface itself. The occluded section of the rim curve, which we will refer to as the invisible part of the rim does not lie on the surface of the visual hull, rather it lies within its volume. This is illustrated in Figure6.16using an imaginary sheet surface patch within the visual hull volume, which meets the visual hull surface at the set of rim edges. The

sheet surface can be chosen arbitrarily as long as the true surface intersects it as shown by the

dotted curve in the illustration.

Formulation I allows the coloring of the reconstructed surface to switch only at a silhouette consistency edge – however, that can only happen at a point on the visual hull surface, not in its interior. This causes silhouette constraints to be enforced at places where they should not be enforced such as along the rim edges shown in Figure6.16.

Consider the region on the visual hull surface around a sequence of rim edges. When these rim edges are shared by an even number of repaired cone-strips, the vertex colors on either side of the rim edges are identical to each other. Formulation I deals with this situation correctly, because the vertex coloring across a rim edge of even multiplicity, will be identical. However, a problem arises when a sequence of rim edges are shared by an odd number of repaired cone-strips i.e. they have odd multiplicty. In this case, the vertex colors on either side of the rim edges are opposite of each other and silhouette constraints get enforced. Such a rim edge is shown in Figure6.16. Since, the coloring of the surface needs to switch here, the cut surface is locally pushed outwards i.e. the invisible part of the rim curve is constrained to lie on the visual hull surface. This problem is not catastrophic – it happens only at a few places and may cause two types of artifacts.

• The reconstructed surface juts out in the form of a thin sheet (see Figure6.17(b)). • Portions of the cut surface detaches from the reconstructed surface, which is recovered

in the correct position and is artifact free.

these artifacts. It treats the points on the thin sheets and detached surfaces as outliers during the final surface reconstruction step and enables us to reconstruct some complex shapes with occasional artifacts as described above. We now look into handling the problem of the invisible rim in our formulation.

(a) (b)

Figure 6.17: (a) Sheet surface patches are first constructed from the rim edges with odd mul- tiplicity. The lattice edges lying on the rim sheet are removed to create a discontinuity in the volume. (b) A cross-section is shown here. Without the discontinuity, the cut surface may jut out. With the discontinuity, the cut surface can terminate on the sheet surface. The cut surface is now a manifold with a boundary.

One possible approach is to detect the problematic rim edges (the ones with odd multi- plicity), and construct imaginary sheet surfaces from them, within the volume. The edges in the graph embeddingGv, that straddle the sheet surface patches will be removed, thereby introducing discontinuities in the interior of the lattice volume (see Figure6.17). As a result, the minimum cut surface will become a 2-manifold with boundary. Its boundaries will lie along the sheet surfaces, and will not be constrained to lie on the visual hull surface. The inverse image of this new cut surface, will be a surface with open seams (or cracks) on it. The Poisson surface reconstruction step will typically merge these open seams. However, it will not be possible to guarantee that the two sides of the surface be geometrically aligned with each other along the seam – this could still result in noticeable artifacts on the reconstructed

surface.

Computing the Sheet Surface Patches: Our approach for generating the sheet surfaces,

involves constructing local offset surfaces in the interior of the visual hull volume and project- ing the sequence of rim edges on to this offset surface. The offset surfaces can be computed approximately using the 3D signed distance transformD(_O)of the visual hull surface_Oand the gradient of this distance transform_∇D(_O), respectively. Note that the distance transform is computed to determine the extent of the graph embeddingGv anyway, as described in Sec- tion6.6 – hence, this is not an extra computational burden. We densely sample pointspon the rim edges, and for each one of them, we take a small step along_∇D(p,O)to generate an offset curve and repeat the step a few times. The gradient field_∇D(O)can however vanish at points within the volume [140], and therefore we resort to a linear combination of_∇D(O)and −∇D(_I)where_I denotes the inner offset surface of the visual hull. The sheet surfaces are made to proceed only to a small depth relative to the visual hull surface. Finally, we recover the subset of voxels that contain all the points samples generated on the sheet surfaces. The relevant edges in these voxels are deleted, to create the discontinuities described earlier and this is also illustrated in Figure6.17.