Like a conventional depth image, an animated depth image suffers from disocclu- sion errors when the viewpoint translates. Moreover, disocclusion errors also occur when sample motion uncovers new samples. We alleviate disocclusion errors by sam- pling the dataset adaptively from multiple viewpoints and at multiple time steps (Section 3.4.1), and by eliminating redundant samples (Section 3.4.2).
Algorithm 5 Adaptive sampling algorithm
Input: Dynamic dataset D, view triangle R, timer interval T Output: ADI tiles set S
1: Initialize the set S of animated depth image tiles to empty. 2: Initialize view parameter queue q with (R, T )
3: while q is not empty do
4: Dequeue the first pair (Rnow, Tnow) from q
5: for all Viewpoint e, time step t in a discretized subset of in (Rnow, Tnow) do
6: Render a depth image DIet
7: Computer non-redundant samples DIet− S
8: S∗
et = T ile(DIet− S)
9: Set = ConstructADI(Set∗)
10: S = S∪ Set
11: if The last processed |DIet− S|/|DIet| > g then
12: Subdivide (Rnow, Tnow) and enqueue the subregions
13: Return S
Our adaptive sampling scheme renders conventional depth images from various locations inside the viewpoint triangle and at various time steps. The sampling pro- cess stops when the percentage of non-redundant samples contributed by a new depth image drops below a threshold g. In order to enable the elimination of redundant sam- ples, depth images and animated depth images are split into square tiles of size tw×tw
(we use tw = 4). Figure 3.7 shows the tiles containing the non-redundant samples
contributed by a depth image. Tiles, like complete images, allow storing connectivity information compactly. Given a tile size tw and a threshold g for the percentage of
new samples contributed by a new image, our adaptive sampling algorithm proceeds as follows.
Figure 3.7. Non-redundant samples gathered by our adaptive algorithm.
The algorithm samples the space of possible viewpoints and time steps recursively, in the form of a stack implementation in Algorithm 5. The recursion start from the input view triangle polygon R and time interval T . For each pair of (R, T ), a predefined viewpoints subset are used to represent the visibility of this entire region. In our implementation, we use the three vertices plus the centroid of the view triangle, and the two end points of the time interval. For each viewpoint and time pair (e, t) in this space, a new depth image DIet is rendered at line 6. Line 7 checks
for each sample in DIet whether it is redundant with the samples already collected in
S. The non-redundant parts of DIet are partitioned into tiles (line 8 ), an animated
depth image tile is computed for each depth image tile (line 9 ), the new set of tiles is added to S (line 10 ).
The subdivision continues recursively if the last examined pair (e, t) contributes sufficient number of non-redundant samples (lines 11-12 ). When the algorithm fin- ishes, the set of ADI tiles are returned (line 13 ). The first depth image computed by the adaptive sampling algorithm corresponds to the center of the viewpoint triangle (also the viewpoint of P P C0) and to t0. Since S is initially empty, S will contain the
entire animated depth image constructed for P P C0, which guarantees highest-quality
Figure 3.8. Projections of samples P0 and P1 onto the image plane as the viewpoint
translates on the viewpoint triangle plane. The distance d between the projections is 0 at C0 where q00 and q10 coincide, and it increases away from C0.
3.4.2 Sample redundancy
The first task is to define sample redundancy. Depth images contain point samples and two samples will in general not correspond to the same 3D point. We define two samples as redundant if and only if they project within one pixel in all views P P Ci,
where P P Ciis identical with P P C0 except that the viewpoint can be anywhere inside
the equilateral viewpoint triangle with radius q. If two samples belonging to different surfaces happen to project at nearby locations from a viewpoint, motion parallax separates the two samples when seen from a different viewpoint, and the samples are correctly labeled as non-redundant. Our definition of redundancy ignores view rotations which only introduce a negligible variation of the distance between the projections of two samples. Given two samples defined at different time steps ta
and tb, the samples are redundant if they are redundant when the second sample is
brought to time step ta.
The second task is to find a method for quickly checking for sample redundancy, given our definition. Figure 3.8 illustrates the projection of a pair of two samples P0
and P1 as the viewpoint translates on the plane of the viewpoint triangle. The image
plane is constant as there are no view direction rotations. Let C0 be the intersection
between P0P1 and the viewpoint plane. For a viewpoint C, the square of the distance
d2 =|q0− q1|2 = (w0− w1)2 · x2 (3.5)
where wi(i = 0, 1) and x are defined as:
wi = |qi− q 0 i| |C − C0| = |Pi− Si| |Pi− Oi| (3.6a) x =|C − C0| (3.6b)
Since w0 and wi do not depend on the current viewpoint C, d2 is a quadratic
function in x, with the minimum value of 0 reached when C = C0. Distance d
increases away from C0. Over the entire viewpoint triangle, the maximum distance
occurs at one of the three vertices of the viewpoint triangle. Consequently, given two samples, the largest distance between the projections of the samples over all viewpoints inside the viewpoint triangle can be easily computed as the maximum over the three distances obtained at the viewpoint triangle vertices. If the maximum is less than one pixel, the new sample is discarded as redundant.