We propose to generalize images to overcome their visibility computation lim- itations described above. Our image generalization paradigm has three elements: sampling pattern generalization, visibility sample generalization, and ray geometry generalization.
1.4.1 Sampling pattern generalization
The first element of the image generalization paradigm is to abandon the uni- form sampling of the image plane used by conventional images and to add sampling locations to the image plane where needed in order to improve the quality of the visibility solution computed. One option is to add sampling locations at image plane regions where the data subsets of higher complexity project. Another option is to add sampling locations such as to find all geometric primitives with a completely visible fragment. A fragment is the intersection between an image pixel and the image plane
projection of the geometric primitive. In other words, the fragment is the part of the pixel square covered by the geometric primitive. Sampling locations are added to make sure that all fragments are sampled, which guarantees finding all geometric primitives with a completely visible fragment, and which in turn guarantees finding all geometric primitives of a front surface, no matter how small their footprint. For our sphere example, sampling all fragments finds all visible triangles at the cost of one sampling location per fragment (Figure 1.1, buttom). Sampling locations are not added heuristically but rather deterministically, based on the dataset geometric primitives and based on the pixel grid. This sampling pattern generalization does not guarantee finding all visible geometric primitives, as some visible primitives that do not have a completely visible fragment are missed. However, the guarantee of finding all geometric primitives of a front facing surface is a strong quality guarantee which in practice results in aggressive visible sets that are close to complete.
1.4.2 Visibility sample generalization
A conventional image sample only stores a scalar visibility value, corresponding to a single visible geometric primitive. When the viewpoint translates or when time changes in the case of a dynamic dataset, the geometric primitive that is visible at an image plane sampling location can change. We propose to generalize the vis- ibility sample to enable visibility computation in the context of dynamic datasets and of viewpoint translation. Visibility sample generalization proceeds in one of two directions. One direction is to increase the dimensionality of the visibility sample. A second direction is to enhance the sample with a record of its trajectory in the dynamic dataset. Increasing the dimensionality of the visibility sample brings two benefits: it enables visibility computation in the context of dynamic datasets and viewpoint translation, and it enables exact visibility computation. Consider the case of a dynamic dataset modeled with triangles rendered from a fixed view. As triangles move over time, multiple triangles can become visible at a given sampling location. What is needed is a 1D visibility sample that records all triangles visible at the sam- pling location over the time interval. The 1D visibility sample proposed by our image generalization paradigm is not a uniform 1D array of conventional samples. Instead, the 1D visibility sample is a subdivision of the time interval into subintervals such that a single triangle is visible for each subinterval. The visibility data stored by the
gles. A single 2D visibility sample is sufficient to compute an exact visibility solution. Like before, the visibility sample is a polygonal subdivision of a 2D space of visi- bility parameters. Whereas before the visibility parameters were the two viewpoint translations, now the visibility parameters are the two output image coordinates.
The second direction for visibility sample generalization is to enhance the sam- ple with its trajectory as the geometric primitive it samples moves in the dynamic dataset. The geometric primitive carries the sample as the primitive moves over time. Instead of recording which geometric primitives are visible at a given sampling location over time, this visibility sample generalization records where each sample moves over time. The advantage of the dimensionality generalization of the visibility sample is a high-quality of the visibility solution computed. The advantage of the trajectory generalization of the visibility sample is lower redundancy of the visibility solution: consider a visible triangle moving with a constant velocity vector; dimen- sionality generalization will record the triangle as visible at all the pixels the triangle touches as it moves; trajectory generalization will record each triangle sample once along with a line segment to indicate the linear motion.
1.4.3 Ray geometry generalization
A conventional image is rendered with a planar pinhole camera whose rays connect the viewpoint to the pixel centers. Therefore, a conventional image can only find scene surfaces to which there is a direct line of sight from the viewpoint. We abandon the restriction that camera rays be straight lines. Generalizing the ray geometry to allow a ray to be any continuous curve enables designing cameras whose rays reach around occluders to sample surfaces that are not visible from the viewpoint. Rendering a
dataset from a reference point with a camera designed to have enhanced disocclusion capability results in a multiperspective image that stores sufficient samples to support quality reconstruction of output frames from viewpoints other than the reference point.
The non-linear rays are designed for the resulting camera to provide a fast projec- tion operation. This way, the multiperspective image can be rendered efficiently with the feed forward approach. The rays are also designed to avoid ray intersections, as ray intersections lead to imaging a geometric primitive multiple times, and therefore to unnecessary redundancy of the set of visible samples.
Like the sample dimension generalization element of our paradigm, ray geometry generalization extends the visibility computation capability of images to support view translation. The two elements are orthogonal and they can be used in conjunction. Sample dimension generalization has the benefit of a higher quality visibility solution. Ray geometry generalization has the advantage of computing the visibility solution faster. Moreover, multiperspective images have a single layer and they are coherent, which makes it human interpretable. The multiperspective image can be shown di- rectly to the user to support occlusion management in visualization applications. In other words, the user can directly see the data subset visible from multiple viewpoints.