Figure 3.3. The combined effect of a few planar impostors can approximate the rendering of a complex 3D geometry, here, foliage, which appears different on both sides of each impostor. Image cour- tesy of Lluch et al. [361].
Image-based rendering techniques [88] make use of textured semitransparent polygons as impostors in a scene. These im- postors can be rendered faster in general than the many detailed 3D geometries that they are replacing, mainly because each impostor is usually built from very few un- derlying polygons. The image-based ren- dering techniques allow such impostors to be used in certain far-away views, bene- fiting from texture filtering within the im- postor surface, instead of facing the alias- ing of rendering multiple small, distant polygons. Impostors can also be interpo- lated between different textures, thus sim- ulating changing appearance under differ- ent view angles, under different light di- rections, or for animated objects (e.g., a walking sequence for a human encoded as a movie sequence). Impostors are particu- larly useful in urban environments because of the many buildings, trees, or humans (crowds) that can be faked almost seam- lessly. They can also be used multiple times as instances in a same scene or combined or textured differently to introduce some variability. Finally, because they are built on simple polygons, impostors integrate well within any polygonal-based rendering system.
Much of the research in image-based rendering has been focused on the com- putation of the seamless view interpolation of the impostors, as well as the compu- tational cost and access speed of many GPU textures representing the impostors. Impostors have been represented by 2D cross images called planar impostors [374], billboards [123], multilayer depth images [520], or volumetric billboards [121]. A single object can be represented by one or multiple impostors, and vice versa, an impostor can represent several different objects.Figure 3.3illustrates the combined effect of a few planar impostors.
In the brief coverage that follows, we present the literature on the shadowing techniques used in image-based rendering, specifically for large numbers of build- ings, trees, and humans (crowds), where the application of image-based rendering appears to be the most obvious. Note that the majority of the shadow algorithms are assumed to be shadows landing on a ground floor.
3.3.1
Trees
In a very traditional approach, Zhang et al. [666] use a set of three orthogonal billboards, each billboard complete with color and alpha channel information. Shadow projection onto the ground plane is achieved with the alpha channel in- formation and a stencil buffer. The method is only valid for distant trees, and these trees must be very symmetrical due to the minimal set of billboards.
Max and Ohsaki [389] simply precompute Z-buffer images from different view directions around a sphere, and project the 3D points extracted from the closest Z- buffer directions to produce an image from an arbitrary viewpoint. The 3D points should provide a better depth complexity, compared to blending between flat bill- boards. Shadow determination for the rendered 3D points comes from a similar construction of a shadow depth map.
Zhang and Nakajima [660] experiment with shadowing of impostors from trees. Features of the trees are captured as irregular lines, and projection of the irregular lines onto the ground plane forms the basis of the shadow generation. Several methods [132, 291, 661, 659] apply computer vision techniques to extract shadow silhouettes from image-based objects to achieve shadow projections of the silhouette on a planar ground.
Meyer et al. [406] render many trees on a landscape by also using image-based rendering techniques. A hierarchy of bidirectional textures is used to store dif- ferent LODs to represent the tree, and the appropriate LOD texture is sent to the GPU. Shadowing is accounted for in the following combinations: a hierarchy of visibility cube-maps is built where each tree’s cube-map accounts for visibility of other trees and other geometries (e.g., landscape); self-shadowing of the tree onto itself is taken care of in the bidirectional texture functions (BTFs) and horizon- mapping techniques [391]; a shadow depth map [630] is used to incorporate all of the above.
Qin et al. [470] use several vertical and a few horizontal 2D buffers to store a sampled representation of the shading and shadowing information. Voxels are also precomputed to approximate the visibility of the hemisphere surrounding a single tree in order to capture skylight shadowing. They handle shadows of a sunlight cast by a tree on itself, on the ground, and on neighboring trees, as well as softer shadows due to skylights.
3.3.2
Crowds
Loscos et al. [370, 583] represent humans walking in virtual cities, in which each human is treated as an animated impostor. Shadows of these humans are cast onto the floor by projecting a colored shadow version of the animated impos- tor itself, using a method similar to the fake shadows of Blinn [57] (see Sec- tion 2.2). Shadowing of buildings onto the humans is achieved using a shadow depth map of the city, and computing its height coverage against the human im-
Figure 3.4.Crowds are rendered using impostors to reduce the complexity of many 3D mod- els. Image courtesy of Tecchia et al. [583]. Computer Graphics Forum ©2002 The Eurographics Association and Blackwell Publishing Ltd. Published by Blackwell Publishing, 9600 Garsing- ton Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. Reproduced by permission of the Eurographics Association and Blackwell Publishing Ltd.
postor. Shadows are displayed using a shadow impostor (the impostor is com- posed of two parts: shadow and semitransparency) mapped onto the human impostor. An example of such crowd rendering is given in Figure 3.4. Dob- byn et al. [136] use instead a stencil-buffer approach for the shadows on the ground in order to remove z-fighting artifacts and to better blend in shadows on a textured ground.
In crowds lit by many static light sources (e.g., street lamp posts), shadowing can become a burden. Ryder and Day [493] combine each of the many shadow maps into a single large shadow map, thus reducing light context switches. The resolution of each shadow map is dependent on its distance in the image. Each light source has also a display list of all static scene elements within its radius of influence, and at run-time, a 3D grid is used to identify the impostors potentially lit by each light source. The impostors are augmented with depth per texel and rendered correctly with these depths in the shadow maps and in the image in order to cast accurate shadows and self-shadows.
3.3.3
Trends and Analysis
The real-time applications of image-based techniques in industry have been popu- lar in architecture projects, virtual reconstruction, and population of heritage sites, synthetic avatars in interactive dialogs, molecular visualization, some forms of hy- brid rendering for a large terrain (further away landscape rendered as an impos- tor), and obviously, within the video game industry. The latter has seen its use of impostors becoming mainstream for vegetation, animated particle-based effects, or captured video sequences to represent phenomena such as smoke, torches, rain, and large crowds.
Standard projection, shadow depth map, and ray tracing algorithms apply in many situations, but completely image-based representations or those from poly- gons augmented with image-based data require adaptations. For trees and crowds, the adaptations discussed previously form a good base, but no single solution proves perfect, and more work is still expected. A number of applications for of- fline rendering have appeared, notably for rendering tree impostors. In the case of multiple cross-plane impostors, these applications often give the ability to the user to select how impostors cast shadows.
Finally, one could draw similarities between impostors and other representa- tions such as point clouds (Section 3.6), voxels (Section 3.7), and heightfields (Sec- tion 3.8). One could suspect that some solutions for shadowing in one representa- tion could successfully be adapted to another.