COLOR QUANTIZATION

For color quantization, we use a simplified version of the approach described in [Winnemöller06]. Suppose that c is the input color in Y’CbCr color space, and nbins is the number of quantization levels. To avoid a sharp transition between different quantization levels, we use the smoothstep function as shown in the following code:

float qn = floor(c.x * float(nbins) + 0.5) / float(nbins); float qs = smoothstep(-2.0, 2.0, phi_q*(c.x - qn) * 100.0)-0.5; float qc = qn + qs / float(nbins);

The parameter phi_q controls the smoothness of the transition. In our examples we use phi_q = 3.4, which results in a cartoon-like effect.

CONCLUSIONS

We presented a framework of image processing techniques that can be used for real-time image abstraction. A smooth vector field whose flow curves follow salient image features can be constructed from the eigenvalues of the smoothed structure tensor in a highly efficient way. Aligning the filter to the local structure can increase the quality of the xy-separable bilateral filter. Flow-based DoG filtering can be implemented as a separable algorithm. It outperforms the classical DoG filter in quality and is efficient to compute. Source code for a sample application is provided on the DVD-ROM accompanying this book.

ACKNOWLEDGMENTS

The authors would like to thank Pascal Barla, Tassilo Glander, Stefan Maaß, Haik Lorenz, and Christian Richardt and for their comments, Matthias Trapp for helpful discussions about GPU programming, and the section editor, Christopher Oat, for his support. This work was supported by the German Research Foundation (DFG), grant DO 697/5-1.

REFERENCES

[Cumani89] Cumani, A., “Edge Detection in Multispectral Images,” Technical report, Istituto Elettrotecnico Nazionale “Galileo Ferraris” (1989).

[Kang07] Kang, H., Lee, S., and Chui, C. K., “Coherent line drawing,” Proc. ACM NPAR (2007): pp. 43–50.

[Kyprianidis08] Kyprianidis, J. E., and Döllner, J., “Image abstraction by structure adaptive filtering,” Proc. EG UK Theory and Practice of Computer Graphics (2008): pp. 51–58

[Marr80] Marr, D., Hildreth R. C., “Theory of edge detection,” Proc. Royal Society London 207 (1980): pp. 187–217.

[Paris07] Paris S., Kornprobst P., Tumblin J., and Durand F., “A gentle introduction to bilateral filtering and its applications,” ACM SIGGRAPH courses (2007).

[Pham05] Pham, T. Q., and van Vliet, L., “Separable bilateral filtering for fast video preprocessing,” IEEE International Conference on Multimedia and Expo (2005).

[Tomasi98] Tomasi, C., and Manduchi, R., “Bilateral filtering for gray and color images,” Proc. International Conference on Computer Vision (1998): pp. 839–846.

[van Vliet95] van Vliet L., Verbeek P., “Estimators for orientation and anisotropy in digitized images,” Proc. ASCI, Conference of the Advanced School for Computing and Imaging, Heijen (1995): vol. 95, pp. 16–18.

[Winnemöller06] Winnemöller, H., Olsen, S. C., and Gooch, B., “Real-time video abstraction,” ACM Transactions on Graphics (2006): Vol. 25, no. 3, 1221–1226.

Part IV

Shadows

You should at least consider it curious that the highly specific subject of “shadows” continues to play such a major part in current graphics research. Although the problem is as old as the computer graphics industry itself, anyone who has attempted to use a modern shadowing solution will tell you that the problem of real-time shadowing is not yet adequately solved. The evils of aliasing, the complications of soft shadowing, greedy memory consumption, and heavy rendering costs— they still conspire to disappoint artists the world over.

But is the situation really all that gloomy? Well, no. Shadows are indeed proving to be a tough subject but regular advances in the field have arrived at a comforting rate, and in this section of the book you will find both consolidation and innovation.

Amongst a sea of alternatives, cascaded shadow maps [Engel06][Zhang07] is one technique that has found a warm reception amongst developers for its practicality and effectiveness. A successful implementation is not without its challenges and it is this topic that Fan Zhang, Alexander Zapriagaev, and Allan Bentham address head-on in the article, “Practical Cascaded Shadow Maps.”

The effects of fluids and gases on light transport are frequently neglected, but Chris Wyman and Shaun Ramsey describe an approach to modelling their influence in, “A Hybrid Method for Interactive Shadows in Participating Media.”

High-dynamic range environment maps can provide a particularly effective and natural means of lighting a scene. The lighting can be computed dynamically with spherical harmonics as in [Ramamoorthi01], but this ignores the effect of visibility. If the geometry can be assumed to be static, then the visibility information can be precomputed [Sloan02], but efficient handling of dynamic geometry is still an active field of research. Mark Colbert and Jaroslav K ivánek present a novel algorithm for environment lighting with dynamic visibility in “Real-Time Dynamic Shadows for Image-Based Lighting.”

The notable main alternative to the splitting approach used in cascaded shadow maps are warped projections, such as

[Stamminger02]. Ray Tran presents a new technique for harnessing the benefits of both camps through a novel hybrid in “Facetted Shadow Mapping for Large Dynamic Game Environments.”

R

[Engel06] Wolfgang Engel, “Cascaded Shadow Maps”, ShaderX5, 2006

[Zhang07] Fan Zhang, Hanqiu Sun, Leilei Xu and Le Kit Lun, “Parallel-split shadow maps for large-scale virtual environments”, ACM VRCIA’06

[Stamminger02] Stamminger, Marc and George Drettakis, “Perspective Shadow Maps”, ACM SIGGRAPH 2002

[Ramamoorthi01] Ravi Ramamoorthi, Pat Hanrahan, “An Efficient Representation for Irradiance Environment Maps” ACM SIGGRAPH 2001

[Sloan02] Peter-Pike Sloan, Jan Kautz, and John Snyder, “Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments”, ACM SIGGRAPH 2002

4.1 Practical Cascaded Shadow Maps

FAN ZHANG, ALEXANDER ZAPRJAGAEV, ALLAN BENTHAM

INTRODUCTION

Like any other visual effect in real-time applications, practical shadow algorithms should allow developers to trade cost and quality. Although it is a popular real-time shadowing technique, shadow mapping [Williams78] cannot achieve this goal in complicated scenes, because it does not adequately trade performance cost for quality. Numerous extensions to this algorithm have been developed toward achieving this goal. Cascaded shadow maps (CSMs) [Engel06][Forsyth05] are an example of such an extension and have recently been attracting more attention with graphics hardware becoming more powerful. This technique can handle the

dueling frusta case in shadow mapping, in which the light and view directions are nearly opposite. Furthermore, unlike warping algorithms [Stamminger02][Wimmer04][Martin04], CSMs do not require careful tuning and special tricks to handle extreme cases. We note that a similar idea was independently developed by Zhang et al. in the concurrent work on parallel-split shadow maps (PSSMs)

[Zhang07][Zhang06]. This paper does not differentiate between the papers, as the basic idea is the same.

Although the prototyping of CSMs is quite simple, integrating CSMs into real games requires us to carefully consider several issues, some of which were not mentioned in the original CSM algorithm. This article discusses practical implementation issues of CSMs, which we have experienced in the Unigine project (see Figure 4.1.1), outlined as follows.

FIGURE 4.1.1 Cascaded shadow maps in Unigine.

Flickering of shadow quality: This problem is also referred to as swimming, which exists in all view-dependent shadow- mapping techniques. To improve available shadow map texture resolution, a popular method is to focus the shadow map on visible objects. Such view-dependent processing can result in different rasterizations in consecutive frames. The flickering of shadow quality is thus perceived. To see this artifact, please refer to the video on the accompanying DVD-ROM. This article presents two methods to handle this issue.

Storage strategy: In CSMs, shadow map textures can be stored in a variety of ways. For example, the most intuitive way is to store each shadow map in a separate texture. However, this storage strategy places considerable demand on the available interpolators. As a result, it is not an ideal choice for complex shading systems that already require a large number of simultaneous textures. Therefore, we should choose the storage strategy that best suits the requirements of the project. This article presents a detailed discussion of different strategies.

Non-optimized split selection: The practical split scheme proposed by Zhang et al. [Zhang07] provides a practical solution for choosing the corresponding shadow map for a given pixel being rasterized. However, in some extreme cases such as when the light and view directions are nearly parallel, this scheme is not optimal. This article presents the solution we used in the Unigine project.

Correct computation of texture coordinates: In typical CSM implementations, the vertex shader feeds all possible texture coordinates to the pixel shader for texture lookup. An alternative is to output the world-space position from the vertex shader. The pixel shader computes the texture coordinates from this world-space position after the split index is determined. The major advantage of this alternative is only using a single interpolator. However, as we will explain in this article, this method is mathematically incorrect because it results in an incorrect interpolation during rasterization.

Filter across splits: A filtering operation is usually needed in shadow mapping to produce anti-aliased shadows. A recent development in this field is prefiltering (i.e., filtering happens before depth comparison) the shadow map texture by exploiting the built-in filtering capabilities available on graphics hardware. This is made possible by the insightful ideas presented in literature on Variance Shadow Maps (VSMs) [Donnelly06][Lauritzen07b][Lauritzen08], Convolution Shadow Maps (CSMs)

[Annen07], and Exponential Shadow Maps (ESMs) [Salvi07] [Annen08]. Though these filtering methods and CSMs are orthogonal, combining them requires careful consideration of discontinuity at split positions. This article explains how this problem occurs and presents two solutions.

The remainder of this article details each of the above issues and provides a summary.