GPU-S IDE W ORK

The motion blur is rendered at the end of the frame as a post-process effect. The first task of the pixel shader is to reconstruct the world space position of the current pixel. To compute this we use the normalized screen-space coordinates along with the current depth. Once we have the current world position of the pixel, we read the object ID of the current pixel from the stencil buffer and use this to index into the array of previous frame matrices. Then we transform the world position by the previous frame matrix to get the previous frame screen position of the pixel. We then subtract this from the previous frame position to get the velocity vector of the pixel and blur along this vector.

fl oat depth = tex2D(DepthMapS am pl er, texCoord);

fl oat s tenc i l = tex2D(S tenc i l S am pl er, texCoord);

fl oat4x4 prevMtx = P revFram eMoti onB l urMtxs [s tenc i l ];

fl oat3 s c reenP os = fl oat3(IN.texCoord.xy * Fram eB ufferS i ze.xy, 1.0f - depth);

Once we have determined the pixel velocity, we use it to blur the scene. We do this by averaging the colors of the pixels along the velocity vector.

There are several different approaches to performing the blur. It is possible to dynamically alter the number of blur samples at each pixel according to the length of the velocity vector. This leads to a lower cost in areas where there is little blur happening. The savings can be significant in applications where the camera is stationary or slow moving.

One potential problem with post-process motion blur is that objects with different velocities will blend with one another. Depending on the application, this may be undesirable. If your camera is slow moving or stationary in the scene, blending moving objects with the background will generally look realistic. However, when the camera is tracking a moving object, this blending between objects can lead to artifacts. The movement of the camera will cause the background geometry to blur (see Figure 3.2.1). When the background geometry is blurred, it will sample pixels belonging to the tracked object. This will result in a blurry halo around the tracked object. Since the object that the camera is tracking is stationary within the frame one would expect a crisp edge around it.

FIGURE 3.2.1 A moving background with a static object (left) leads to object and background pixels blending together. A moving object against a static background (right) looks correct when this happens.

The halos are fairly easy to fix since we already have objects tagged. When the pixel shader is collecting blur samples, we simply discard samples that come from a different object than the current one we are processing.

fl oat3 s am pl eA verage = 0.0f;

for (i nt i = 0; i < num S am pl es ; i ++) {

fl oat2 c urrentV el = pi xel V el oc i ty * (i +1);

fl oat2 texCoord = IN.texCoord0.xy + c urrentV el oc i tyS tep;

fl oat3 s am pl e0 = tex2D(QuarterS i zeMap, IN.texCoord0.xy+c urrentV el );

fl oat s tenc i l V al 0 = tex2D(S tenc i l S am pl er, IN.texCoord0.xy + c urrentV el );

fl oat s tenc i l Di s t0 = s tenc i l V al 0-bas eP i xel S tenc i l ; s tenc i l Di s t0 = s tenc i l Di s t0 < 1.0f ? 0 : s tenc i l Di s t0;

fl oat3 s am pl e1 = tex2D(QuarterS i zeMap, IN.texCoord0.xy-c urrentV el );

fl oat s tenc i l V al 1 = tex2D(S tenc i l S am pl er, IN.texCoord0.xy-c urrentV el );

fl oat s tenc i l Di s t1 = s tenc i l V al 1-bas eP i xel S tenc i l ; s tenc i l Di s t1 = s tenc i l Di s t1 < 1.0f ? 0 : s tenc i l Di s t1;

s am pl eA verage += fl oat4(s am pl e0*s tenc i l Di s t0, s tenc i l Di s t0);

s am pl eA verage += fl oat4(s am pl e1*s tenc i l Di s t1, s tenc i l Di s t1);

}

fl oat3 fi nal Col or = s am pl eA verage.xyz / s am pl eA verage.w;

A side effect of the halo-fixing step is that it creates a hard edge between objects with different velocities (see Figure 3.2.2). This artifact is exacerbated if the blur is performed on a lower-resolution target to increase performance. This causes the hard halo-fixed edges to exhibit pixel artifacts equal in size to the low-resolution target. In practice, however, these are not visually disturbing since they only occur on objects that are moving quickly across the screen.

FIGURE 3.2.2 Halo fixing leads to crisp edges around objects in the scene. This looks correct for a static object against a moving background (left) but less correct for a moving object against a static background (right).

INTEGRATIONWITHA POST-PROCESSING PIPELINE

This method of blurring is well suited for integration into an existing post-process pipeline. In our implementation we reuse a half-resolution frame buffer texture that has been previously generated. Depth of field could also be combined into the same pixel shader as the motion blur.

COPING WITH HARDWARE LIMITATIONS

The motion blur technique described above requires a GPU that is capable of indexing into constant registers in the pixel shader. If your platform is not capable of this, you can store the object matrices in a 1D floating-point texture.

Platforms that do not grant general stencil buffer read access to pixel shaders can instead put object IDs into the alpha channel of the frame buffer. If you use this method, it may become necessary to draw alpha-blended objects twice, once normally and then again to fill the frame buffer alpha channel with the correct value. Alternatively, an additional render target can be used to avoid extra passes over alpha-blended objects.

C^ONCLUSION

Replacing a velocity map implementation of motion blur with the method presented here has many advantages, but there are some tradeoffs. This method is best suited for rigid objects, such as cars, because it records a single velocity vector for each object. The technique is very fast, however.

Adding velocity vector calculation to the same shader as the blurring makes sense from a performance standpoint, since the velocity vector calculation consists mainly of arithmetic instructions, and the blurring operation relies on numerous texture reads. In practice, we achieved performance of just under two milliseconds for the entire blur operation on an Xbox 360 running in 720p resolution.

ACKNOWLEDGMENTS

I would like to thank Wolfgang Engel, Thomas Johnston, Raymond Kerr, and Steve Reed for discussing this motion blur technique with me and for providing their valuable insights.

REFERENCES

[Shimizu03] Clement Shimizu, Amit Shesh, Baoquan Chen, “Hardware Accelerated Motion Blur Generation,” University of Minnesota Computer Science Department Technical Report 2003, available online at www.dtc.umn.edu/~ashesh/our_papers/motion-blur.pdf

[Green03] Simon Green, “Stupid OpenGL Shader Tricks,” Games Developers Conference 2003, available online at http://developer.nvidia.com/docs/IO/8230/GDC2003_OpenGLShaderTricks.pdf

[Rosado07] Gilberto Rosado, “Motion Blur as a Post-Processing Effect,” GPU Gems 3, Hubert Nguyen, ed., Addison Wesley Professional, 2007

3.3 Real-Time Image Abstraction by Directed Filtering

J^AN E^RIC KYPRIANIDIS AND J^ÜRGEN D^ÖLLNER

FIGURE 3.3.1 An image abstracted using our framework.

Left: Original image (Copyright Anthony Santella). Right: Abstracted result.

INTRODUCTION

In this chapter we present a framework of automatic image processing techniques that create high-quality, simplified, stylistic illustrations from color images, videos, and 3D renderings. Our method extends the approach of [Winnemöller06] to use iterated bilateral filtering for abstraction and Difference-of-Gaussians (DoG) for edge detection. We present enhancements to these techniques to improve the quality of the output by adapting them to the local orientation of the input.

To smooth low-contrast regions while preserving edges, we present a fast separable implementation of the bilateral filter. Our approach works by aligning the bilateral filter with the local structure of the image. The first pass filters in the direction of the gradient, and the second pass filters in the direction of the tangent. To extract salient edges, we present a fast two-pass algorithm. A 1D DoG filter is applied in the direction of the gradient, followed by a second pass that applies smoothing along flow curves that are derived from the local structure of the image. Both techniques require a smooth vector field that represents local orientation. We show how such a vector field can be computed from the eigenvectors of the smoothed structure tensor. Besides gradient calculation, only linear separable smoothing with a box or Gaussian filter is necessary.

A schematic overview of our framework is shown in Figure 3.3.2. Input is typically an image, a frame of a video, or the output of a 3D rendering.

We start with the construction of the flow field by calculating the structure tensor. The structure tensor is then smoothed, and the eigenvector corresponding to the local tangent direction is stored in a texture map. Then the input is iteratively abstracted by using the orientation-aligned bilateral filter. We perform a total of n_α iterations. We apply color quantization to the result. After n_e n_aiterations, we extract edges from the intermediate result using the separable flow-based DoG filter. In our examples we use n_α and n_a = 4 Finally, the extracted edges are superimposed on the output of the color quantization.

FIGURE 3.3.2 Overview of our abstraction framework.