This approach aims to achieve interactive rendering performance by exploiting the texture mapping hardware. In general terms, there are two ways of doing this: either using 2D or 3D textures. The next sections present both approaches, highlighting advantages and disadvantages of each.
2.9.1
2D Textures
The idea of the 2D textures method is very simple, inspired by the earlier work of Lacroute and Levoy [86] (as we described in section 2.7): the original volume is sliced into three sets of 2D planes (images), each set perpendicular to one of the coordinate axes (figure 2.5). Then the bounding box of the volume is computed (which is normally called proxy geometry) and a series of texture mapped, parallel polygons are drawn from the back of this box towards the front, along one of the coordinate axes. To decide which axis to use (which also selects the set of textures), the angle between the polygon normals and the viewer is computed: the minimal angle gives the best set of textures to use. This approach automatically provides filtering, as the hardware is capable of bilinear filtering, and back to front compositing (as described in section 2.6.2), as colours and opacities are accumulated when drawing overlapping polygons (i.e. we assume that a transfer function
Chapter 2 27 Volume Visualization
has been applied to the original data). Note that the number of slices used directly influ- ences both image quality and rendering time - for optimal quality, this can be set to the maximum dimensions of the volume data, but not higher.
(a) X-aligned textures (b) Y-aligned textures (c) Z-aligned textures
Figure 2.5: Volume rendering through 2D texture mapping: the three sets of 2D textures required for this method are presented with a few slices each.
This is a very fast method and as it uses the commonly available 2D texture mapping, does not require specialized hardware. However, this method has some disadvantages: first, it requires three times the amount of texture space as the original data. Second, if the viewpoint or viewing direction changes, the texture set being used can abruptly be switched to another, and this causes a visible change in the final result. Finally, depending on the viewpoint, aliasing artifacts at the edge of polygons may be visible due to an insufficient sampling rate - as noted by Rezk-Salama [133], this cannot be solved as the number of polygons is fixed in this algorithm. Because of this, he proposed a solution based on multitextures [134], where the artifacts were minimized by interpolating the texture values between two adjacent slices.
As this 2D method is fast and can be easily implemented with almost all graphics hardware, some research has been done to allow this kind of visualization over networks, in particular the Internet. For example, Hendin et al [57] proposed a system based on Java and VRML to allow the visualization of volumes over the Internet, also providing isosurfacing integration into the rendered volume. Later, Engel and Ertl [41] created a similar system, but with multiple user capabilities (such as tagging of volumes and offering a chat feature) and also incorporating compression to minimize network traffic.
Chapter 2 28 Volume Visualization
2.9.2
3D Textures
This idea was first suggested by Akeley [5], and Cabral et al [24] published the seminal work on 3D texture-based volume rendering, complemented by Wilson et al [170] provid- ing the mathematical equations required to define 3D texture coordinates, and Cullip and Newmann [35] analyzing and discussing the implementation issues and the advantages of viewpoint-aligned versus object-aligned slices. In this case, as the hardware can directly work with 3D data, it is just a matter of loading the volume as a 3D texture and drawing texture-mapped polygons - now orthogonal to the viewing direction, again from the back towards the front of the proxy geometry. This process is illustrated in figure 2.6.
Figure 2.6: Volume rendering through 3D texture mapping (from [167]): a few slicing polygons are shown in the left image, and the texture-mapped result is presented in the centre image; finally, the right image displays the same volume, but rendering with a much greater number of slices.
However, as the polygons now are not necessarily aligned with a coordinate axis, the actual rendering algorithm is slightly more complex. It can be described by the following steps [61]:
1. Transform the proxy geometry vertices from world to view coordinates (see section 4.2);
2. Identify the minimum (zmin) and maximum (zmax) z coordinates of the vertices in
view coordinates. Create n slicing polygons, using equidistant spacing from the viewer, where n is computed by dividing the maximum number of slices (see below) by the sampling rate;
3. For each polygon from zmax to zmin:
(a) Check for all intersections with the edges of the proxy geometry. Store inter- sections in a vertex list - there will be at most six intersections (see middle polygon in figure 2.7);
Chapter 2 29 Volume Visualization
(b) Sort intersections in clockwise or counterclockwise order - this can be done by first computing the centre of the polygon formed by all intersected vertices and then calculating the angle between a vector from this centre to the first vertex on the x axis (as reference), and a vector from the centre to each vertex; (c) Create triangles from the sorted vertices (tessellation) and associate texture coordinates to each vertex - in OpenGL, the 3D texture coordinates for each vertex can be obtained by e.g. the texture generation functions (texgen) or by applying the same transformation to the texture matrix;
(d) Draw the texture-mapped triangles.
viewer
zmax zmin
Figure 2.7: View-aligned (3D texture) volume rendering: three slicing polygons are shown through the proxy geometry - the two triangles can be directly drawn but the poly- gon at the centre will have to be tessellated, as it has six vertices.
The maximum number of slicing polygons can be computed according to the longest possible diagonal through the volume. If (dimx, dimy, dimz) are the dimensions of the
volume in each direction, then the diagonal will measureqdim2
x+ dim2y+ dim2z - note that
this will be higher than any of the original volume dimensions. And as more slices require more rendering time, this is not desirable. Hence an adaptive algorithm can be used [21], which calculates the optimal number of slices according to the viewing direction: first the dot products between the viewing vector, and each coordinate axis scaled by the respective volume dimension are computed. The sum of these dot products will give the number of voxels that are visible along the viewing direction, so to obtain the optimal number of slices we just need to divide this result by the sampling rate. For example, supposing that we are viewing the volume directly from the front, then the dot product for the x and y axes will be zero and for the z axis it will be equal to the depth of the volume, i.e. dimz.
But as the observer starts moving to the right, the dot product for the z axis will become progressively smaller, while increasing for the x axis.
Chapter 2 30 Volume Visualization
A final observation regards the effect of the sampling rate over the opacity [43]: if we use fewer slices, then the opacity of each voxel will have to be increased so we maintain the same overall intensity in the final image. This can be done by a scaling formula, where α is the original opacity, s0 is the original sampling rate (e.g. 1), s is the desired sampling rate and α0is the corrected opacity:
α0= 1 − (1 − α)
s s0
2.9.3
Discussion
The 2D texture mapping method is very fast and is also available in most older graphics hardware. However, it requires three sets of textures, which adds considerably to its memory requirements - as opposed to the 3D textures method, which needs just a single 3D texture. The 3D textures method also automatically provides trilinear interpolation, whereas the 2D method only provides bilinear interpolation. This could be improved with the work described in [134] (see section 2.9.1), but the aliasing artifacts are just minimized and not completely eliminated. Another consideration is the sampling rate: because of the 2D slices, it is not constant in the 2D textures method, varying according to the viewpoint - obviously with 3D textures, this does not happen and the sampling rate is constant (at least for parallel projections and a reasonable approximation for perspective views). The slicing procedure itself leads to artifacts in the 2D case, especially as the angle between the viewing vector and the slice normal increases.
In summary, the texture mapping methods, in particular the 3D textures version, are possibly the fastest approaches to carry out direct volume rendering, efficiently exploiting the graphics hardware and allowing interactive rendering rates - this is essential for many applications, especially in medicine. However, if higher image quality is desirable then methods such as ray casting (section 2.6) or splatting (section 2.8) should be used instead.