Synthesis of imagery based on the principles outlined in this chapter is known as rendering. The following sections outline common rendering approaches for surfaces and participating media, with a focus on those used in this thesis.
2.4.1 Rasterization
The main idea behind rasterisation is to project the geometry of the scene, typically triangles, onto the image plane. To resolve visibility for the primitive, the depth of each fragment is compared to the previously stored depth (if it is less, then it is visible) using a data structure called a z-buffer. The colour values stored at the vertices of the primitives are interpolated in order to shade the fragment. This technique is useful when one needs speed but on the other hand it cannot naturally deal with all types of light transport such as reflection, refraction and global illumination. Therefore, it is not considered further in this thesis.
2.4.2 Ray tracing
Figure 2.6: Primary and secondary rays in Ray Tracing
One of the earliest rendering techniques, ray casting, was developed by Appel [3]. In this method a virtual camera position is placed into the scene and for each pixel on the image plane a ray starting from the camera position to a pixel in the image plane is generated. The ray starting from the camera is known as primary ray and for each primary ray the first hit point in the scene is calculated. Afterwards another ray called shadow ray is sent from the hit point to the light source to check if the point receives light from the light source. Whitted [145] improved this method by keep tracing rays which are called secondary rays in the scene. When the ray hits an object in the scene, according to the material of the surface it is reflected or refracted. The shadow, reflected and refracted rays are secondary rays and
shown in the Figure 2.6. This method produces more realistic lighting effects compared to Appel [3] such as shadows and refractions.
2.4.3 Path Tracing
Figure 2.7: Path Tracing
Path tracing was introduced by Kajiya [74]. This is based on using the ray tracing methodol- ogy with additional stochastic sampling in order to simulate full global illumination effects. The central idea is to calculate the contribution of all possible light sources to the current hit point and let the ray to bounce until a stopping criteria is met.
L(x→wo)=Le(x→wo)+Ld(x→wo)+Li d(x→wo) (2.47)
whereLe is the light emitted directly from the point,Ld is direct lighting andLi d is indirect
lighting, see Figure 2.7. The direct illumination corresponds to evaluating the radiance di- rectly coming from the light source. This is known as Next Event Estimation. For each hit point in the scene, a point on the light source is sampled and the visibility between the hit point and the point sampled on the light source is computed. Direct lighting is calculated as:
Ld(x→wo)= 1 N N X i=1 fr(x,wi→wo)Le(x←wi)cos(Nx,wi) pd(wi) (2.48) whereLei is the incident radiance in the direction ofwi, andpd(wi) is the PDF of sampling
the light source in the directionwi.
Indirect illumination calculates the interaction with other objects within the scene before reaching a pointx. This is a recursive process which typically samples the BRDF, generates a new point via ray tracing, and then computes direct and indirect lighting at that point by
using the reflected lightLr, see Equation 2.20: Li d(x→wo)=
Z
Ωfr(x,wi →wo)Lr(x←wi)cos(Nx,wi)d wi (2.49)
This tracing process needs a stopping condition to prevent paths from having an infinite length. For this purpose Russian Roulette is used as an unbiased stopping condition. Path tracing can compute all indirect light paths except those from point light sources seen though specular surfaces, as the probability of generating these paths is zero.
The rendering steps of path tracing is explained in Algorithm 1.
Algorithm 1Path Tracing Algorithm
1: foreach pixel in the image planedo
2: Compute the ray direction from eye through the pixel
3: Find the first intersection point of the ray with scene
4: Compute direct illumination valueLd
5: whilenot terminateddo
6: Sample a new directionw
7: Compute indirect illuminationLi din the direction ofw
8: Find the first intersection point of the ray with scene
9: Compute direct illumination valueLd
10: Decide to terminate path
2.4.4 Light Tracing
Figure 2.8: Light Tracing
Light Tracing by Arvo [4] deals with the opposite idea of path tracing. Instead of sending rays from the eye, they are sent from the light source into the scene. At each hit point, a deterministic connection is made to the camera and radiance is splatted onto the image plane. Similar to path tracing, this method can fail to connect a specular surface to a pinhole camera as the probability of generating this path is zero. However, it is an effective way to simulate caustics which are difficult to implement by path tracing, see Figure 2.8.
2.4.5 Bidirectional Path Tracing
Figure 2.9: Bidirectional Path Tracing
This method by Veach and Guibas [135] and Lafortune and Willems [89] generalizes both path tracing Section 2.4.3 and light tracing Section 2.4.4. It combines the advantages of the paths that these two methods can generate. In this method two paths are generated simultaneously, one from the light source (light pathxLof lengthNL) and another is from the
eye (eye pathxE of lengthNE) and the end point of each subpath is combined. Ifi<NE and j<NL are indices of path vertices on the eye and light paths respectively, a path is created: xE(i)xL(j). Each connection between the two subpaths evaluates the geometry term and
BRDFs of the vertices at the end of each subpath. This path then has to be weighted by all the possible ways it could have been created. For example, the same path could have been generated usingi−1 light path vertices and j+1 eye path vertices. In order to reduce variance, Multiple Importance Sampling is used to weight the sampled combination against all the possible combinations of subpath vertices. This process is repeated for allNE andNL
vertices, and is illustrated in Figure 2.9.
2.4.6 Metropolis Light Transport
Metropolis Light Transport by Veach and Guibas [137] is another approach to solving global illumination problems. This method is similar to the idea in the Metropolis Sampling algo- rithm. Initially a set of pathsx0,x1, ...,xN are generated (the authors used bidirectional path
tracing for this purpose). These paths are mutated by deleting, adding or replacing subpaths according to the Metropolis Hastings algorithm.
Three types of mutation strategy are introduced by Veach and Guibas [137]. In theBidirec- tional Mutationstrategy, a subpath of the path is selected to be deleted and a new subpath is interchanged with the deleted one. If the deleted and added subpaths have similar length, then the new subpath will have a high probability of being accepted. Another type of mu- tation is calledPerturbations. Perturbations make small changes to an existing path. These
are efficient at exploring regions of path space which make a high contribution to the image plane, such as caustics. InLens Subpath Mutationa subpath which ends at the eye is cho- sen to be deleted and replaced. This mutation type helps to stratify samples over the image plane.
The MLT algorithm works as follows:
Algorithm 2Metropolis Light Transport Algorithm, by Veach and Guibas [137]
1: Specify an initial pathX
2: setimageto an array of zeros
3: forj←1 toNdo
4: Y ←mut at e(X)
5: a←Accep t P r ob(Y |X)
6: Generate a random numberU
7: ifU<athen
8: setX ←Y
9: Recor d Sampl e(image,X)
10: Returnimage
There has been research into enhancing the MLT algorithm. Cline et al. [24] created short chains of mutations and improved stratification over the image plane. Kelemen et al. [76] used Primary Sample Space to mutate a path. Primary Sample Space consists of the unit hy- percube of random numbers, and a single coordinate in this space defines a path. Two types of mutation strategies are defined in this approach: small mutations make slight changes to the random numbers which generate a path, and large mutations generate a new set of random numbers and therefore a new path, which ensures ergodicity.
2.4.7 Photon Mapping
Figure 2.10: Photon Mapping.
The Photon mapping algorithm was introduced by Jensen [71]. It is based on two steps. In the first step, photons from the light source are shot into the scene and their energy is stored
at each hit point. To perform efficient lookups in the second step, the photons are stored in a kd-tree structure. In the second step, rays from the camera are shot into the scene and for each hit point the flux of the photons within an area of the hit point are used to evaluate the radiance coming to the camera. By using density estimation from the stored photons, the exitant radiance at a pointxis estimated:
L(x→wo)= n X p=1 fr(x,ωo,ωi)∆φp(xo,ωi) πr2 . (2.50)
whereris the radius of the sphere containing thennearest photons,∆φpis the flux received
by the pointxcoming from the photon p in the direction ofωi andωois the outgoing di-
rection. This method is limited by the number of photons that can be stored in the memory. It also generates biased results, but consistent. To reduce the low frequency noise, a final gathering is commonly implemented: rays are sent from the camera and at each primary hitpoint, the BRDF is sampled and rays are shot. Density estimation is used to calculate the outgoing radiance at each of the secondary hitpoints.
2.4.8 Progressive Photon Mapping
This method by Hachisuka et al. [51] solves the memory issues of Photon Mapping. As a first step, rays are shot from the eye to calculate primary hit points. These hit points store information describing the position, flux and radius. In subsequent passes, photons are shot from the light sources. When a photon falls within the footprint of a hit point from the first pass, its transported flux is added to the stored quantity at each hitpoint. Additionally, the radius is reduced such that the estimate converges in a consistent manner. This is expressed as: L(x→wo)≈ 1 4A n X p=1 fr(x,ωo,ωi)4φp(xp,ωi)= 1 πR(x)2 τ(x,ωo) Nemi t t ed (2.51) whereτ(x,ωo) is the sum of multiplication of the BRDF and unnormalized flux forNphotons
at the pointx.
2.4.9 Irradiance Caching
In diffuse scenes, Ward et al. [143] made the observation that irradiance changes slowly across the surfaces of the scene. To exploit this, irradiance is cached at a sparse set of lo- cations in world space, and the irradiance values are interpolated between cache points. Cache entries are added when rays from the camera do not find any nearby points, or if the
estimated error exceeds a threshold. A cache entry is computed via Monte Carlo sampling of light paths over the hemisphere surrounding the cache point. As this method stores irradi- ance, Irradiance Caching can only be applied to diffuse surfaces.
Irradiance caching was extended glossy surfaces in the radiance caching algorithm Krivanek et al. [86]. This method is used to render scenes with low frequency glossy surfaces and uses spherical harmonics to represent the incoming radiance.
2.4.10 Instant Radiosity
Instant radiosity by Keller [77] shoots paths from the light source into the scene and at each hit point stores a point light source, known as a Virtual Point Light (VPL), on the surfaces. A second pass approximates both direct and indirect lighting as the average of the lighting contributions from each VPL:
L(x→ωo)= 1 Nv pl s Nv pl s X i=1 Li(xi→x)f r(x,wi →wo)G(xi↔x) (2.52)
whereNv pl s is the total number of stored VPLs,Li(xi →x) is the emitted light from thei0t h
VPL in the direction tox. xi is the position of thei0t h VPL andwi is the direction from the
VPL to the hit pointx.
The accuracy of this method depends on the number of virtual light sources. As the number of light sources increases, the computational cost required to produce images also increases. To deal with this problem, many lights methods have been proposed which do not use all VPLs to reconstruct the image, but use a subset which minimise some error metric, such as Walter et al. [140].
2.4.11 Vertex Connection and Merging
Georgiev et al. [44] combines photon mapping with bidirectional path tracing (BPT) and gains the advantages of both methods. This method uses the path sampling technique of BPT with density estimation used in photon mapping. Light paths are generated from the light sources, and a path vertex is stored at each interaction with the scene. Then eye paths are created which connect each vertex on the eye path to the light paths in the vertex con- nection step. Density estimation is also performed at each eye path vertex, in an approach the authors called vertex merging. These approaches are weighted by Multiple Importance Sampling in order to reduce the variance from the different path sampling approaches.