Processing and Representation

As mentioned earlier, the captured light probe images are processed and represented in a way which is easier to store, edit and retrieve during the rendering. The first step is to create a ray-database from the light probe images. There are a few different ways to store this ray database next. The representation relevant to this work is the ILF slab and the Source Light Field (SLF). The next sections will describe them in more detail.

4.3.1.1 Ray-Database

Recalling the plenoptic function, the distribution of the light rays are represented as P = P(x, y, z,✓, ) or in short, P = P(x,!) where ! is the direction and x is the origin position. Each pixel in the captured light probe images represents a light ray with direction !. The capture positions serve as the origin position x for the particular ray. This way, every pixel in every captured images can be represented as an individual light ray. The whole spatial and angular distribution of these light rays are stored in a ray-database which is easier to retrieve during rendering.

Figure 4.9: Light probes to a ray-database. The environment has the light probes at position P1 P5. A sample light probe from the position P4 is shown which is converted to a Lat-Long environment map. Each pixel on this lat-long map is a light ray with direction(✓, ) and the origin position P4.

There are a number of ways a ray-database can be constructed based on the intended representation and application. The original literature describes methods such as the Ray-binning where the rays are grouped and stored by their angles so that every ray in an angular bin has the same angles but different origin positions.

Another approach makes use of volume based representation of the light rays in Energy voxels. As these approaches are out of scope for this work, details of both can be found in the original ILF literature [Unger, 2009].

This work represents the light rays in a ray-database where the light rays are grouped spatially. The spatial dimension in question can either be the captured spatial dimension in case of the simplest implementation of ILF described in the literature or a projected dimension on an imaginary plane. The later is achieved by a technique called the “ILF Re-projection” which is described next.

4.3.1.2 ILF Re-projection

The captured light probes often are irregularly spaced which results in a large pro- cessing time during rendering to find the correct light ray with the correct origin position from billions of light rays. For this reason, the ILF ray distribution is pro- jected to an imaginary bounding-box enclosing the rendering space. Each side of these boxes are called theILF plane[Unger, 2009]. The dimensions of the bound- ing box can be arbitrary as long as it encloses the capture area. This method has been described as the ILF re-projection in the literature [Unger, 2009]. Figure 4.10 describes the re-projection of light rays to the ILF planes (P1, P2, P3, P4, P5,) sur- rounding the capture area A.

Figure 4.10: The ILF re-projection. The red dots are the light probe samples in the shaded capture area. The bounding-box around the capture area are made of “ILF planes” named ⇧1 for the back side, ⇧2 for the ceiling, and ⇧3 for the left side in this diagram. The rays from the light probes are reprojected on these planes. The new projected rays will have the new origin positions (blue dots) on the ILF planes.

Apart from the ILF planes, Unger et al. also described a method to extract the illuminants from the ILF ray-database by a semi-manual process. This process uses thresholding to determine high energy rays and group them together as a light source while the operator (the user) manually decides the actual position and size of the light source from a visualisation. These high energy rays are stored separately in a similar spatial data structure as ILF planes and are called Source Light Fields (SLF).

Originally the ILF planes do not need to adhere to any specific dimension as long as the bounding-box is convex to the capture space. Only The SLFs needed to be in the correct place. However, for the specific purposes of developing the Temporal ILF (described in Chapter 6), the ILF planes are made with the dimensions from the boundary of the capture scene. For example, even if the ILF capture space is a small part of the entire scene, the ILF planes are made according to the boundary of the scene and not any arbitrary convex bounding box. This does not affect the rendering quality as the it is not dependent on the size of the position of the ILF planes. It however facilitates mapping the spatial ray-data database easily to the scene geometry. This is a crucial requirement of the works that has been presented in this thesis. Moreover, the SLFs are implemented in this implementation but the illuminant extraction technique is different from the literature which is further described in the next chapter as a part of the DCP technique. More clarification on this particular design decision has been discussed in both Chapter 5 and Chapter 6.