Image Distortion Correction

By placing the virtual camera in various positions and orientations within the sphere, it was observed that an undistorted view can be generated when the virtual camera is placed at or near the centre of the sphere. As the virtual camera moves away from the sphere centre, distortion begins to appear, and becomes severe when the virtual camera is positioned near the inner sphere wall. The distortion is clearly visible in Figure 4-7 and Figure 4-8 shown in the last section, where the vertical buildings are seen to be curved outwards.

4.3.1 Distortion Illustration

In order to show the distortion effect in a clear manner, a regular texture map based on multiple strips (shown in Figure 4-9) is used for image generation. The corresponding views generated based on the virtual camera being placed at the same viewing points as those in Figure 4-7 and Figure 4-8, are shown in Figure 4-10 and Figure 4-11, respectively.

49

Figure 4-10 Texture map viewpoint result 1

Figure 4-11 Texture map viewpoint result 2

Based on the image results shown above, the cause of the distortion was analysed by comparing the virtual camera positions with and without distortion. As the original spherical image is acquired at the centre of the unit sphere, this is the distortion-free point because the scene captured by the virtual camera at this point corresponds to the scene captured by the real camera. If the virtual camera is moved forward or backward with its viewing direction maintained at normal with respect to the sphere wall, an undistorted view is also generated as shown in Figure 4-12 with the virtual camera placed at ( ), , and .

50

Figure 4-12 Distortion free viewpoint result

As shown by the 3D visualisation model on the left of Figure 4-12, the centre of the virtual camera and the four corners of the captured area form a pyramidal polyhedron – . The base of the pyramid is a regular spherical surface, the principal axis (the green arrow) is perpendicular to the base, and the four edges are equal in length, i.e. .

When the virtual camera is moved away from the sphere centre, and the principal axis is no longer perpendicular to the base (as shown on the left part of Figure 4-7 and Figure 4-8 in the last section), the polyhedron becomes an oblique pyramid. The irregularity of the pyramid can be measured by comparing the different lengths of the four edges , , , and . The greater the difference is, the worse the distortion occurs.

4.3.2 Methodology and Implementation

This sub-section presents the methodology for correcting the image distortion, and the method is based on generation of an intermediate projection plane acting as the base of the polyhedron to provide a better balance between the lengths of the four edges. The introduction of the intermediate projection plane results in two projecting steps for image generation, namely, projection from the spherical surface to the intermediate projection plane, followed by projection from the intermediate plane to the image plane of the virtual camera.

There are a few possible positions where an intermediate projection plane could be placed. Using the coordinates of the original points on the sphere being captured by the virtual camera as reference positions (see Figure 4-13(a)), Figure 4-13(b), Figure

51

4-13(c), and Figure 4-13(d) show three options for the placement of the intermediate projection plane.

Figure 4-13 (a) Original points on the sphere captured by virtual camera (b) Option 1 for intermediate projection plane (c) Option 2 for intermediate projection plane (d) Option 3 for intermediate projection plane

Whilst Figure 4-13(b) shows the intermediate projection plane being anchored at the four corner points of the captured area, Figure 4-13(c) shows the intermediate projection plane being placed based on the result of least squares plane fitting (Li and Zhang, 1998) of all the captured points on the sphere, and Figure 4-13(d) shows the intermediate projection plane being placed at tangent to the captured spherical surface with respect to the principal axis of the virtual camera (shown as the green arrow).

To choose from the three options, some tests have been carried out to compare their results. Although all of the options were seen to reduce the curvature distortion,

52

different positions of the intermediate projection plane were seen to cause the image content in the generated view to be shifted by different extents, due to the varying distance between the intermediate projection plane and the original spherical surface position. With the image contents found to be shifted the least from the original view based on Option 2, it is considered as the optimum intermediate projection plane position.

With the optimum position for the intermediate projection plane decided, the algorithm for visualisation with the virtual camera being placed at an arbitrary position within the sphere can be divided into four steps. The first step is to estimate the immediate projection plane. To do this, the fundamental part of the visualisation process presented in the last section is repeated firstly, i.e. the virtual camera is placed within the unit sphere to capture points on the sphere. The captured points are then used for fitting a plane among themselves in a least squares sense. If points on the fitted plane (or the immediate projection plane) are denoted by , and is any point on this plane, the application of least squares fitting gives:

( ) Equation 4-9

where is the direction cosines of the normal to the fitting plane.

The second step is to locate the points captured by the virtual camera on the intermediate projection plane, and this is similar to locating the captured point on the sphere as explained in Sub-section 4.2.2. For a point denoted by on the intermediate projection plane with the distance to the camera centre denoted by , if denotes the coordinates of the camera centre, and denotes the direction vector of , then following the approach described in Sub-section 4.2.2 gives:

Equation 4-10

Substituting Equation 4-10 in Equation 4-9 gives:

( ) Equation 4-11

Solving Equation 4-11 for gives:

( ) Equation 4-12

Substituting Equation 4-12 in Equation 4-10, all the points on the intermediate projection plane being captured by the virtual camera can be located.

53

The third step is to find out the corresponding points of on the sphere. As the sphere is modelled as a unit one, this is done simply by normalising the modulus of to one. The fourth and final step is to do image interpolation based on the coordinates of the points on the sphere (same as that explained in Sub-section 4.2.2) and produce the distortion corrected results. Target to source mapping with bilinear interpolation is again used.