Impact of Gamma Correction on Quality of Geospatial 3D Reconstructions though Photogrammetry

(1)

CSEIT1831404 | Received : 12 Feb 2018 | Accepted : 25 Feb 2018 | January-February-2018 [(3) 1 : 1609-1616 ]

1609

Impact of Gamma Correction on Quality of Geospatial 3D

Reconstructions though Photogrammetry

Swati R Maurya

*1

_{, Ganesh M Magar}

2

*1_{Department of Computer Science, S. K. Somaiya College of Arts, Science & Commerce, Mumbai, India} 2_{P. G. Department of Computer Science, S. N. D. T. Women’s University, Santacruz (W), Mumbai, India}

ABSTRACT

Coding luminance value using nonlinear operations on 2D images acquired using the camera and other imaging systems is a key aspect to explore the information contained in them. Gamma correction is a way to encode and decode luminance values in still images. If applied as a preprocessing step in any reconstruction activity, it has potential to reveal several important aspects especially in the reconstruction of 3D objects. This paper presents experimental validation capabilities and influence of gamma corrections on the 3D reconstruction of geospatial objects using the photogrammetric pipeline. The proposed model is applied and validated on natural terrain region and man-made building structure as well. Results show that for gamma values between 0.0 and 0.3, the surface parameters like number of point clouds, faces and mesh vertices show steep linear degradation and thus the reconstructed surface. The tie points controlling the photogrammetry exhibited power law behavior with error minimizing for gamma values after 0.6. This paper exhibits the relevance of luminosity corrections based on gamma values and explains its role in defining the structural properties of the objects during photogrammetric reconstructions.

Keywords: Gamma Correction, Photogrammetry, Point Cloud, Surface Reconstruction, Mesh Generation

I. INTRODUCTION

Reconstruction and the analysing surface of geospatial objects is now an important area of investigation. 3D reconstruction in more relevant than ever before due to improved hardware support and rendering capabilities. Advancements in virtual reality and mixed reality have also motivated research in the areas of 3D reconstruction. Reconstruction and visualization of natural and man-made geospatial structures have several applications. While 3D visualization of the spatial object can be done using LiDAR point clouds[1] [2], digital terrain data and digital elevation models[3]; they are often costly and may not provide high surface detail and accuracy.

Reconstructing 3D objects and scenes from multiple views [4] and color-Induced methods[5] shows new ways of experiencing 3D object around us. Popular

methods like photogrammetry-based 3D

reconstruction[6] can provide a low cost alternative for the same. However, photogrammetry highly depends on several parameters of quality of still images input to it. These parameters include brightness and contrast, hue and saturation, and gamma corrections. Each of these parameters has significant impact on the information conveying capability during visual inspection and needs further exploration.

(2)

the software module updates the gamma-encoded binary pixel values directly to the video memory. Therefore, it also has the visual impact on the color tone perception and thereby the objects contained in the image. Gamma correction has been a tool for enhancing the image-viewing experience.

Fundamentally, gamma corrections is all about reproducing levels of brightness of a given image similar to observed experience on the target output devices It implement methods to establish visual correspondences of the real world experience to screen viewing experience. However, its effect on the 3D reconstruction of objects is rarely studied. Since gamma corrections and adjustments are simple but effective means to analyse contents of the image, it is also interesting to test its effect on 3D reconstruction from digital images.

During the object reconstruction process, the effect of gamma cannot be overlooked and forms potentially important parameters to assess the quality of the reconstructed surface. The 3D spatial coordinates of points corresponding to the surface of the object obtained after gamma correction using well-known 3D reconstruction methods can be further analysed using shaded and wireframe models to validate and assess its performance on surface reconstruction. The resulting 3D surface would present an opportunity to explore the surface and deformation details due to varying gamma levels.

This forms the motivation behind applying gamma corrections to digital images prior to feeding them to the photogrammetric pipeline. A correspondence can be set between the gamma levels and acceptable quality of the reconstructed surface. This paper evaluates the effect of varying gamma over a range of values on digital images and evaluates the quality of reconstructed surface.

II. RELATED WORK

The effect of gamma adjustments finds application in several application areas. However, most of these research works were focussed extracting information in 2D forms only. It was observed that visual gamma corrections have several limitations [8] even for experience with LCD displays. It is often subjective in nature due to angular shifts and viewer visual capability of perception. Therefore mere gamma correction to match the visual experience in 2D is not sufficient.

An improvement of improving 2D experience was shown using adaptive gamma correction in [9] and suggested a way to overcome limitations of the image-capturing device. It discussed the use of weighting distribution together with gamma correction to highlight correct details in the image. Usefulness of gamma correction in enhancing images has also been highlighted in recent works such as in [10],[11] and [12]. While results are satisfying for 2D investigations of objects in the image, they clearly lack support for 3D and interactive user experience.

Recently, some attempts to study gamma correction for a three-dimensional object such as in [13] shows its usefulness in improving 3D user experiences. It implemented phase measuring profilometry and filter response of the whole image. However, they did not discuss surface topology in terms of discrete point arrangements and thus quantitative surface quality assessment was not possible. Based on the similar ideology of fringe projection. Few other 3D measurement research investigations through not directly implementing gamma correction include work by Yuankun Liu in [14] and ChaoZuo in [15]. Their work was largely based on fringe projection method. With nearly no or little work on assessing gamma correction in the 3D domain, we explore its

effect on the 3D reconstruction using

(3)

III. THE GAMMA CORRECTION

This paper focuses on using gamma corrected image data sets as a pre-processing step in the photogrammetric pipeline. Before we present the results of our findings, we describe the concept behind gamma corrections and image data sets used for the experimental verification purpose.

A. The Concept of Gamma Correction

Gamma correction is essentially a nonlinear operation [] to adjust luminance values in the still images to improve and align the human perception of illumination conditions of the image. From a technical perspective, the gamma correction obeys power-law expression or approximates a power function. Use of gamma is a way to encode and decode luminance in images and thus becomes a potential means to analyse and highlight contents in the given image.

The nonlinear behavior of gamma correction can be modeled in a formal way. The relationship between the perceived Intensity value denote by I and the image signal U normalized in the scale 0 to 1.0 can be expressed by a power function as

…(1)

Where, describes the maximum allowed Intensity value and set as constant for the given setup. The value of corresponds to the gamma of the output system. The quality of the perceived image can vary for the range of values. Together with other parameters, it can significantly influence the graphics and rendering pipeline and hence the reconstruction process. A typical response is shown in Figure 1.

Figure 1. Input-Output Gamma Response and Correction

In a typical situation to produce intensity levels identical to what the acquiring device captured initially and aligning then to the viewing experience, the signals corresponding to the visual perception are to be modified. Depending on the human perception and visualization, systems like CRT and LCD screens gamma values may be adjusted for a range of values. This is required to compensate for the non-linear responses of the output. Most digital images generated from modern image acquisition system are gamma encoded. The typical gamma values used in is set near the value 2.2.

B. Evaluating Gamma Correction for 3D Reconstruction

The effect of adjustments such as gamma correction is studied with less rigor in 3D domain especially. Even though the images can use non-linear encoding to generate optimal results for human perception, they may not produce the desired results in texturing and describing 3D objects. Notably, the algorithms in 3D also largely assume linear encoding of intensity and brightness and therefore their effect during object reconstruction may not be as per the usual expectations.

Some key facts that drive the efforts to test the contributions of gamma correction in 3D reconstruction are listed below.

(4)

 Computing image values do not necessarily depend only on RGB values especially in a linear way.

 Software interpretation of brightness is quite different from reality. For example, the 50% brightness in software may correspond to a brightness value of 22% in reality.

 Problems additionally occur due to global illuminations, motion blur, and presence of transparent objects.

The human experience is essentially a non-linear perception of changes in physical attributes including intensities and color. More importantly, these changes are relative and not absolute ones and therefore can be measured on the logarithmic scales and satisfy power law responses. It is seen that there is a perception gap when treating aspects of images in 3D space. Hence, it is required to test and validate these theoretical responses to its counterpart 3D domain. This can be done by testing these gamma-encoded images further for range of gamma values for correcting them for visual performance and assessing them for quality of surface reconstruction.

IV. MATERIAL AND METHODS

With sufficient motivation behind extending gamma correction, this paper evaluates the effect of gamma

corrections in 3D reconstructions using

photogrammetry. The input dataset for evaluating the effect of gamma corrections comprised of a multi-view photography of a portion of a building. The data set used in this paper is described next. A. Dataset Description

The digital image dataset input to the photogrammetric pipeline comprises of a set of six digital photographs taken from Canon EOS 5D digital camera with F-stop value and exposure time of

sec. The ISO speed was ISO-200 and focal length of the camera was 24mm. The resolution of the input images were . The horizontal and vertical resolution was with 24-bit depth

These images in the data set shows a portion of a man-made textured building in daylight (see Figure 2).

Figure 2. Input Data Set-Building from Multiple Views

These image samples were acquired from digital camera in bright sunlight with a clear sky.

B. Integrating Gamma Correction with Photogrammetric Pipeline

It is known that gamma correction improves visual perception on the output devices. However, their effect on the quality of 3D reconstruction is not known. There are several aspects for evaluating the quality of surface reconstruction including a number of tie points, number of points in the point cloud, and number of faces in the reconstructed surface.

One well-established method is to use

photogrammetry. Our method proposes the

(5)

help to study their effect on the point cloud generation and their spatial positioning in 3D space. Note that under and/or over gamma-correction of images can affect the color balance and thus quality and capability of 3D reconstruction methods from 2D intensity maps stored as images. Therefore, a careful math and empirical analysis are required for choosing suitable values of gamma. The proposed model of integrated gamma correction with photogrammetry is shown in Figure 3.

Figure 3. Integration of Gamma Correction in Photogrammetric Pipeline

Each image in the image data set is corrected for a range of gamma values typically between 0 and 3.0 and stored as a separate sample. Then the set of images with specific correction is used as input and the point cloud is generated using photogrammetry. These point clouds are further decimated for surface mesh generation and further for textured 3D components of the pixels of the image and pow is the power function.

V. RESULTS AND DISCUSSION

(6)

(c) Gamma-Mesh Vertex Response

(d) Gamma-Surface Face Count Response

(e) Gamma-Point Cloud-Face Response

Figure 4. Quantitative Response after 3D Reconstruction

A careful analysis of the reconstructed surface on different parameter provides insight into the response of the photogrammetry on reconstruction capability due to gamma corrections. The relationship among various surface parameters are presented by the statistical analysis in Table 1.

Quartile 2848 2240464 75981 149364 Median 3108 2254645 76439 150309 Third

Quartile 3153 2267447 76838 151163 Standard

Deviation 534.60 286813.54 8270.80 16372.30

Variance 285802.85 82262009230.37 68406259.55 268052333. 81 Standard

Error 7.80 4189.4033 120.80 239.145

The inter-relations ship between various surface parameters is shown in Figure 5. It suggest that there is a fairly good correlation between gamma values and tie points (control points) in the reconstructed surface after applying photogrammtery. This in turn at later stages in the photogrammteric pipeline exhibit higher association with point cloud and mesh vertices respectively.

Figure 5. Correlation between surface parameters

The performance and the effect of range of gamma corrections on 3D reconstruction is evident from the squarred error shown in Figure 6. This graph clearly shows that the number of tie points controlling the photogrammetry and hence the 3D reconstruction

degrades exponentially below . However, the

number of faces on the reconstructed surface shows steep linear degradation for values below . For values between 0 and 0.25 the point the resulting point cloud and mesh vertices shows large error in the reconstructed surface.

1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930

Cou

(7)

The visual inspection of the reconstructed surface also validates the quantitative results as shown in figure 7. It is clearly seen that for very low gamma values the surface parameters are too erroneous and therefore result into imperfect structures and in some cases result in insufficient data. Acceptable performance for reconstruction is obtained for gamma values between

to . Although beyond there is washout effect on the textured surface after reconstruction but the surface structure is preserved. The portion of final reconstructed mesh and textured surfaces are shown in Figure 7.

(a) Mesh Surface (left); Textured Surface (right) for

(b) Mesh Surface (left); Textured Surface (right) for

(c) Mesh Surface (left); Textured Surface (right) for

(d) Mesh Surface (left); Textured Surface (right) for

Figure y. Portion of Reconstructed Surface

VI. CONCLUSION

The effect of gamma corrections on the quality of 3D surface reconstruction has been highlighted in this paper. It was seen that surface tie point suffer exponential loss for lower values of gamma. The number of points in the point cloud, number of faces suffer from steep linear loss for gamma values below 0.3. For higher values of gamma, there is relatively less loss as far as spatial occupancy of surface points are concerned but the textural details are washed-out. Compared to the capability of two-dimensional gamma analysis, three-dimensional aspects based on gamma correction are more useful in understanding the effect of luminosity contribution in surface reconstruction. The nonlinear and linear responses of various aspects for 3D reconstruction by integrating gamma correction in photogrammetric pipeline has been successfully demonstrated.

VII. REFERENCES

[1]. O. Kreylos, G. W. Bawden, and L. H. Kellogg, "Immersive Visualization and Analysis of LiDAR Data," Springer, Berlin, Heidelberg, 2008, pp. 846-855.

[2]. B. Kovač and B. Žalik, "Visualization of LIDAR datasets using point-based rendering technique," Comput. Geosci., vol. 36, no. 11, pp. 1443-1450, Nov. 2010.

[3]. S. S. K. Pratibha P. Shingare, "Review on Digital Elevation Model," Int. J. Mod. Eng. Res., vol. 3, no. 4, pp. 2412-2418, 2013.

(8)

[5]. F. Li, Y. Du, and R. Liu, "Color-Introduced

Frame-to-Model Registration for 3D

Reconstruction," Springer, Cham, 2017, pp. 112-123.

[6]. J. Baqersad, P. Poozesh, C. Niezrecki, and P. Avitabile, "Photogrammetry and optical methods in structural dynamics - A review," Mech. Syst. Signal Process., vol. 86, pp. 17-34, Mar. 2017.

[7]. "Deriving the Standard Formula for Gamma Correction," in Colour Reproduction in Electronic Imaging Systems, Hoboken, NJ, USA: John Wiley & Sons, Inc., 2015, pp. 667-672.

[8]. C. Alejandro Parraga, J. Roca-Vila, D. Karatzas, and S. M. Wuerger, "Limitations of visual gamma corrections in LCD displays," Displays, vol. 35, no. 5, pp. 227-239, Dec. 2014.

[9]. S. Rahman, M. M. Rahman, M. Abdullah-Al-Wadud, G. D. Al-Quaderi, and M. Shoyaib, "An adaptive gamma correction for image enhancement," EURASIP J. Image Video Process., vol. 2016, no. 1, p. 35, Dec. 2016. [10]. G. K. S. H Singh, Anil Kumar, L. K. Balyan,

"Dark image enhancement using optimally compressed and equalized profile based parallel

gamma correction - IEEE Conference

Publication," in 2017 International Conference on Communication and Signal Processing (ICCSP), 2017, pp. 1299-1303.

[11]. I. M. O. Widyantara, N. M. Ary Esta Dewi Wirastuti, I. M. D. P. Asana, and I. B. P. Adnyana, "Gamma correction-based image enhancement and canny edge detection for shoreline extraction from coastal imagery," in 2017 1st International Conference on Informatics and Computational Sciences (ICICoS), 2017, pp. 17-22.

[12]. Y. Chang, C. Jung, P. Ke, H. Song, and J. Hwang, "Automatic Contrast Limited Adaptive Histogram Equalization with Dual Gamma Correction," IEEE Access, pp. 1-1, 2018.

[13]. X. Ye, H.-B. Cheng, H.-Y. Wu, D.-M. Zhou,

dimensional object measurement by phase measuring profilometry," Opt. - Int. J. Light Electron Opt., vol. 126, no. 24, pp. 5534-5538, Dec. 2015.

[14]. Y. Liu, Q. Zhang, and X. Su, "3D shape from phase errors by using binary fringe with multi-step phase-shift technique," Opt. Lasers Eng., vol. 74, pp. 22-27, Nov. 2015.