Discussion and Limitations

This method generates 3D cloud volumes from HDR images. However, HDR merging tech- niques for obtaining consistent sky images are still imperfect. Initial experiments with var- ious merging techniques produced different results. The irradiance value of overcast skies given in the literature compared with the irradiance value of the merged, overcast sky HDR images were too bright and on occasions too dark images. The best results compared to the physical values were obtained using the commercial software package Photoshop. Hence, the images merged with Photoshop are used in the results.

Another limitation of this method is the contribution of the inscattered light, path (6) in Figure 8.2, is not investigated. It causes a loss the light scattered and absorbed in the at- mosphere. With further improvements on atmospheric scattering in this limitation could eventually be overcome.

mented between the pixels which are classified as clear sky. The pixels classified as clouds are matched to the closest value in the histogram. If the sky is mostly covered with clouds, the histogram cannot obtain colour information from the cloud pixels. If these pixels are larger or smaller than the values of clear sky pixels, they will be matched to the brightest or darkest values in the histogram which will generate unnatural sky models. The solar irradi- ance generated by Hošek Sky Model is the incident radiance from the sun’s direction to the ground attenuated through the atmosphere. Using this model for the optimisation causes blue or red appearance of the clouds since the sun light at cloud level differs from the value generated by Hošek Sky Model.

The contribution of this method in comparison to the previous methods can be seen in Table 8.5.

8.7 Summary

This chapter has proposed a new cloud modelling and illumination technique generated from a single HDR capture. The cloud model is generated by minimising the error function between the rendered image of the proposed cloud model and the input image. The method contributes to the literature by using the optical and spatial properties of real clouds. The cloud model once generated can be relit with different sky scenarios. The level of accuracy obtained by the normalised RMSE results demonstrates that this method can be used as an environment map for lighting virtual environments.

Figure 8.8: Cloud modelling for cumulus cloud images. Left: Input image, Middle: Rendering the volume with the input sky. Right: Rendering the volume with a new sky

Figure 8.9: Cloud modelling for cumulus cloud images. Left: Input image, Middle: Rendering the volume with the input sky. Right: Rendering the volume with a new sky

Figure 8.10: Cloud modelling for stratocumulus cloud images. Left: Input image, Middle: Rendering the volume with the input sky. Right: Rendering the volume with a new sky

Figure 8.11: Cloud modelling for stratocumulus cloud images. Left: Input image, Middle: Rendering the volume with the input sky. Right: Rendering the volume with a new sky

Figure 8.12: Cloud modelling for cirrus cloud images. Left: Input image, Middle: Rendering the volume with the input sky. Right: Rendering the volume with a new sky scenario.

Figure 8.13: Cloud modelling for cirrus cloud images. Left: Input image, Middle: Rendering the volume with the input sky. Right: Rendering the volume with a new sky scenario.

Table 8.4: References for the models in the Section 3.7

Method Reference Method Reference Method Reference

Nishita et al., 1996 [1] Bouthors et al., 2006 [5] Dobashi et. al., 2010 [9] Dobashi et. al., 2000 [2] Bouthors et al., 2008 [6] Dobashi et. al., 2012 [10] Harris and Lastra, 2001 [3] Wither et al., 2008 [7] Yuan et al., 2014 [11] Schpok et al., 2003 [4] Dobashi et. al., 2008 [8] Presented Method [12]

Table 8.5: Comparison of the Methods. References of the methods can be found in Table 8.4

Methods Scattering Phase Function Cloud Model Generation Cloud Base Level Rendering Real Time Cloud Type

[1] S, M, A, G HG × × RS × Cumuliform

[2] S, A, G × User Defined F RS × Cumuliform

[3] S, M RY User Defined × RS _X Small Clouds:C u

[4] × × User Defined × RS × C u/St/w i sp y/C i

[5] S, M, A, G MT* User Defined F RS _X Stratiform

[6] S, M MT* × × RS _X Tested with:C u,Sc

[7] × × Generated from an Initial Draft F RS × Cu

[8] × × Generated from an Initial Draft × × X Cumuliform

[9] S, M × Fitting a Model to Input Cloud Image F × × Cu/Ac/C i

[10] S, M HG User Defined × RS × Any cloud photograph

[11] S RY Fitting a Model to Input Cloud Image × RS × Cu

[12] S, M , A, G, Cl HG Fitting a Model to Input Cloud Image C VPT _X Any Cloud Type

• S: Single Scattering • M: Multiple Scattering • A: Atmospheric Scattering • G: Reflected light from Ground • Cl: Reflected light from other clouds

• HG: Henyey Greenstein Phase Function • RY: Rayleigh Scattering

• MT: Mie Theory • F: Flat

• C: Curved

• RT: Ray Tracing • RS: Rasterization

• VPT: Volumetric Path Tracing • ×: Not investigated

13

Chapter 9

Conclusion

This thesis has presented novel clear sky and cloud illumination methods for Computer Graphics. Chapter 5 has presented a method to produce clear sky illuminations using a ma- chine learning technique trained with analytic sky models and environment maps. Chapter 6 outlined a new cloud modelling technique which was explained in more detail in Chapter 8. This technique uses the information gained from classifying the clouds with the method introduced in Chapter 7. A summary of these methods, contributions and future work is presented in this chapter.

9.1 A Machine Learning Driven Sky Model

A generic framework for sky light modelling including a fast and robust sky illumination method were introduced. The proposed method is suitable for both off-line and interactive applications. It is based on machine learning, particularly via the use of a neural network which learns the illumination from analytic sky models or captured environment maps and is capable of reproducing skies with similar properties. The neural network is trained on a, relatively, small number of features, and returns the incident radiance in a direction, for a given sun direction and turbidity value. A network architecture based on a single hidden layer model was chosen to maintain low memory costs and high computational efficiency. Training times for the method are relatively short, and it can potentially be used with newer analytical models if and when they are developed. In case of analytic sky models results have shown that it significantly speeds up the calculation time of two commonly used methods by Preetham et al. [117] and Hosek and Wilkie [61] with results that are accurate to within 4% of the originals based on normalised RMSE calculations. Furthermore, it enables the use of sparsely captured environment maps to produce animated sequences of sky lighting, which removes the need for intensive data capture in order to produce such animations. Re-

sults for the newly generated environment maps are within 2% of the captured images. The straightforward nature of the method permits a GPU implementation which can be run at sub millisecond speeds at high resolutions on modest GPUs.

9.2 Physically-Based Cloud Modelling

Clouds are an essential component of skies and are responsible for much of the appear- ance and lighting from the atmosphere. However, these are neglected by existing analytic sky models, and while there are methods for adding clouds to images in computer graph- ics, these are largely ad-hoc models without a strong basis in reality. This thesis addressed this problem, and developed methods for extracting clouds from captured real-world skies, which can then be reused in new sky configurations. This problem was solved by Chapter 7 and Chapter 8, outlined below.

9.2.1 Cloud Classification

A per pixel cloud classification method from captured whole sky HDR images was presented. This method used a patch-based feature extraction technique which provides statistical and textural features about the image; these features are clustered with k-Means Clustering. This technique was trained on a set of captured data, and the resultant clustered features were used to classify pixels in new whole sky HDR images. Results show an 80% success rate for per pixel classification, and 97% if this methods is applied to whole sky classification as is common in the literature. These results are on par with the state-of-the-art in Meteorological Science, with the added advantage that this approach provides per pixel results which no other method currently achieves.

9.2.2 Cloud Modelling

Subsequently, a framework for extracting clouds from HDR captures of the sky was pre- sented. First a theoretical foundation for the minimisation problem of matching rendered clouds to a captured image was introduced. This generic solution, however, is intractable, and this chapter developed methods for a practical solution to this problem. Based on Mie Theory, and the cloud classification, from the previous chapter, the dimensionality of the problem was reduced, and the range of parameters to be optimised bounded. Finally, an iterative approach to the optimisation was presented; this can compute a feasible represen- tation of the structure and optical properties of the clouds present in the original captured

image. Normalised results for synthetic images generated by this method demonstrate ac- curacy of up to 90% when compared to ground truth captures. The clouds extracted by this method can then be relit by arbitrary atmospheric and solar lighting conditions, thereby adding necessary cloud detail to analytic sky models. This enables real-world clouds to be used in computer graphics applications such as environment lighting for architecture.