8.3 Performance evaluation
8.3.5 Comparative results
Fig. 59 shows the MAE performance of the estimation techniques presented in Section 8.2 applied to the Tmax data of the longest 8 stations listed in Table 3, after performing adjusted QC. 80015 75031 82039 80023 49002 74128 74034 77042 Station ID 0.00 0.25 0.50 0.75 1.00 MAE [ ◦C]
Comparative estimation error performance (with adjusted QC data)
Plain IDW Modified IDW Plain ANN Modified ANN
Figure 59: MAE results for the three estimation techniques.
The two plain techniques can be used to estimate temperature data at unsampled loca- tions, whereas the modified techniques are intended to infill missing data in existing time series. The plain techniques can also be used to infill missing data, although this is not what they were designed for. The performance of each plain technique will be the same whether it is used to estimate data at unsampled locations or if it is used to infill missing data, as the plain techniques assume there are no data available at the location where estimation is to be performed.
From Fig. 59 it is clear that modifying the data improves performance for all cases shown. Plain ANN is better than plain IDW in almost all cases shown, and the ANN technique would therefore generally be superior in estimating data at unsampled locations. The two modified techniques show similar results, and would therefore provide similar infilling performance.
9
Discussion
This report considered the reconstruction of historical temperature data with a focus on Australia. Several aspects of temperature measurement were discussed to highlight the need for temperature reconstruction. Factors contributing to a discontinuous and non- homogeneous temperature record, including changes in measurement methods and the surface area surrounding weather stations over history, were discussed in Section 2.
The standard approach of dealing with unwanted artefacts present in the temperature record is homogenisation. Homogenisation methods and the official temperature recon- struction of the Australian BoM known as ACORN-SAT were presented with the associ- ated historical temperature trends in Section 3.
The aim of the study presented in this report is to investigate alternative temperature reconstruction techniques, starting from the raw measurements made available by the BoM. Part of such an investigation is finding methods that can be used as benchmarks, that are easily reproducible for any geographical area. To that end, an overview of a number of existing spatial interpolation techniques were given in Section 4. These techniques are commonly used in the estimation of spatially-distributed environmental variables, including temperature. Inverse-distance methods were found to be suitable benchmarks for the development of alternative temperature reconstruction techniques.
As the temperature reconstruction methods considered in this report use raw measure- ments, quality control of the data must first be performed to remove potentially invalid measurements. Quality control techniques including control charts and nearest neighbours were presented in Section 5. Control charting is a statistical process control technique commonly used in manufacturing industries, which may also be applied to evaluate the validity of individual temperature series in isolation. The nearest-neighbour technique was presented as an alternative to identify outliers in time series by comparing individual values with their geographic neighbours for the same time index.
Two temperature reconstruction techniques alternative to the homogenisation process were presented in this report, including the nearest-neighbour technique in Section 6, and the neural network technique in Section 8.
The nearest-neighbour technique was used to infill all monthly TmaxandTminseries across Australia and for each state separately, from which long-term trends were determined. The infilled series and trends were also compared with ACORN-SAT and found to be
similar. The similarity between ACORN-SAT and the nearest-neighbour anomaly trends was found to be particularly striking for Victoria, which has arguably the best station distribution in Australia. The areas between weather stations are small across Victoria, resulting in the raw data already being approximately area weighted. This is probably the reason why the area-weighted homogenised ACORN-SAT trend and the trend derived from the nearest-neighbour infilled raw data was found to be nearly identical for Victoria.
In preparation for the neural network reconstruction, background AI and neural network theory were given in Section 7, with further mathematical derivations in Appendix A. The reconstruction technique was then presented in Section 8, with the focus on an area around Deniliquin, a town in New South Wales. The technique was compared with inverse distance methods when several individual Tmax series were estimated. Overall, the neural network technique was shown to perform better than IDW when temperature data at unsampled locations are to be estimated. Further development and performance evaluation are required before the neural network technique could be used to estimate historical temperature trends over larger regions.
10
Future work
Areas of future work to improve the temperature reconstruction methods presented in this report are discussed below.
• The ANN technique could be developed further to improve its performance. Other ANN structures and training strategies could be investigated. Performance could possibly be improved by training the network on entire time series at each location, instead of only one temperature sample. Other inputs could also be included to further guide the training process.
• Further experimentation is needed to improve station selection for infilling and esti- mation purposes. In this report, the closest N neighbouring stations were included to perform estimation. An improved method may be to rather select stations based on their correlation with neighbours, and excluding those stations that don’t corre- late well (even though they may be located closer).
• The estimation techniques considered in this report should also be tested on other and larger areas. Ultimately, a continental reconstruction should be performed.
Other aspects that need to be investigated further are considered below.
• The effect of UHIs (discussed in Section 2.2) on the temperature record should be considered further. The BoM excludes 8 urban sites (as discussed in Section 3.2) when calculating regional averages, whereas the alternative reconstructions pre- sented in this report included all available data. It is also possible that other sites may be affected by UHI and a mechanism to detect and remove such artificial warm- ing should be considered.
• The effect of changing from classical thermometers to AWSs (as discussed in Section 2.1) should be investigated further, as this may also contribute to non-climatic warming. Excluding AWS measurements is one approach, although this will remove a significant amount of recently-recorded data from the analysis. A separate study on the effect of the change should be conducted, which will involve statistical analysis of parallel data (classical and AWS measurements at the same location).
• The effect of station relocations should also be revisited. The station locations provided in the BoM weather station directory [108] were used to train the ANN in this report. These locations are presumably the latest station locations. In some cases stations were moved, although the same station ID was retained. Further details are obtainable in the basic climatological station metadata (e.g. [109]), which should be investigated individually for each weather station to uncover more detail regarding historical site locations.
The three above-mentioned aspects may have a significant impact on the historical tem- perature trends hidden in the raw record. It is possible that the alternative reconstructions presented in this report will provide trends similar to ACORN-SAT, as already indicated in this report. All these reconstructions start from the same raw data and similar trends are therefore not unexpected. Further analysis of the raw record may therefore be an important next step.
11
Acknowledgements
A
Mathematical derivations for online ANN training
This appendix presents mathematical derivations which were necessary during the de- velopment of software to train the networks presented in this report, using online ML algorithms. Only a 2-2-2 ANN with the classical log-sigmoid transfer function as shown in Fig. 60 is considered here. Both forward propagation and backpropagation are consid- ered, after which a numerical example based on [110] is presented.Figure 60: 2-2-2 ANN with initial weights, input and target values of one training example.