Comparing the accuracy of and confusion by the models

The results of the accuracy assessment of the three classification models are shown in Tables 4.5, 4.6 and 4.7. All the three models predicted the seven soil types and a non-soil class (water). As shown in Tables 4.5 – 4.7, the rows correspond to soil classes in reference map and the columns correspond to classes in the classification output maps. The diagonal elements in the matrices represent reference soil class that actually obtained the same soil class during classification (Vayssières et al., 2000). The off-diagonal elements represent soil types that ended up in another category during classification. Off-diagonal row elements represent the reference soil types of a certain category which were excluded from that soil type during

128

classification leading to errors of omission (Congalton, 1991). Off-diagonal column elements represent reference pixels of a class wrongly included in another classification class leading to errors of commission (Wade, 1999).

User’s accuracy corresponds to an error of commission and producers accuracy corresponds to an error of omission, they reveal if an error was evenly distributed between classes or if some classes were really bad and some really good (Congalton, 1991). In other words the producer’s accuracy (model sensitivity) indicates a measure of how well the test data are classified and the user’s accuracy (as actual purity) is the measure of how likely a test sample classified into a given category actually belongs to that category (Scull et al., 2005, Kempen et al., 2009).

In terms of producer’s accuracy, the classes which were more accurately predicted by MLR model were ARE, GLY, FLU, and PLI all with producer’s accuracy above 80% (Table 4.5). These soils also had relatively higher user’s accuracy of over 78%. While the soil classes with the lowest producer’s accuracy were ACR, FRA and NIT. These categories also had the lowest user’s accuracy (Table 4.5).

ARE and PLI had the highest producer accuracy for BT model. ACR and NIT had the lowest producer accuracy. FLU, ARE, and GLY had the highest user accuracy. The BT model had ACR and NIT with lowest user accuracy (Table 4.6). ARE, GLY, and PLI had the highest producer’s accuracy for RF model. On the other hand, NIT and PLI had the lowest producer’s accuracy. In terms of user’s accuracy, FLU and GLY had the highest user’s accuracy. ACR and NIT had the lowest user’s accuracy (Table 4.7). All the three models indicate GLY, ARE, and FLU had consistently higher user’s accuracy compared to other categories. ACR and NIT had the lowest

129

user’s accuracy in MLR and BT models. ARE had the highest producer’s accuracy in each of the three models.

The average producer’s accuracy, average user’s accuracy, overall map accuracy as well as Kappa statistics were highest for RF followed by MLR and BT models. The lowest Kappa statistic for the overall error matrix were; 0.642, 0.711, and 0.731 for BT, MLR, and RF models, respectively (Table 4.5, 4.6, and 4.7). The RF model had the highest overall accuracy of 77.1% compared to 75.3% and 69.3% for MLR and BT, respectively. While the RF model had the highest overall classification accuracy, PLI class was better predicted by BT and MLR.

Table 4.5 Error matrix for soil type classes by MLR MLR

ACR FLU NIT ARE GLY FRA NWT PLI

Row totals User accuracy (%) ACR 21 0 0 1 0 10 0 4 36 58.3 FLU 0 25 0 0 3 4 0 0 32 78.1 NIT 1 2 26 0 0 12 0 4 45 57.8 ARE 2 0 0 25 1 0 0 0 28 89.3 GLY 0 0 0 0 44 5 0 0 49 89.8 FRA 6 3 11 2 3 53 0 0 78 67.9 NWT 0 0 0 0 0 0 25 0 25 100.0 PFR 3 0 2 0 1 3 0 34 43 79.1 Column totals 33 30 39 28 52 87 25 42 Producer accuracy (%) 63.6 83.3 66.7 89.3 84.6 60.9 100.0 81.0

Average producer’s accuracy: 78.7, average user’s accuracy: 77.5, sample size: 336, overall accuracy (map purity): 75.3%, Kappa: 0.711

130 Table 4.6 Error matrix for soil type classes by BT

BT

ACR FLU NIT ARE GLY FRA NWT PLI

Row totals User accuracy (%) ACR 19 3 0 0 6 4 0 4 36 52.8 FLU 2 26 0 0 4 0 0 0 32 81.3 NIT 4 1 26 0 3 11 0 0 45 57.8 ARE 1 0 0 22 2 3 0 0 28 78.6 GLY 0 1 1 0 41 6 0 0 49 83.7 FRA 5 4 15 0 4 48 0 2 78 61.5 NWT 0 0 0 5 0 0 20 0 25 80.0 PLI 3 1 2 0 3 3 0 31 43 72.1 Column totals 34 36 44 27 63 75 20 37 Producer accuracy (%) 55.9 72.2 59.1 81.5 65.1 64.0 100.0 83.8

Average producer’s accuracy: 72.7%, average user’s accuracy: 71.0 %, sample size: 336, overall accuracy (correctly classified): 69.3%, Kappa: 0.642

Table 4.7 Error matrix for soil type classes by RF RF

ACR FLU NIT ARE GLY FRA NWT PLI

Row totals User accuracy (%) ACR 27 0 2 1 0 3 0 3 36 75.0 FLU 2 27 0 0 2 1 0 0 32 84.4 NIT 5 1 26 0 0 12 0 1 45 57.8 ARE 3 0 1 22 1 1 0 0 28 78.6 GLY 0 2 0 0 38 8 1 0 49 77.6 FRA 3 3 5 0 2 64 0 1 78 82.1 NWT 0 0 0 0 0 0 25 0 25 100.0 PLI 3 1 3 0 1 5 0 30 43 69.8 Column totals 43 34 37 23 44 94 26 35 Producer accuracy (%) 62.8 79.4 70.3 95.7 86.4 68.1 96.2 85.7

Average producer’s accuracy: 80.6 %, average user’s accuracy: 78.1%, sample size: 336, overall accuracy (map purity): 77.1%, Kappa: 0.731

As stated above, the producer’s and user’s accuracy were dissimilar in the predicted soil classes by the different models (Table 4.5, 4.6, and 4.7). Although ARE, FLU and GLY had consistently high prediction accuracy in all the three models, these soils are generally influenced by

131

drainage. FLU and GLY were reported in other studies to be influenced majorly by topography and slope (Debella-Gilo and Etzelmüller, 2009), as whenever these two attributes are included in the model, the level of class accuracy improves significantly. Globally GLY are largely located in lowlands and wetlands as they are mainly formed in areas saturated with groundwater for long enough to develop a characteristic gleyic colour pattern (FAO-WRB, 2014). On the other, hand FLU are formed in alluvial deposits or lacustrine and marine deposits.

The soil classes which had the highest confusion in all the three models are the NIT and FRA. According to FAO-WRB (2014), these soil classes are both deep, well-drained and red and are both dominated by Kaolinite. Although NIT are considered more fertile than FRA, the high confusion could be attributable to their general occurrence in similar landscape positions. The error could also have resulted from the field soil misclassification as the data was obtained from multiple sources. For instance, while harmonising the soil map of Africa Dewitte et al. (2013) noticed several inconsistencies in the general north-south soil classification and distribution of the soils in Zambia and Malawi which was attributable to possible mistakes that during the soil survey and field classification which relied on subjective expert knowledge.

Although PLI had one of the highest frequency of observations RF and BT models predictions mapped it as one of the least represented classes and ACR which had the lowest observation frequency was mapped among the most dominant soil classes. This indicates that those models predicted the soil classes without much influence of the observation frequency. Moran and Bui (2002) observed that the best models are achieved by sampling in proportion to the spatial

132

extent of mapped classes. RF and BT models predicted dominant soil classes irrespective of the number of point observation and were, therefore, the best prediction models.