FMD seropositivity model validation results

Comparison between model predictions and data

There were 58.9% seropositive results amongst the 2694 livestock. When the significant fixed effects (age, species and livestock practice) were used to predict seropositive results, 65.5% were predicted positive (Figure 4.9). The fixed effects prediction equalled the true result 68.2% of the time and the Kappa statistic for agreement between the prediction and reality was 0.326 (fair agreement). When random effects (village and herd) were added to the significant fixed effects, 66.9% of the livestock were predicted to be seropositive. The model prediction equalled the true result 77.6% of the time and the Kappa statistic was 0.52 (moderate agreement).

Figure 4.9: A comparison between the laboratory data and the model inference for each animal.

NSP positive or negative (x-axis) and probability of being NSP positive (y axis).

When 2694 data-points were compared to model predictions, 602 (22.3%) of the model predictions did not agree with the data (Figure 4.9). There was a greater proportion of small ruminants compared to cattle amongst the 197 positive data points that were predicted to be negative. Pastoralist livestock made up the greatest proportion of the 403 negative data points predicted to be positive followed by agropastoral livestock, with smallholder livestock making up the smallest proportion (Table 4.6). The kappa statistic for agreement between the model prediction and the cross-sectional serology result was 0.53 (moderate agreement). When the model’s predictions were based only on fixed effects (i.e. without herd and tribe level random effects), the prediction was the same as the data 68.6% of the time and the kappa statistic (0.34) reflected fair agreement.

0.00 0.25 0.50 0.75 1.00

Negativ e

Positiv e

Result from data

Model inferred probability of positive

Comparison between model inferences for NSP positive livestock and the data

Table 4.6: Data-points where model inference did not match the NSP result from the data.

NSP = Foot-and-mouth disease non-structural protein antibody ELISA.

Number (Percentage of total wrong positive or negative predictions) Predicted negative but really positive

Predicted positive but really negative

Total wrong predictions out of 2694 198 404

Herds with one or more livestock with wrong predictions 51 73

Cattle 50 (25.3%) 225 (55.56%)

Small ruminants 148 (74.7%) 179 (44.2%)

Agropastoral livestock 49 (24.7%) 132 (32.59%)

Pastoral livestock 78 (39.4%) 222 (54.81%)

Rural smallholder livestock 71 (35.9%) 50 (12.35%)

Outliers amongst the random effects

Three villages in the rural smallholder area were identified as having more seropositive animals than would be expected given the fixed effects in the model (Figures 4.10 and 4.11). The two herds with the highest positive effect on the intercept were from two of the outlier villages (Figure 4.10).

Figure 4.10: A map of the smallholder herds around Arusha National Park highlighting the three outliers amongst the household level random effects.

These two herds had more NSP positive animals than was expected given their explanatory variables in the final model.

NSP = Foot-and-mouth disease non-structural protein antibody ELISA.

Figure 4.11: A plot of the distribution of household related random effects.

Household level random effects from the final model are on the y-axis with bars indicating their 95% confidence intervals.

36.4 36.6 36.8 37.0 37.2 NkoanenkouKenyamontaIseresereEsilaletiKomoloKikwe Miririnyi NarakauwoSoitsambuRingwaniNambalaMbuguniSakalaOljoroTerrat Miseke Sukuro Maaloni EngareseroMasanguraNatambisoNyamburiMgonoSale Maruango NyamsingisiSmaniangoMagaiduruKimotorakEmboreetPinyinyiOltukaiKono Mswakini juuOlkungwadoKarangaiKolila

(Intercept)

−2 0 2

Random effects

Village

The villages defined as outliers amongst the random effects were Kolila, Olkungwado and Karangai in the Arusha smallholder area (Figure 4.10).

In Kolila village, the one household had a high proportion of seropositive animals, which would not be expected based on the final model, but the other household had lower levels of seropositivity. Possible reasons for the difference between the two households were investigated (Table 4.7). The only difference detected was that the outlier household was the only Maasai household amongst the rural smallholders. As Maasai people are conventionally pastoralists, it is possible that this outlier household had some different management practices that were not picked up upon through the questionnaire.

Table 4.7: A comparison between management factors and seropositivity in the Kolila rural-smallholder outlier household compared to the other household in Kolila.

Outlier Non-outlier

Tribe Maasai Mmeru

Number of cattle 21 20

Number of young cattle (<=2.5 years) 9 6

Number of small ruminants 27 22

Time walked to reach grazing and water 10 minutes 3 hours

Number of acquired animals in past 4 months 0 14

Buffalo sighted weekly or more frequently No Yes

Non buffalo FMD susceptible wildlife sighted weekly

or more frequently No Yes

Proportion of young cattle positive 77.7% 0%

Proportion of total cattle positive 92.3% 47.2%

Distance to buffalo area (Km) 7.16 5.05

Proportion of total livestock positive 92% 47%

Distance to buffalo area (Km) 7.16 5.05

In Olkungwado village, only one herd, the outlier that had unexpectedly high levels of seropositivity, was sampled. This household was based less than 500 m away from the boundary of Arusha NP. In contrast to the other households in Arusha, it was sampled in May 2012 rather than in late 2011.

In Karangai, the third outlier village, the two herds sampled had seroprevalences of 70%

and 83% respectively, which is higher than expected for the rural smallholder area.

Power analysis for the cross-sectional survey

The results of the retrospective power calculation for the cross-sectional study are summarised on Figure 4.12. For 2688 livestock from 84 herds and 42 villages, when buffalo sightings had no effect, Wald p values were less than 0.05 for 6.1% of simulations.

When buffalo sightings increased the probability of livestock in the herd being seropositive by 0.2, Wald p values were less than 0.05 for 85.8% of simulations. When the probability increased by 0.25, p values were less than 0.05 for 96.1% of simulations. This power analysis suggests that the sample size of the cross-sectional study was acceptable.

Figure 4.12: Results of power analysis by simulation for detecting an effect of buffalo sightings on the probability of being seropositive

(with baseline probability seropositive = 0.5).