3.5 Model Assessment
3.6.1 Simulated Data
Initially we simulated data sets using daily, 2-day, 3-day intervals and so on up to an interval of one week. We used the values of p0, q0 and φ that obtained from the p-MRSA Data Set without taking into account the antimicrobial treat-ment (the results for this data set are presented later in this chapter). So, we set p0 = 0.8859, q0 = 0.0800 and φ = 0.8568. The summary statistics for the daily, 3-day and 7-day intervals are shown in Table 3.2. It can be seen that the simulated values obtained for these three data sets are very close to the values set. Moreover we can see that as day intervals increase, the posterior correla-tion between p0 and q0 also increases. Figures 3.1 and 3.2 show the posterior density estimates from the MCMC output for p0and q0, and φ respectively. The negative predictive value for the three data sets was ψ = 0.8586 for the daily interval, ψ =0.8395 for the 3-day interval and ψ=0.7925 for the 7-day interval transitions.
Table 3.1 shows the equal-tailed 95% quantiles from the model fit simulations.
In the same table there are also the observed counts for each day interval and transition. We can see that the model fits very well for every day interval, as all the observed counts lie in the equal-tailed 95% quantiles.
3.6.2 Using the day intervals from the p-MRSA Data Set
To verify that our method works using irregular day-intervals, we simulated data using the day-intervals from the p-MRSA Data Set. To simulate the data we set p0 =0.8859, q0 =0.0800 and φ =0.8568. We used these values because
Day Intervals
Table 3.1:Model fit for the model using simulated data with 1-day, 3-day and 7-day interval transitions. The intervals in red color show that the equal-tailed 95% quantiles include the number of the observed tran-sition counts.
1-day Interval
parameters E[ · |Y](s.d.) 95% CI p0 0.8887(0.0065) (0.8760, 0.9014) q0 0.0835(0.0063) (0.0712, 0.0958) p0, q0posterior correlation −0.2906
φ 0.8580(0.0083) (0.8411, 0.8746) 3-day Interval
parameters E[ · |Y](s.d.) 95% CI p0 0.8826(0.0090) (0.8640, 0.8998) q0 0.0828(0.0112) (0.0624, 0.1065) p0, q0posterior correlation −0.5104
φ 0.8425(0.0271) (0.7904, 0.8968) 7-day Interval
parameters E[ · |Y](s.d.) 95% CI p0 0.8561(0.0231) (0.8037, 0.8923) q0 0.0938(0.0235) (0.0539, 0.1437) p0, q0posterior correlation −0.9549
φ 0.8050(0.0532) (0.7083, 0.9125)
Table 3.2:Summary statistics and posterior correlation for p0, q0 and φ using the simulated transitions for intervals of 1-day, 3-days and 7-days without antimicrobial treatment. The true values are p0 = 0.8859, q0 =0.0800 and φ=0.8568.
these are the parameter estimations for the p-MRSA Data Set excluding the antimicrobial treatment information.
Simulating data using the day-interval structure from the p-MRSA Data Set means that we will keep the same number of patients and number of transi-tions for each patient as in the p-MRSA Data Set. A problem that arises when simulating a data set of this kind (i.e. when including only patients with at
0.86 0.87 0.88 0.89 0.90 0.91 0.92
0204060
Kernel Density Estimate for p0 −daily intervals
p0
Density
0.06 0.07 0.08 0.09 0.10 0.11
0204060
Kernel Density Estimate for q0 −daily intervals
q0
Density
0.84 0.86 0.88 0.90 0.92
010203040
Kernel Density Estimate for p0 −3 day intervals
p0
Density
0.04 0.06 0.08 0.10 0.12
0102030
Kernel Density Estimate for q0 −3 day intervals
q0
Density
0.75 0.80 0.85 0.90 0.95
0246810
Kernel Density Estimate for p0 −7 day intervals
p0
Density
0.05 0.10 0.15 0.20
0246810
Kernel Density Estimate for q0 −7 day intervals
q0
Density
Figure 3.1:Kernel density plots for parameters p0 and q0 from the simulated data and for simulated 1-day, 3-day and 7-day transition intervals without antimicrobial treatment. The true values are p0 = 0.8859 and q0=0.0800.
0.82 0.84 0.86 0.88
01030
Kernel Density Estimate for φ −daily intervals
φ
Density
0.75 0.80 0.85 0.90 0.95
051015
Kernel Density Estimate for φ −3 intervals
φ
Density
0.6 0.7 0.8 0.9 1.0
0246
Kernel Density Estimate for φ −7 intervals
φ
Density
Figure 3.2:Kernel density plots for sensitivity φ from the simulated data and for simulated 1-day, 3-day and 7-day transition intervals without antimicrobial treatment. The true value is φ=0.8568.
least one positive test) is that the output will almost always also include pa-tients with only negative tests. Here we will investigate two cases of simulated data. The first generates a data set that has the same structure as the p-MRSA Data Set. This means that it includes patients with no positive tests. The second case uses only the patients of the first case that have at least one positive test.
We will show that in both cases the model fits equally well.
Case 1: Simulated data set with the p-MRSA day-interval structure
The simulated data set we are using here has the same structure as the p-MRSA data set as shown in Section 2.2. The summary statistics of the posterior density are shown in Table 3.3. It can be seen that the parameter estimates are very close to the values we set to construct the simulated data set. Figures 3.3 and 3.4 show the marginal posterior density estimates from the MCMC output for p0and q0 and φ respectively. The posterior mean of the negative predictive value ψ was found equal to 0.8472. The correlation between p0 and q0 is shown in Figure 3.5. We notice that parameters p0and q0are strongly correlated.
no antimicrobial treatment
parameters E[ · |Y](s.d.) 95% CI p0 0.8645(0.0167) (0.8275, 0.8935) q0 0.0930(0.0003) (0.0590, 0.1323) p0, q0posterior correlation −0.8009
φ 0.8596(0.0018) (0.7735, 0.9444)
Table 3.3:Summary statistics for p0, q0and φ for the simulated transitions us-ing the day intervals from the p-MRSA Data Set without antimicro-bial treatment. The true values are p0 = 0.8859, q0 = 0.0800 and φ=0.8568.
Finally, The results of the equal-tailed 95% quantiles as well as the transition counts from the observed data are shown in Table 3.4. We can see that the model fits very well.
0.80 0.85 0.90 0.95
0510152025
Kernel Density Estimate for p0
p0
Density
0.00 0.05 0.10 0.15 0.20
05101520
Kernel Density Estimate for q0
q0
Density
Figure 3.3:Kernel density plots for p0and q0for the simulated transitions us-ing the day intervals from the p-MRSA Data Set without antimicro-bial treatment. The true values are p0 =0.8859 and q0=0.0800.
0.7 0.8 0.9 1.0
02468
Kernel Density Estimate for φ
φ
Density
Figure 3.4:Kernel density plot for sensitivity φ for the simulated transitions using the day intervals from the p-MRSA Data Set without antimi-crobial treatment. The true value is φ=0.8568.
Day Intervals
1 2 3 4 5 6 7
obs. counts 19 23 19 19 21 43 174
N→ N (17, 26) (18, 28) (13, 22) (13, 24) (14, 24) (29, 45) (157, 191)
obs. counts 9 10 8 11 11 22 143
N→C (2, 11) (5, 15) (4, 14) (6, 16) (8, 18) (20, 36) (125, 158)
obs. counts 2 8 7 11 8 17 140
C→ N (0, 6) (3, 12) (4, 12) (6, 16) (7, 17) (11, 24) (119, 153)
obs. counts 12 20 17 18 21 25 175
C→C (8, 13) (16, 25) (12, 20) (13, 23) (12, 23) (18, 31) (161, 196) Table 3.4:Model fit for the model using simulated data but with the day
inter-vals from the p-MRSA Data Set. The interinter-vals in red color show that the equal-tailed 95% quantiles include the number of the observed transition counts.
0.3 0.4 0.5 0.6 0.7 0.8
0.10.20.30.40.5
p0
q0
Figure 3.5:Scatterplot showing the correlation between p0and q0for the simu-lated transitions using the day intervals from the p-MRSA Data Set without antimicrobial treatment.
Case 2: Simulated data set having the p-MRSA Data Set day-interval structure but including only the patients with at least one positive test of Case 1.
In this section we use the same data set as in Case 1 but include only the patients with at least one positive test. So for this dataset we had to remove 79 patients who had only negative tests.
Table 3.5 shows the summary statistics of the posterior densities of the param-eters. It can be seen that the parameter estimates are different to the values we set to construct the simulated data set. This is because in this case we have excluded some patients. Figures 3.6 and 3.7 show the marginal posterior den-sity estimates from the MCMC output for p0and q0and φ respectively. Finally, Figure 3.8 shows the correlation between p0and q0. The posterior mean of the negative predictive value was found equal to φ =0.8272.
Table 3.6 shows the 95% quantiles of the model fit. It can be seen that the model fit is quite satisfactory.
no antimicrobial treatment
parameters E[ · |Y](s.d.) 95% CI p0 0.6527(0.0876) (0.4434, 0.7818) q0 0.2569(0.0043) (0.1577, 0.4139) p0, q0posterior correlation −0.8900
φ 0.9789(0.0190) (0.9291, 0.9993)
Table 3.5:Summary statistics for p0, q0and φ for the simulated transitions us-ing the day intervals from the p-MRSA Data Set without antimicro-bial treatment and including patients with at least one positive test.
0.3 0.4 0.5 0.6 0.7 0.8
012345
Kernel Density Estimate for p0
p0
Density
0.1 0.2 0.3 0.4 0.5 0.6
01234567
Kernel Density Estimate for q0
q0
Density
Figure 3.6:Kernel density plots for p0and q0for the simulated transitions us-ing the day intervals from the p-MRSA Data Set without antimi-crobial treatment and including patients with at least one positive test.
0.88 0.90 0.92 0.94 0.96 0.98 1.00
0102030
Kernel Density Estimate for φ
φ
Density
Figure 3.7:Kernel density plot for sensitivity φ for the simulated transitions using the day intervals from the p-MRSA Data Set without antimi-crobial treatment and including patients with at least one positive test.
0.2 0.3 0.4 0.5 0.6 0.7 0.8
0.10.20.30.40.50.6
p0 q0
Figure 3.8:Scatterplot showing the correlation between p0and q0for the sim-ulated transitions using the day intervals from the p-MRSA Data Set without antimicrobial treatment and including patients with at least one positive test.
Day Intervals
1 2 3 4 5 6 7
obs. counts 9 10 9 7 8 25 130
N→ N (8, 16) (6, 15) (4, 12) (4, 12) (4, 13) (14, 27) (102, 135)
obs. counts 9 10 8 11 11 22 143
N→C (2, 11) (5, 14) (5, 13) (6, 14) (6, 14) (20, 33) (137, 171)
obs. counts 2 8 7 11 8 17 140
C→ N (1, 7) (5, 16) (5, 14) (7, 18) (7, 18) (13, 24) (120, 154)
obs. counts 12 20 17 18 21 25 175
C→C (7, 13) (12, 22) (10, 19) (11, 22) (11, 22) (18, 29) (160, 195) Table 3.6:Model fit for the model using simulated data but with the day
inter-vals from the p-MRSA Data Set and including patients with at least one positive test. The intervals in red color show that the equal-tailed 95% quantiles include the number of the observed transition counts.
3.7 Results using GSTT Data
In this section we fit the hidden Markov model to the GSTT data. Initially, we use only the p-MRSA Data Set and fit the model under various assumptions.
Next, we obtain the parameter estimates for the p-Wounds and p-Respiratory Data Sets.
3.7.1 p-MRSA Data Set
Firstly, we will ignore the information about the antimicrobial treatment and find only the baseline probabilities p0 and q0 and sensitivity φ. Secondly, we will consider all antimicrobial treatment as one group both for the whole p-MRSA Data Set and starting from first positive test. We make the latter assump-tion because in this chapter we have ignored that there is patient-to-patient transmission in the ICU ward. Lastly, we will consider each antimicrobial treat-ment separately.
We also perform model assessment under all the assumptions listed above. We will see that the model fit is not always adequate.
Results assuming no antimicrobial treatment
Here we present the results of fitting the model without taking into account the information about antimicrobial treatment. From the MCMC output, we derived the posterior mean of probability p0 of patients who remain in a non-colonised state equals to 0.8859, while the probability of MRSA clearance, q0, is 0.0800. Moreover, the posterior mean of the swab test sensitivity, φ equals to 0.8568. Swab test sensitivity ranges between 66.7%−87% according to the body site the swab is taken, [Hope et al., 2004; Keene et al., 2005]. The posterior mean of the negative predictive value ψ is 0.8453. Table 3.7 shows the summary statistics of the posterior distributions of p0, q0 and φ as well as the posterior correlation between p0 and q0. It can be seen that parameters p0 and q0 are highly correlated, possibly because here we consider only patients with at least one positive test and there might be some dependencies in the data.
Figure 3.9 presents the density plots for p0 and q0 and Figure 3.10 shows the
density plot for sensitivity φ.
no antimicrobial treatment
parameters E[ · |Y](s.d.) 95% CI p0 0.8859(0.0139) (0.8549, 0.9100) q0 0.0800(0.0140) (0.0559, 0.1114) p0, q0posterior correlation −0.6821
φ 0.8568(0.0326) (0.7931, 0.9218)
Table 3.7:Summary statistics for p0, q0and φ for the p-MRSA Data Set without antimicrobial treatment.
0.82 0.86 0.90
051015202530
Kernel Density Estimate for p0
p0
Density
0.04 0.08 0.12 0.16
051015202530
Kernel Density Estimate for q0
q0
Density
Figure 3.9:Kernel density plots for p0 and q0 from the p-MRSA Data Set as-suming no antimicrobial treatment.
Table 3.8 shows the results of the 95% quantiles as well as the transition counts from the observed data. We can see that the model fit is not adequate as some of the observed transition counts are not in the 95% intervals.
0.75 0.80 0.85 0.90 0.95
024681012
Kernel Density Estimate for φ
φ
Density
Figure 3.10:Kernel density plot for sensitivity φ from the p-MRSA Data Set assuming no antimicrobial treatment.
Day Intervals
1 2 3 4 5 6 7
obs. counts 17 19 19 8 17 31 243
N →N (8, 19) (13, 25) (12, 24) (13, 25) (14, 28) (24, 41) (180, 234)
obs. counts 10 11 15 18 14 26 132
N→C ( 6, 14) ( 10, 19) (13, 23) (14, 24) ( 12, 22) (25, 39) (173, 207)
obs. counts 8 11 8 12 10 26 72
C→ N (9, 18) (17, 29) (6, 15) (12, 22) ( 13, 25) (26, 40) (172, 203)
obs. counts 7 20 9 21 20 24 185
C →C (2, 9) (1, 8 ) ( 1, 8) (1, 9) (1, 8) (3, 14) (31, 62) Table 3.8:Model fit for the model of the p-MRSA Data Set without
antimicro-bial treatment. The intervals in red color show that the equal-tailed 95% quantiles include the number of the observed transition counts.