Analysis of Data Accuracy

Accuracy is determined on a variable by variable basis for both the data entered at point-of-care (PoC) as well as the data entered retrospective. Accuracy levels are reported for weight, ART regimen, pill count, attend work/school and peripheral neuropathy. In the following subsections, raw accuracy data are displayed in 2x2 tables for each variable.

5.4.2.1 Weight

Of the 624 visits 20 have the patient’s weight left blank on the master card. Consequently these visits will be excluded from analysis. During analysis we noted that a very high level of discordance existed among the weight values recorded in the dataset. Further analysis revealed that of those discordant a high percentage differed only by 0.1 Kg. On inspection of the master cards we noted that weights were generally recorded to two decimal places. We subsequently confirmed that the electronic scale in the clinic does report weights to two decimal places, but the data input screen in the software application is designed to accept only one decimal place. Based on this we realize that the discrepancy of 0.1Kg is most likely accounted for by inconsistent rounding to one decimal place. Consequently, for the purpose of determining concordance, we consider two values to be concordant if they differ by no more than 0.1 Kg.

There are 14 occurrences where a weight is not entered at the point-of-care but is recorded on the master card and captured electronically during the retrospective entry by the nurse. We suspect that the patient was seen by the clinician prior to being weighed and therefore was unable to enter the weight into the system. We recognize that this is a systemic problem and not a data entry error. Consequently we exclude these visits from the analysis of accuracy for weight. To compare accuracy for the weight field we tabulate levels of concordance of both point-of-care data and retrospectively entered data with the master cards below. A total of 587 were considered in the analysis.

Table 20: Comparing point-of-care and retrospective errors for patient weights Point-of-Care data

Weight Concordant* Discordant*

Concordant* 424 64

Retrospective

Data _{Discordant* 65 34}

* Concordant/Discordant referrers to the agreement/disagreement of the field in the point-of-care record and retrospective record with the master card.

5.4.2.2 Outcome Status

We recognize that master cards get updated after the visit to reflect changes in patient outcome of death and transfer out. Five occurrences of visits that are concordant between the point-of-

care and retrospectively entered dataset but discordant with the master card. They comprise three “transfer out”, one death and one default. Additionally, we note very little variability in the data (“Alive” 96% of visits). Based on these observations we exclude outcome status from our analysis.

5.4.2.3 ART Regimen

Of the 624 visits 14 have the regimen field left blank on the master cards. Consequently these visits will be excluded from analysis. To compare accuracy for the ART regimen field, we tabulate levels of concordance of both point-of-care data and retrospectively entered data with the master cards.

Table 21: Comparing point-of-care and retrospective errors for ART regimens Point-of-Care data

ART Regimen Concordant Discordant

Concordant 589 7

Retrospective

Data _{Discordant 12 2}

5.4.2.4 Pill Count

Of the 624 visits 125 have pill counts recorded as 0 from the master cards. Unfortunately we did not undertake to keep track of blank pill count fields on the master card when capturing the data electronically. Consequently a 0 in the pill count column in the study dataset can mean either a pill count of 0, or that the field was left blank on the master card. For the purpose of analysis we exclude all 125 visits with a pill count of 0 in the dataset transcribed from the master card. We also eliminate four additional visits based on the presence of a “-1” in the field placed in there by the ARV system to indicate “Unknown” at the time of data entry.

To compare accuracy for pill count we tabulate levels of concordance of both point-of- care data and retrospectively entered data with the master cards below.

Table 22: Comparing point-of-care and retrospective errors for pill counts Point-of-Care data

Pill Count Concordant Discordant

Concordant 392 47

Retrospective

Data _{Discordant 42 14}

5.4.2.5 Work / School

Of the 624 visits 23 have the ambulatory field left blank on the master cards. Consequently these visits will be excluded from analysis. To compare accuracy for the work/school field we tabulate levels of concordance of both point-of-care data and retrospectively entered data with the master cards below.

Table 23: Comparing point-of-care and retrospective errors for work/school Point-of-Care data

Work / School Concordant Discordant

Concordant 551 16

Retrospective

Data _{Discordant 19 15}

5.4.2.6 Side Effects

Of the 624 visits 17 have the side effects field left blank on the master cards. Consequently these visits will be excluded from analysis. To compare accuracy for PN we tabulate levels of concordance of both point-of-care data and retrospectively entered data with the master cards below.

Table 24: Comparing point-of-care and retrospective errors for peripheral neuropathy Point-of-Care data

PN (Peripheral Neuropathy) Concordant Discordant

Concordant 489 42

Retrospective

Given the low occurrence of side effects other than PN in the data we see no value in reporting on them.

5.4.2.7 ARV Recipient

Of the 624 visits 73 have the “ARV recipient” field left blank on the master cards. An additional 24 master cards indicated ARVs had been dispensed to both the Patient and the Guardian. We believe this data is of extremely low quality, possibly confounded by different interpretations of what the data is supposed to represent. Based on these observations we exclude outcome status from our analysis.

5.4.2.8 Summary of Results for Data Accuracy

We summarize the overall error rates and PoC vs. Ret differences for five variables in Figure 22 below.

Figure 22: Overall error rates and Difference (PoC - Ret)

Left: Overall error rates; Average, PoC (P), Ret (R)

Right: Difference between PoC error rates and Ret error rates with 95% confidence intervals (Statistically significance for Peripheral Neuropathy, P =0.0023)