To take advantage of the increasingly available observational electronic medical record data to support post-marketing drug effectiveness and drug safety surveillance, it is important to validate the outcomes that will be used in such studies. In particular, hospitalization diagnoses are important markers of adverse event severity. To be able to identify adverse events within a specified exposure time window, it is also important to able to ascertain the precision of
hospitalization dates. Validation of The Health Improvement Network (THIN) adverse events and hospitalization dates in this study can thus inform many important future observational THIN studies of the outcomes of medication use, for antibacterial drugs as well as for other types of medications. The hypothesis is that hospitalization diagnoses and dates will be valid within clinically important limits.
After U.S. FDA approval, drugs are used for many more patients, for a wider variety of indications, and for a more heterogeneous patient population than in pre-approval trials. The importance of post-marketing drug safely surveillance is increasingly recognized. Observational studies can take advantage of accumulating electronic medical record data to enhance post- marketing outcome studies. Electronic data come in two basic flavors, with some overlap. Because they are used for billing, administrative data tend to have information on drugs
dispensed, and relatively precise hospital admission dates and discharge diagnoses. Electronic medical records on the other hand tend to have rich longitudinal clinical data at the individual patient level, however inpatient data regarding hospitalizations often are not directly linked to the outpatient record and instead need to be entered manually.
The projects in this dissertation used data from The Health Improvement Network (THIN). THIN is an electronic medical record database containing longitudinal primary care data from patients in the United Kingdom including demographics, visits, diagnoses, prescription
medications, laboratory testing, mortality, cause of death, and hospital admissions. However, as indicated above, the software used for THIN was developed as an ambulatory medical record
system; inpatient and ambulatory medical records are not yet integrated in the U.K., and hospitalization data are entered manually by patients’ general practitioners after they review patients’ hospital discharge summaries.
There are three main areas of uncertainties to be addressed if THIN hospitalization data are to be useful for drug outcome research. First, when THIN hospital admission codes indicate a patient was hospitalized, did the patient truly have an admission to a hospital, or what is the positive predictive value of the THIN hospitalization coded for identifying a hospitalization? Second, if the patient was indeed hospitalized, is the primary discharge diagnosis recorded in THIN the true primary discharge diagnosis from the hospitalization? Third, what is the relationship between the recorded hospital admission date and the true hospital admission date? If the hospitalization is recorded after receipt of the discharge summary, it may be recorded with a later date than the true admission date. Even if the true hospitalization date falls within the exposure window of interest, if the recorded date erroneously falls outside the exposure window of interest, the hospitalization event might be missed.
The objective of this study is to validate hospitalizations in the THIN database. The specific aims were to:
1. Assess the positive predictive value (PPV) of a hospital admission for community acquired pneumonia identified using THIN hospitalization codes. Our hypothesis was that the PPV of a pneumonia hospitalization is 100%.
2. Assess the relationship between THIN hospital admission date and true hospital admission date. Our hypothesis was that 100% of THIN hospitalizations will be recorded as occurring within a 14-day window of the true hospitalization date.
Methods
This study design was a retrospective cohort study. Adults with ambulatory primary care visits for acute nonspecific respiratory tract infections in the THIN database from June, 1985
through August, 2006 were identified using Read diagnostic codes. (Table 1) These visits are hereafter referred to as our cohort of ARI visits.
Study Outcome
Adults with overnight hospital admissions for community acquired pneumonia within 30 days of an ambulatory encounter for acute respiratory tract infection in the THIN database were identified using Read diagnostic codes for pneumonia and THIN hospitalization codes. Of these adults with hospital admissions for pneumonia, sixty were randomly selected for validation. Validation of 60 hospital admissions for pneumonia was feasible given available resources, and would give us the power to test both of our hypotheses within clinically significant limits (see below).
We focused on hospitalizations for community acquired pneumonia for this aim for several reasons. First, hospital admission for community acquired pneumonia is a relatively common event following ARI, giving us a robust sample of reasonably similar outcomes to be able to correctly estimate the measurement error. Additionally, we had a separate clinical research interest in whether risk of hospitalization for community acquired pneumonia is elevated after an ARI visit, and whether antibiotic treatment reduces this risk. We pursued this related question in Chapter 7 of this dissertation.
Gold Standard Outcome
Each subject’s de-identified THIN patient and practice identification codes, and a date window including 90 days before and following the ARI visit were forwarded to the subject’s general practitioner (GP) through EPIC Database Research Company. The GPs identified records from their patients’ charts that are supplementary to the electronic THIN data. For the specified patients, GPs returned de-identified photocopies of all hospitalization discharge summaries, consultants’ letters, and any additional material related to any overnight hospitalizations within the specified date window.
EPIC checked the data to ensure complete anonymization prior to forwarding them to investigators. We then examined patient records for the following information:
a) Did the subject have an overnight hospitalization within the window? b) What was the hospital admission date?
c) What were the primary and additional discharge diagnoses?
This study was approved by the University of Pennsylvania Institutional Review Board and the Medical Research Ethics Committee, National Research Ethics Service of the U.K. National Health Service.
Analysis
For our first aim, the PPV of a THIN hospital admission for community acquired pneumonia during the 30 days following the ARI visit was calculated as the number of patients with GP-validated hospital admissions divided by the total number of THIN hospitalizations, with exact binomial confidence intervals. The PPV of the specific pneumonia THIN adverse event
hospitalization diagnosis was calculated as the number of patients with confirmed GP-validated
pneumonia diagnoses divided by the total number of confirmed hospitalizations, with exact 95%
binomial confidence intervals. Two-sided two-sample t-tests were used to compare means of characteristics between admitted and nonadmitted patients, assuming unequal variances between groups.
For our second aim, analysis included data only for those patients with confirmed overnight hospital admissions. The mean and median absolute difference in dates between the THIN and the actual hospital admission date were defined. Stata, versions 9.2 and 10.0, were used for all analyses (StataCorp College Station TX, 29-Jan 2007 and 1 Oct 2009).
Power
PPV of a hospital admission for community acquired pneumonia
We included 60 patients in this validation study. Lewis and colleagues, using the General Practice Research Database, a precursor to THIN, found the PPV for identifying inflammatory bowel disease hospitalizations was only near 50%.[1] With a 2-sided α=0.05, and N=60, examples of confidence intervals predicted for a widely representative range of PPVs are shown in Table 2.
Table 2. Positive Predictive Value (PPV) of a THIN-coded Hospital Admission for Community Acquired Pneumonia after a Ambulatory THIN ARI Visit