Validating EHR Reports

1

Cardiac Learning & Action Network:

Improving Cardiac Population Health through Primary Care Redesign

Validating EHR Reports

October 2012

Introduction

The use of electronic health records (EHRs) has greatly improved the ability of medical practices to

access clinical quality data through registries and clinical reports. Although these functionalities can seem straightforward in some EHR systems, a number of steps are necessary in order to make sure that practice staff can trust the data in the reports.

Various team members have important roles in the reporting process:

• A clinical person should confirm that all staff follow a protocol for putting clinical information related to the report into the correct fields of the EHR.

• A quality improvement person should confirm that the data included in the numerator and denominator reflect the information generated in the report.

• A database/software person should confirm that the report is pulling data from the correct fields in the EHR and that there are no potential software errors.

After these steps have been taken, the validation process assures a practice that the report is accurate (or at least contains minimal error). There are many reasons that reports can generate incomplete or inaccurate information; report validation identifies errors so that the practice can correct any mistakes.

Types of Reporting Errors

There are two primary types of errors to look for when validating your report:

1. Inclusion errors: Patients are in your report but should not be

Patients may be erroneously included in your report for a number of reasons, including outdated problem lists or providers using ICD codes that are not part of the practice protocol. Please see the “Finding Inclusion Errors” worksheet, page 4, for assistance in documenting your efforts to resolve these types of errors.

2. Exclusion errors: Patients are not in your report but should be

Patients may be erroneously excluded from your report due to reasons such as rounding of vital signs that inappropriately exclude patients from the numerator or denominator, or patients being prescribed medications that are not pulled in the report. Please see the “Finding Exclusion Errors”

worksheet, page 5, for an outline of the steps required to uncover these types of errors.

Methods for Finding Errors

Chart reviews and complementary reports, outlined below, are two useful methods for finding

potential inclusion and exclusion errors. The worksheets provided on pages 4 and 5 can help your team systematically review each round of reports.

It may take several iterations of the report before everyone is satisfied that the data are accurate and complete. Staff may want to include a review of clinic workflows to prevent future data-entry errors.

Chart Review

Although chart review is time-consuming, it is worth the effort to ensure that the information in the report is trustworthy. To make the best use of your time, review a sample of records rather than the entire data set included in the report.

The first step in a chart review is to determine how many samples you must take (i.e., how many charts you must review) to get an accurate representation of your report’s entire data set.

One shorthand method is to select the inverse of the desired error rate. For example, to find at least one instance of an error that occurs in 5% of the charts, you need to review 20

charts that have been randomly selected. Although there is no magic number of charts to review, most experts find reviewing 30 charts to be a good target that will find errors that occur in 3–4% percent of records in a report and at the same time does not represent an impossible task.

There are far more scientific ways to determine sample size; however, this basic method will provide a place to start. For further information on more robust techniques, please contact your Cardiac LAN consultant.

1 We arrive upon the sample size of 20 charts by selecting the inverse of the desired error rate (which in the example is 5%, so 1/5% = 20). If you

want to find an error that occurs in 3% of the charts, you will review 1/3%, or 33 charts. If you want to find an error that occurs in 1% of the charts,

you need to review 1/1% or 100 charts.

3

Complementary Reports

An additional way to validate that a report is accurate is to run a complementary report that looks for the data in an alternative way.

Validating a chronic disease diagnosis:

1. Reporting from claims data for the past two years

• If the patient has a chronic disease diagnosis coded for two or more encounters, and this diagnosis does not appear on the EHR problem list, have the PCP review the chart for accuracy.

2. Reporting on common procedures

• Patients with two or more A1c tests without a diagnosis of diabetes • Patients with elevated LDLs without a diagnosis of hyperlipidemia 3. Reporting from the medication list

• Patient on cholesterol-lowering medications without a diagnosis of hyperlipidemia • Patients on antipsychotics without a mental health diagnosis

• Patients on bronchodilator inhalers and steroid inhalers without a diagnosis of asthma

Validating vaccination reports:

1. Write a report from claims data for vaccination CPT codes and compare to the data in the initial EHR report.

2. Compare results in your vaccination reports to purchasing orders to see if they are in close proximity.

Conclusion

Generating reports from your EHR is an important part of quality improvement. Validating those reports, using the suggestions outlined here, will help ensure that your quality improvement strategies are based on accurate information.

www.QualisHealthMedicare.org/cardiac

This document is based, in part, on the whitepaper authored by Jeff Hummel, MD MPH for Deep Domain entitled “Get More from your EHR: An

expert guide to EHR Reporting” and is used here with permission. No part of this document may be reused without the prior written permission of

Deep Domain, Inc. (www.deepdomain.com). The content was adapted by Qualis Health, the Medicare Quality Improvement Organization for Idaho

and Washington, under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human

Services. The contents presented do not necessarily reflect CMS policy. ID/WA-C9-QH-759R-10-12

Finding Inclusion Er ror s Char t Review The purpose of these steps is to find patients who are in your repor t but should not be. From the records included in your repor t, systematically select patient c har ts until reac hing the agreed upon sample siz e. F ully review eac h c har t before mo ving to the next one. Patient ID Does patient belong in the repor t’s denominator?

Does patient belong in the repor t’s numerator?

Compare the data in the repor t to relevant data in the c har t. For example, confirm that the patient w as diagnosed with IVD and had the LDL result indicated in the repor t. List all discrepancies, then discuss with the clinical team and the database analyst/repor t writer . www .QualisHealthMedicare.org/cardiac 4

R epor t Name _______________________________ R un Date ___________________________________ R eviewer(s) _________________________________

5

Finding Exclusion Errors

The purpose of these steps is to find patients who are not in your report but should be.

Step 1: Chart Review

Report Name _______________________________

Run Date ___________________________________

Reviewer(s) _________________________________

 Steps Notes

1. Locate all of the patients on the report that belong to the panel of (or are attributed to) the provider who is participating in the validation process.

2. Open the validating provider’s clinic schedule in the most recent months for which data should be in the report. Working backwards one day at a time, select the patients who would be expected to be included in the report based on the provider’s knowledge of their medical condition.

3. Verify that each patient found on the schedule who should be in the report actually is in the report, and that the data in the report match the data in the electronic record.

4. For those patients who are missing from the report (but should be included) review the chart to determine if the criteria for inclusion in the a) denominator, and b) numerator are met in the chart.

5. List all discrepancies, then discuss with the clinical team and database

analyst/report writer.

Step 2: Complementary Reports

Complete the following steps for each complementary report.

 ^{Steps Notes}

1. Write a new report that seeks the same information in a different way.

2. Validate the report with a chart review.

3. Identify any patients who are included in the complementary report but are missing from the initial report. Determine whether they actually belong in the initial report.

4. List all discrepancies, then discuss with the clinical team and database

analyst/report writer.

www.QualisHealthMedicare.org/cardiac