1
Importance of Post-Implementation Analysis in EHR Implementation
Public Comment of
Gary C. David, PhD Associate Professor of Sociology
Department of Sociology gdavid@bentley.edu ehealthrecords.wordpress.com Bentley University Waltham, MA June 26, 2009
2 David Blumenthal, M.D., M.P.P.
National Coordinator
Office of the National coordinator for Health Information Technology U.S. Department of Health and Human Services
200 Independence Avenue Washington, D.C. 20201
Dr. Blumenthal and Members of the HIT Policy Committee,
I am Gary C. David, PhD, an Associate Professor of Sociology at Bentley University (Waltham, MA). For the past two years, I have been conducting an intensive study of how medical records are created. This research began with examining the work of medical transcriptionists1,2 (MTs) and the medical transcription industry. The project has expanded to include technologies that are employed in this process, such as speech recognition technology3 (SRT) and now electronic health records (EHRs) and electronic medical records (EMRs). Along with speaking with MTs and others in this industry, I have had the opportunity to speak with HIM directors, hospital senior administrators, physicians, and other healthcare providers regarding the creation of medical records, as well as the impact of technologies on this process. I also have had the opportunity to review the materials4 associated with the committees work around meaningful use, and wanted to take the opportunity to share my thoughts based on my research findings to date.
Impact of EHRs on Medical Record Content
Upon reviewing the materials that are publically available from the HIT Policy Committee, along with other government and industry sources, there are some concerns that I have regarding the impact that the broad introduction of new medical record production technologies will have on the content of medical records. It can be expected that once the mode of creation is changed (e.g., from dictation-transcription to template-based EHR direct entry), the content of the medical record also will change. At this point, it is too early to make determinations of whether this change will constitute „better‟ or worse‟. It can be safely assumed, and is borne out by my own (as well as other research), that change will occur. Given the
1 Gary C. David (2007) 2007 Survey of Medical Transcriptionists – Preliminary Findings.
http://www.ahdionline.org/scriptcontent/downloads/MTSurveyReport-Preliminary.pdf
2 Juliann Schaeffer (2008) Getting in Tune – New Survey Spotlights the MT‟s Role in Healthcare. For the Record,
20(15), p.14. http://www.fortherecordmag.com/archives/ftr_011909p6.shtml.
3 Gary C. David, Angela Cora Garcia, Donald Chand and Anne Warfield Rawls (2009) Listening to What is Said –
Transcribing What is Heard: The Impact of SRT on the Practice of Medical Transcription. Sociology of Health and Illness. In press.
4Since the minutes and transcript from the June 16, 2009 meeting have just been made available, and I have not had
3
importance of the medical record, this change must be understood and needs to be part of the conversation around the implementation of EHR systems.
The medical record is not only important to patient care; medical records are vital to all aspects of health care. This point will be evident to members of the committee, but bears restatement because of its importance. Those whose work relies on the medical record is large and increasing. Beyond the doctor, this includes: other doctors and healthcare providers, billing personnel (such as coders), legal personnel, researchers, insurance and reimbursement agents, auditors, and patients. Each party has their own reason for using the medical record, and as a result different (and at times divergent) needs exist in terms of content. The more that medical records become accessible, the greater the potential harm that could result from errors in those records.
On June 14, 2009, the House Energy and Commerce Subcommittee on Oversight and Investigation held a hearing on the act of post-claims underwriting and rescission practiced by insurance companies. Part of the testimony touched on how errors in medical records can have drastic repercussions on patient coverage and care. The testimony of Robin Beaton, a 59 year-old nurse, illustrated this point:
In May 2008, I went to the dermatologist for acne. A word was written on my chart and interpreted incorrectly as meaning pre-cancerous. Shortly thereafter, I was diagnosed with Invasive HER-2 Genetic Breast Cancer, a very aggressive form of breast cancer….The Friday before the Monday I was scheduled to have my double mastectomy, Blue Cross red flagged my chart due to the dermatologist report. The dermatologist called Blue Cross directly to report that I only had acne and please not hold up my coming surgery. Blue cross (sic) called me to inform me that they were launching a 5 year medical investigation into my medical History (sic) and that this would take approximately 3 months5.
The point of this story is not raise the issue of rescission, but rather to illustrate how errors in medical records can have dramatic impacts on patient care in often indirect ways. The impact of medical records is far reaching. Anything that impacts their creation has to be carefully studied and understood.
While the concept of „quality‟ was a constant theme in the documents related to meaningful use, the word „accuracy‟ is not prominent. Notable exceptions include p.7 of the May 11, 2009 meeting minutes, which refers to a statement by Dr. Neil Calman, “Dr. Calman raised the issue of patient access to information. He pointed to examples of inaccurate information being transferred, noting that this is a significant concern.” Another example is from p.22 of the May 11, 2009 meeting transcript, where Scott White is quoted as saying “The (sic) early on, we learned about computers, it was garbage in, garbage out, unless we have the workforce trained properly, understanding their role on the bigger picture, we will fail from the beginning.”
5
House Committee on Energy and Commerce, Subcommittee on Oversight and Investigations, Testimony of Robin Beaton, Hearing on Terminations of Individual Health Policies by Insurance Companies, 111th Cong. (June 16, 2009).
4
Quality and accuracy can be different things. Quality is a more subjective assessment based on the needs of the person using the record. So „quality‟ really is about quality for what purpose. Accuracy is a more objective assessment in that does the record provide a reliable and accurate account of what occurred in the medical encounter. In my own research, I have found numerous instances where a document might be „quality‟ in terms usability, but not accurate (as well as other examples of records being low quality but accurate). The impact of EHRs on both quality and accuracy needs to be considered carefully.
Impact of Sharing Records with Patients
A stated objective in the meaningful use criteria is that documentation will be readily shared with patients. This is a noteworthy goal. The Association for Healthcare Documentation Integrity (AHDI), the professional association for medical transcriptionists, has a campaign called “Have you read your medical record?” This campaign has attempted to create awareness among patients regarding what is in their medical records, which would have the impact of finding errors and correcting them before harm could occur. It is not surprising to find that even though people have the legal right to their medical record, the vast majority never see them.
One thing that has not been considered is what the impact of this sharing will be on the doctor-patient relationship. In research interviews with medical transcriptionists, I have been told repeatedly of instances where doctors make negative assessments (both medical and personal) about their patients. Some of these might appear relatively benign, such as referring to a patient as “fat” or “obese.” Others are more derogatory. One story spoke of a doctor characterizing his patient as having “frequent unprotected sex,” while leaving out the additional information that this was with her husband. MTs have told me of going to doctors and asking them if they wanted to reconsider what they said in their dictation. They often have reconsidered their statements.
Dr. Neil Calman‟s personal blog (February 16, 2008) related a story of a patient who panicked after receiving a letter from his office which stated that her mammogram results were “negative”. Since the word “negative” is often associated with bad news, she interpreted the letter to be indicative of illness. Dr. Calman‟s blog entry then provides an introspective examination of how doctors word their clinical documentation, and how confusing this could be to patients. One could envision a situation where doctor‟s offices are inundated with calls from concerned or upset patients regarding what is in (and what patients think is in) their medical records.
I have spoken with physicians who do not believe that their notes should be shared with patients. The reason why is easy to understand. Doctors and other healthcare personnel have to make frank assessments of their patient‟s physical and mental well-being. The documentation becomes the thing that is repeatedly referred to in order to provide some continuity of care. I frequently ask physicians, “If you knew that your patients would in every instance see your notes, would it change what you write?” The answer is typically, “Yes.” Referring back to the issue of quality versus accuracy, a record may accurately depict what occurred in the exam, but is not „quality‟ in terms of the impact it has on patient
5
satisfaction and the doctor-patient relationship (as in Dr. Calman‟s example). As I tell my students, when you write your papers or give a talk, you have to consider your audience. For doctors, this point is greatly complicated by the fact that there are multiple audiences that make use of their clinical documentation. When patients are included into this mix, they must be considered.
I have come across medical practices that have begun to run parallel documentation systems: one for patients and one for themselves. If a patient wishes to have a medical record, one is provided for them. However, there is another record that exists for more institutional purposes, such as billing and medical care. I asked one doctor if patients routinely see their records, how will he communicate sensitive information to other physicians? He responded that he will likely use the phone. It is then ironic that a system that is meant to make communication between physicians more expedient may increase the reliance on phone calls.
There has been some growing concern regarding the impact of computers on the doctor-patient exam. Some research exists on this point6. Everyone seems to have a story about a computer freezing or a healthcare provider having difficulty accessing screens. Another potential issue is how the technology begins to structure the nature of the interactions. When asking physicians what determines what questions they ask and when, they will list a variety of factors that go into making this judgment. When using systems that are not flexible in terms of question order, healthcare providers may follow the prompts rather than their judgment. It is not clear what, if any, impact this will have. But, it is a change in the ways which exams are done, information collected, and records created.
Need for Post-Implementation Analysis and Assessment
Whenever a new technology is introduced into a workplace, its impacts and use can be highly varied. This is especially the case when an implementation is done at the enterprise level. It then becomes important to understand that use. As has been noted in the work of the HIT Policy committee, and other related materials, the presence of a technology does not equate with use. A letter from NCVHS7 (May 18, 2009) to Dr. Blumenthal makes the point, “The impact (positive or negative) of the EHR on the user‟s workflow can be as important as the EHR functionality” (p.3). This impact can be seen in how the technology is used on a daily basis. Re-engineering attempts in other industries have shown how such projects can be highly disruptive to work. The difference is that the margin of error is healthcare is extremely low. Any disruption to medical work can have drastic repercussions.
6
David Greatbatch, C. Heath, P. Campion and P. Luff (1995) How do desk-top computers affect the doctor-patient interaction? Family Practice, 12(1):32-36.
7Letter from Harry L. Reynolds, Jr., Chairman, National Committee on Vital and Health Statistics, to Dr. David
6
This creates the need for what I refer to as a post-implementation analysis and assessment. The idea of this is to carefully examine how technology is used after it has been implemented. Current recommendations of the committee revolve around what functionalities are used. These metrics can be generated from the system itself, and are in essence a proxy of use (rather than a direct indicator of how the technology is used). While information is in the system, it does not speak to how it got into the system. So, if a system is being used 85% of the time by physicians, it is not an indication of what they are doing with it. Actual use remains a “black box” for healthcare administrators.
In one instance, I was speaking to hospital administrator who was overseeing the implementation of a new EHR system in which physicians would no longer be able to dictate but directly enter information into the system. I inquired what would be a metric of success, and was told that as long as the system “worked” and was being used, it would be a success. I asked further whether there would be any examination of documentation before and after the implementation to gauge whether there would any impacts, and was told this would not be done. Given how important medical records are to all aspects of healthcare, this is troubling. We have no way of knowing whether use was „meaningful‟ without examining all aspects of outcomes, and in the end ultimately examining actual use.
On p.14 of the May 11, 2009 HIT Policy Committee meeting transcript, Dr. Calman references the need for a post-implementation analysis and assessment:
So I think that there needs to be an extension there which would mean that certification might need to include something like some post-implementation assessment of the sites where products were implemented to make sure that they‟re meaningfully used.
This echoes an article that I co-wrote and which appeared in the HIT publication For the Record8. In this article, we state:
We advocate a reorientation of perceptions about what is involved in an enterprise system implementation, one that deviates from an exclusive focus on design and implementation and instead concentrates on iterative cycles of design/implementation/analysis/assessment/recommendation/and (re)design. Based on users‟ lived experiences, there are often opportunities in the post-implementation period to modify (or even customize) the configuration to realize more benefits.
By examining technology-in-use, there are opportunities to increasing the success of the implementation, which in this instance must be measured by the quality and accuracy of the clinical documentation. These include the following:
Workforce Training: As has been pointed out in the committee‟s materials, workforce training is a critical success factor in implementing EHRs. Training before implementation is at best an abstraction of intended use, and not a reflection of
8
Susan Newell and Gary C. David (2009) Rethinking EHR Project Timelines. For the Record. 21(2), p.16. http://www.fortherecordmag.com/archives/ftr_011909p6.shtml.
7
actual use. Being able to capture best (and worst) practices that are located within the workplace will augment workforce training through real-life examples.
Documentation Audit: The quality of medical records can be best understood in their various contexts of use. It is not possible to know all the instances that they will be used. However, it is possible to compare what the documentation looks like before and after implementation. Healthcare organizations should undergo a process of post-implementation clinical documentation audit to begin to determine the potential impact of the implementation.
Analyzing Multiple Modes of Entry: In and of itself, the implementation of an EHR system does not determine how information gets into the system. However, EHR solutions often are sold as a way to cut costs through the elimination of medical transcription. This is illustrated by the Congressional Budget Office paper9 “Evidence on the Costs and Benefits of Health Information Technology”, which states that EHRs can “Eliminate the use of medical transcription and allow a physician to enter notes about a patient‟s condition and care directly into a computerized record” (p.1). While this might decrease the turn-around time of creating documentation, it does not necessarily increase the quality or accuracy of the records. It also does not necessarily make physicians more efficient in terms of seeing patients. Research needs to be devoted to comparing the various modes of entry, including MT-based EHR entry, the use of front-end SRT systems, and template-based entry, in order to weigh efficiency gains across the creation of documentation and actual delivery of treatment.
System (Re)Design: Companies that are responsible for developing EHR systems have a difficult task in trying to envision use across a variety of specializations and healthcare settings. Physician complaints often center on the extent to which the system does not integrate into their particular needs. While there is a need to standardize, and to some extent re-engineer, how documentation is created, there also is a need for a system to be flexible enough to handle all the contingencies that physicians face. While it was previously thought that physician narratives would no longer be a needed element of the record, recent thinking demonstrates the need for the flexibility that narrative afford10. Understanding how systems are being used can provide an opportunity to (re)design systems to fit how work is done.
9 Congressional Budget Office (2008) Evidence on the Costs and Benefits of Health Information Technology. May. 10 Stephen B. Johnson, Suzanne Bakken, Daniel Dine, Sookyung Hyun, Eneida Mendonça, Frances Morrison,
Tiffani Bright, Tielman van Vleck, Jesse Wrenn, and Peter Stetson (2008) An Electronic Health Record Based on Structured Narrative. Journal of the American Medical Informatics Association, 15(1): 54-64.
8
Create user buy-in: One principal challenge in implementing a new system is getting user buy-in. When users feel that their input and concerns are being listened to, it can aide in getting their buy-in. Observing technology-in-use provides that opportunity for users.
Innovation and Organizational Change: Learning and innovation, as well as organizational change, ultimately comes down to communication among users. There are numerous examples of innovations existing within an organization, but not being leveraged due to the lack of dissemination of lessons learned. By conducting a post-implementation analysis, organizations can create a more fertile ground for distributed innovation which will ultimately assist in generating organizational change.
I am currently working with a small team at Bentley University, along with Erik Vinkhuyzen, Ph.D. of the Palo Alto Research Center (PARC), to study the use of EHRs in the medical setting. The goal of our research is to carry out post-implementation analyses and assessments of EHR implementations in order to begin answering the questions being raised in this document. Another goal of the project is to create a scalable post-implementation toolkit that can be implemented by healthcare organizations in order to empower them to answer these questions for themselves. Currently, there appears to be hesitancy among healthcare organizations to address these questions. Part of this is likely due to a lack of awareness of these issues, along with a lack of resources to do so. However, despite these barriers, the need to conduct such an examination remains.
I applaud the work of the HIT Policy Committee for their diligence in tacking on the concept of meaningful use. Doing so in a way that is simple yet complete is an onerous task. I thank the committee for the opportunity to provide these comments, and do so in the hope that the content will be useful in your work. I hope that the information in this document is useful to the committee. If there is any clarification or follow up needed, I would welcome the opportunity. Thank you, and I look forward to following the committee‟s work on the topic.
Sincerely,
Gary C. David, PhD
Associate Professor of Sociology Department of Sociology
