Medical Liability and Patient Safety Planning Grant
Agency for Health Research and Quality Grant Number 1R21HS019535‐01Background
This evaluation planning was designed to analyze pre‐ and post‐implementation data for key indicators associated with the performance of safe harbor legislation. Evaluation methods were designed to provide an understanding of how policy impacts the core concepts of medical liability system costs, clinical practice guideline (CPG) use, patient safety, and health system costs. We chose evaluation methods based on (1) areas where safe harbor would have the most impact, (2) opportunities for continuous monitoring and analysis of data so policymakers can revisit and readjust the policy as needed over time, (3) balancing opportunities for quantitative and qualitative measurement.
In preparation of our evaluation matrix,1 we consulted the Milliman Report,2 considered to be the definitive evaluation of two Maine tort reform measures, including the Maine Safe Harbor Demonstration Project, throughout the 1990s. We used the Report as a preliminary guide for how we would look at data gathered to measure outcomes of safe harbor legislation in Oregon. Because the Milliman Report’s task was solely to examine the impact of tort reform on medical liability costs and claim frequency before and after tort reforms, we decided to also include others’ ideas for determining the feasibility of safe harbor.
This matrix can guide stakeholders and policymakers as they weigh the options of implementing safe harbor in Oregon. Policymakers should consider the systematic impact of safe harbor legislative reform on the health care system overall, not just on the cost of medical liability claims and premiums.
Limitations
Plans to evaluate the outcomes of safe harbor legislation should take into consideration the possible limitations associated with measuring key indicators: • Reporting requirements for some of existing databases have changed over time. Such changes may make it difficult to effectively measure changes over time. • Most existing medical liability claim databases only record information from individually insured clinicians and do not capture data from the self‐insured market. • This evaluation plan did not assess the reliability or validity of data collected by existing databases. More work would be needed to ensure that the data being collected is consistently reported and maintained.
• Some data that would be needed to evaluate key indicators does not currently exist. In these cases, the ability to measure changes over time that are associated with safe harbor legislation would be limited.
• The ability for evaluation measurements to establish a direct causal or correlated relationship between outcomes and safe harbor legislation may be limited. The ability to
measure specific relationships between legislation and outcomes will depend, in part, on the safe harbor guidelines implemented, and the ability of data sources to collect information specific to the issues addressed by those guidelines. The most reliable methods for drawing direct connections related to the impact of safe harbor legislation are likely to be the most costly.
Medical Liability System Costs: Direct and Indirect
Medical liability system costs can be measured by analyzing data relative to the key indicators discussed by Mello, et al.3: administrative expenses, indemnity payments, claim frequency and time to resolution. In conversations surrounding tort reform, these costs are often the first mentioned, especially when they soar, as they periodically do.3
For many of these key indicators, we have data available in active state databases. In other cases, we would need to implement changes to existing data collection and reporting. Establishing the additional data collection processes necessary can be done relatively easily and inexpensively.
Administrative expenses: Direct
Administrative expenses are described as legal and attorney fees for plaintiff and defense, as well as overhead costs for malpractice insurers.3 Some administrative expense data is captured by organizations like the Oregon Medical Board (OMB) or the Oregon Consumer & Business Services Insurance Division (OCBS), but there are gaps in recorded/reported data, such as the cost of plaintiff legal fees. Monitoring administrative expenses is essential when examining the impact of safe harbor legislation on the costs of medical liability reform. Plaintiff legal fees: database review
Most often, plaintiff legal fees are paid by the defendant if ruling is in favor of the plaintiff. Plaintiff legal fees are usually lumped into the total indemnity payment amount when the information is reported to the OMB. While the actual amount of the plaintiff’s legal fees is not well known, estimates seem to be reliable.3 One widely accepted estimate sets plaintiff attorney fees at around 35% of the indemnity payment to the plaintiff.4 At the moment, however, this data is not captured by OMB. A new line added to the OMB claim reporting form could capture this information. A before/after comparison would be necessary to measure any change, therefore, the reporting requirement would need to be added before the enactment of safe harbor legislation. Defense attorney fees: database review The amount of defense legal fees is currently collected by the OMB in the form of the variable “Loss Adjustment Expense Paid to Defense.” Monitoring defense attorney fees in addition to
plaintiff’s attorney fees would be worthwhile both before and after implementation to gauge whether safe harbor legislation has an impact on this administrative cost. Current estimates of defense costs are $0.19 per each indemnity dollar paid.3 Insurance Premiums: report review Medical liability reform does not always have an impact in the medical malpractice premiums. Safe harbor has also not been shown to have an effect on premiums.2,5 While Mello, et al.3 consider it inaccurate to count premiums in the overall cost of medical liability since premiums amount to what the malpractice insurance industry predicts their indemnity and administrative payouts to be, the trend of rate increases or decreases could help evaluate the impact of safe harbor. Oregon’s Department of Consumer & Business Services’ Division of Insurance keeps a record of medical malpractice insurance company premium trends and publishes the report.6
Indemnity payments: Direct
Indemnity payment types: database reviewIndemnity payments represent a significant portion of medical liability costs and can vary widely between cases, depending on a number of factors that are too varied to predict. Medical malpractice insurers report indemnity payments to the OMB, though the types of indemnity (economic, non‐economic, and punitive) are not often specified. The total indemnity amounts are often reported in the form of the variable “unspecified indemnity”. Differentiation among the types of indemnity payments is often required only in states that have caps on certain types of indemnity payments and Oregon does not have such caps in place. Oregon could require insurers to report this information to OMB, but more work would be needed to determine the feasibility of requiring this level of detail in reporting.
Time to resolution: Indirect
Claim duration: database reviewThe indirect cost of lengthy trials and claims resolution can negatively impact clinician productivity, emotional stability and professional reputation, as well as the direct cost of paying for legal services.3 One possible outcome of a safe harbor medical liability reform would possibly be shorter time for claims resolution. We can easily, and with relatively low cost, measure this by examining the data already collected by the OMB that includes the date of alleged injury and the date each claim is closed. A drawback to this simple measurement is that it will not capture the exact dates of the active claim if a statute of limitation is reached, or it may not capture important nuances of dates such as the difference between the first opportunity to make a diagnosis and the date when the missed diagnosis is identified. A before and after comparison could highlight whether safe harbor had an impact on the length of a claim, but without a more detailed case‐by‐case review, this may not be the best way to measure time to resolution.
Claim disposition: database review
Most malpractice claims are settled out of court.3,4 Comparing the ratio of paid claims that settle after trial or out‐of‐court may be a helpful measurement. The OMB currently collects this information in their standard claim database as the variable “Claim Disposition.” This variable may be the best indicator of time to resolution as cases settled out‐of‐court usually have shorter duration and safe harbor liability reform may increase the number of cases settled out‐ of‐court, thereby shorting the overall time to resolution.
Expert witness involvement: closed claim file review
The need for expert testimony may continue to be necessary post‐implementation of safe harbor legislation; however, there is some possibility that expert testimony will not be necessary if an appropriate clinical practice guideline is cited by the defense.5 Tracking the use of expert witnesses may provide pre‐ and post‐measurement of safe harbor’s impact on the length or outcome of malpractice claims. This information can be gathered from closed claim files.
Frequency of malpractice claims: Indirect
Claim frequency: database review A major goal of any medical liability reform is to reduce the number of malpractice claims. The Milliman Report states that the Maine Safe Harbor Demonstration Project was likely responsible for a reduction in the number of malpractice claims for the specialties covered under the safe harbor rule.2 The possibility exists that safe harbor’s initial implementation may address a limited number of specialties or specific delivery locations (outpatient/ambulatory clinics and surgery centers, for example), which is information that the OMB does not currently collect.7 A question about specialty or delivery site could be added to the OMB claim reporting form post‐implementation. Additionally, closed claim file review of the relevant specialties and delivery sites may provide necessary pre‐implementation data.Clinical Practice Guidelines
The evaluation of clinical practice guidelines is a vital component of assessing the impact of legislation. Although this core concept is more closely related to process than outcomes, understanding the extent to which guidelines are adopted and used is essential to the success that can be expected from safe harbor legislation.
We have noted that in other states’ experience with safe harbor, guidelines were used by expert witnesses often, but rarely as safe harbor in an affirmative defense.2 We have also
learned from another state’s experience8 that adopting a safe harbor rule requires decisions regarding how best to certify guidelines according to the design of safe harbor legislation. We expect many guidelines to become obsolete over time as new science is modifies best practices. The state must plan to regularly evaluate guidelines to assure clinicians who participate in safe harbor that they have access to the newest and best practices. This continuous evaluation, though essential, will likely require significant resources.
Measure the use of CPG & safe harbor in legal proceedings
CPG used in clinician defense: database review Measuring how often the defense cites guidelines in trial or during out‐of‐court settlement is essential. This data is not currently collected. Data could be reported to OMB by the defense and could be collected at the time of case closure when insurance companies are required to submit a report to OMB. Safe harbor as affirmative defense: database reviewClosed medical liability claim review of medical malpractice insurance company cases would determine whether safe harbor is used by a defendant as an affirmative defense.9 Maine’s Safe Harbor demonstration project used an analysis of medical liability claims to determine that there may have only been one physician who used safe harbor as an affirmative defense.10 Although this information is not currently collected in Oregon, an affirmative defense is initially offered in response to the filing of a claim and the OMB could collect this data when insurance companies submit their initial claim filing report.5 Plaintiff use of CPGs: database review Recording plaintiff use of guidelines is useful for establishing whether the standard of care was not met, whether the clinician defendant used an incorrect guideline or the guideline used was inappropriate for various reasons. The defendant’s closed claim files may not capture the plaintiff use of guidelines. There is no apparent existing mechanism for collecting information from plaintiff attorneys. It may be appropriate to require the plaintiff to report this to the OMB or another state agency associated with medical liability reform. If this method is chosen, a database or similar mechanism for reporting this information would have to be created.
Adopted CPG evaluation: document review
Toward the end of Maine’s Demonstration Project, Maine’s Bureau of Insurance reported that clinicians were not using safe harbor as a defense because they were warned of the obsolescence of many of the guidelines.8 Knowing this from the start, Oregon could assure clinicians that guidelines will undergo regular review for correctness. This continuous review of adopted guidelines, however, would be expensive as such a responsibility may potentially be borne by individual hospitals/clinics, municipalities (county governments).
Patient Safety
While the costs associated with medical malpractice drive efforts to reform the medical liability system, patient safety is a primary focus of reform initiatives.11,12 Patient safety should reveal to observers how medical liability reforms are doing. Indirectly, an environment where liability reform has been implemented should see an improvement in patient safety.Patient Safety Indicators
Adverse outcome reporting: database reviewIn a study published in 2010 by the RAND Institute for Civil Justice,13 investigators found a strong correlation between improved patient safety and reduced malpractice claims in counties throughout the state. While the article advocates for patient safety initiatives as a possible alternative to traditional malpractice reform, it also provides a framework for monitoring general patient safety indicators that do not necessarily have direct association with malpractice claims.
Health System Costs
Defensive medicine costs are generally considered to be the tally of procedures and tests that clinicians order for patients for the sole purpose of protecting themselves against potential malpractice claims.3 The cost of defensive medicine is considered by Mello, et al.3 to be an indirect medical liability system cost, often because it is very difficult to quantify its true cost. Mello and Kachalia 5 theorize that clinicians who participate in safe harbor will rely less on ordering extra tests and will instead stick to adopted guidelines.
The prevailing method of determining the cost of defensive medicine, proposed by Kessler and McClellan14 is considered by some to be flawed.15 In addition, clinician surveys may not be able to capture a clinician’s subconscious use of defensive medicine, or account for bias or anecdotal evidence of defensive medicine.16
Costs of defensive medicine and/or overutilization
Gauge clinicians’ use of defensive medicine: survey One way to measure defensive medicine costs may be to conduct a survey to gauge clinician reliance on defensive medicine before and after safe harbor rule implementation. The costs associated with surveys are high, and the responses could show bias and/or may not capture clinicians’ subconscious decisions to practice defensively.16 These limitations should be a
consideration when deciding to move forward. Surveys could be created from scratch or modeled after other clinician survey studies from recent years, such as Massachusetts Medical Society’s 2008 survey of physicians and their use of defensive medicine.17
Quantify defensive medicine costs: database review
Quantifying defensive medicine costs may prove difficult. There are disagreements in the literature about the best way to capture these costs. The prevailing approach, is to compare the costs of treatment for a carefully chosen set of ICD codes at discharge, both before and after enacting liability reform measures. This method, outlined by Kessler & McClellan,14 could be very simple, given the new All Payer All Claims (APAC) database that will collect this information from Oregon insurance carriers, including private and public insurers. Reporting for this database is mandatory for insurers on all paid claims. A potential drawback may be the question of Kessler & McClellan’s experiment design as some scholars have not been able to reproduce their findings.15 Another potential drawback is that the APAC database captures most patient visits, but leaves out self‐pay, charity care and other types of visits that do not result in an insurer paying, or being billed, for the visit.
Table 1. Matrix of Evaluation Core Concepts, Key Indicators, Methods & Sources Core
concepts IndicatorsKey Methods & Sources
Cost $ – $$$ Does necessary data exist? Advantages Limitations Medical liability system costs Administrative expenses Database review Oregon Medical Board (OMB) database. Variable: plaintiff legal fees. $ No. Consider adding to existing OMB questionnaire. A relevant, inexpensive and easy data point to gather. May be difficult for insurance company to accurately assess and report. Database review OMB database. Variable: Loss Adjustment Expense Paid to Defense $ Yes. Data relevant, inexpensive and easy to gather. None. Report review Oregon Consumer & Business Services (CBS), Insurance Division annual [name of report]. Variable: medical malpractice premium costs $ Yes. Report is very accessible and easy to monitor. Malpractice premiums may have little to do with litigation epidemics and more to do with market investments. Also, medical liability reform may have little impact on malpractice premiums. Indemnity payments Database review OMB database. Variables: punitive, economic, non‐economic and unspecified indemnity payments $ Partially. Data required, but not always detailed. If collected, the data would help understand whole indemnity payment picture This may be impossible to capture given the way payments are determined in court vs. settlement in Oregon (states with specific indemnity caps keep track of this information.). Claim frequency Closed claim/database review Pre‐implementation data: review of insurance company closed claim files Post‐implementation data: OMB database Variables: count of claims reported, $$/$ Pre‐implementation data: No. Post‐implementation data: Yes. Data will show whether targeted areas see a decline in claims.
Core
concepts IndicatorsKey Methods & Sources
Cost $ – $$$ Does necessary data exist? Advantages Limitations Time to resolution Database review OMB database. Variables: date of injury, closure date. $ Yes. Would offer exact information about length of case Concerns exist about reliability of “date of injury.” Some companies may report this as the initial claim file date, court filing date, or date of alleged injury. Database review OMB database. Examine ratio of claims going to trial and settling out of court. Variable: claim disposition $ Yes. Reliable estimate of case length, easy and inexpensive to gather. Reliance on the assumption that settlement cases are shorter than court cases. Closed claim file review Review of insurance company closed claim files to monitor the role and number of expert witnesses used by defense and plaintiff. $$ Partially. Insurance company files may not fully document plaintiff expert witnesses. May help to understand lengthy cases Closed claim file review is costly. The use of expert witnesses may be difficult to gather for both sides. Clinical practice guidelines (CPG) adherence, use and monitoring Use of safe harbor/clinical practice guidelines in legal proceedings Database review Defense to report CPG use when reporting claim closure New variable: defense documentation of CPG use in litigation $ No. Consider adding to existing OMB questionnaire. Essential to monitor guideline use in medical malpractice claims. None. Database review OMB database. New variable: used safe harbor as an affirmative defense. $ No. Data reported when defense initially responds to claim – easy to gather from existing OMB questionnaire/database. None.
Core
concepts IndicatorsKey Methods & Sources
Cost $ – $$$ Does necessary data exist? Advantages Limitations Database review Plaintiff‐supplied data: were CPGs used to show that the clinician did not adhere to guideline or used an inappropriate guideline. $$ No. Consider adding to existing OMB questionnaire. New requirement for OMB database claim reporting. Would be helpful to gauge impact of safe harbor medical liability reform effort CPG use in clinical practice Document review Hospital/clinic endorsement and/or policies regarding the use of safe harbor CPGs $ No. Inexpensive, reliable source Roll out of CPG adoption may make this difficult. May also be difficult for collected data to target areas relevant to adopted CPGs. Chart review Patient medical chart review to identify clinician use of CPGs. $$ No. Consider amending Electronic Health Record (EHR) to include acknowledgement of CPG use by clinician. Depending on the EHR implementation, the data could be easy to capture and reliable, as the clinician would report use during or soon after treating a patient. Could be difficult to integrate changes to EHR in existing clinical practice. CPG analysis Document review Periodic review of adopted CPGs to assess the relevancy of current guidelines in use. Guideline updates would follow. $$$ No. Up‐to‐date guidelines must be available to clinicians at the time service is provided. Is essential to success of safe harbor feasibility. Review of CPGs will be costly and time consuming, and may need to be done locally or by individual treatment centers. Patient safety Number and types of adverse medical outcomes Database review Oregon Patient Safety Commission (OPSC) database. Variable: adverse report information. Other options (AHRQ, Patient Safety Net) may be $ Yes.. OPSC database may be the best option of adverse reporting in Oregon. The voluntariness and possible under‐ reporting may not capture a true picture of patient safety. Also, reported information is limited to hospitals,
Core
concepts IndicatorsKey Methods & Sources
Cost $ – $$$ Does necessary data exist? Advantages Limitations better. excluding clinics and independent practitioners. Database review University Hospital Consortium (UHC) PSN for single‐hospital. May have an option for mandatory reporting of all adverse events. $‐$$ Partially. Already in use at OHSU.. If instituted as mandatory reporting, could be an excellent example of captured patient safety data for pilot site. May only be feasible in a pilot setting; under‐reporting may abound even with mandatory requirements. Chart review Hospital/clinic patient medical chart review of adverse events that may not fit into OPSC/UHC report forms. $$$ No Excellent and reliable way to capture adverse events without depending on reporting systems. Expensive. Health system costs Costs of defensive medicine and/or overutilization Survey Gauge clinician reliance on defensive medicine $$ No. Survey ahead of implementation. May provide useful before/after measurement of clinician perceptions about defensive medicine Surveys provide unreliable biased data; low response rate expected. Database review All Payer All Claims (APAC) database. Before/after comparison of hospital billing/procedure frequency. $ Yes. Not yet mandatory for all payers. Best method for determining defensive medicine costs. Other researchers who have used this method have not always found reliable results. Others have not been able to replicate Kessler & McClellan’s results.
1 Adapted from Gelmon, SB, Foucek, A & Waterbury, A. (2005). Program Evaluation: Principles and Practices. 2nd ed. Portland: Northwest Health Foundation. Retrieved from http://nwhf.org/resources/ 2 Sanders, RL, Karls, CC. (2000). Maine Bureau of Insurance: Evaluation of Medical Malpractice Tort Reform and Collateral Source Payment Reduction Maine Demonstration Project. Prepared by Milliman & Robertson, Inc. 3 Mello, MM, Chandra, A, Gawande, AA, & Studdert, DM (2010). National Costs of the Medical Liability System. Health Affair 29(9), 1569‐77. 4 Studdert, DM, Mello, MM, Gawande, et al. Claims, Errors, and Compensation Payments in Medical Malpractice Litigation. New Engl J Med. 2006 354(19), 2024‐33. 5 Mello, MM & Kachalia, A. (2010). Evaluation of options for medical malpractice system reform. Washington, DC: Medicare Payment Advisory Commission (MedPAC). 6 Oregon Department of Consumer & Business Services. (2010, April 15). Oregon medical malpractice rates continue to decrease. Retrieved from insurance.oregon.gov/news_releases/2010/041510‐malpracticedecrease.pdf 7 Before 2010, the OMB collected the specialty information on their claim report. This was removed at the beginning of 2011. 8 Matthews, J.R. (1999). Practice Guidelines and Tort Reform: The Legal System Confronts the Technocratic Wish. J Health Polit Polic 24(2), 275‐ 304. p. 300. 9 Affirmative defense is described as “an excuse or justification that protects the defendant from liability to the plaintiff and that relies on factual issues not raised in the complaint.” Edwards, LL and Edwards, JS (2002). Introduction to Paralegal Studies and the Law: A Practical Approach. Albany, NY: Delmar/Thomson Learning. P. 253. 10 Rosoff, RJ. (2001). Evidence‐Based Medicine and the Law: The Courts Confront Clinical Practice Guidelines. J Health Polit Polic. 26(2), 327‐68. P. 343) 11 Clancy, CM. (2010, September/October). Patient Safety and Medical Liability Reform: Putting the Patient First. Retrieved from http://www.ahrq.gov/news/commentaries/comptsafty.htm 12 Clinton, HR & Obama, BH (2006). Making patient safety the centerpiece of medical liability reform. New Engl J Med, 354(21), 2205‐2208. 13 RAND Institute Report. Greenburg, MD, Haviland, AM, Ashwood, JS, & Main, R. 2010. Is Better Patient Safety Associated with Less Malpractice Activity? 14 Kessler, D & McClellan, M. (1996). Do Doctors Practice Defensive Medicine? Q J Econ, 111(2), 353‐390. 15 Sloan, FA & Shadle JH. (2009). Is there empirical evidence for defensive medicine? A reassessment. J Health Econ, 28(2009), 481‐491. 16 Studdert, DM, Mello, MM, Sage, WM., et al. (2005). Defensive Medicine Among High‐Risk Specialist Physicians in a Volatile Malpractice Environment. JAMA‐J Am Med Assoc, 293(21), 2609‐2617. 17 Massachusetts Medical Society. (November 2008). MMS First‐of‐its‐kind Survey of Physicians Shows Extent and Cost of the Practice of Defensive Medicine and its Multiple Effects of Health Care on the State. Retrieved from http://www.massmed.org/AM/Template.cfm?Section=Advocacy_and_Policy&TEMPLATE=/CM/ContentDisplay.cfm&CONTENTID=23559.
