PRACTICAL APPLICATIONS FOR
MANAGERS
Predicting future healthcare
costs: how well does
risk-adjustment work?
Michael A. Cucciare and William O’Donohue
Department of Psychology, University of Nevada, Reno, Nevada, USA
AbstractPurpose– Risk-adjustment is designed to predict healthcare costs to align capitated payments with an individual’s expected healthcare costs. This can have the consequence of reducing overpayments and incentives to under treat or reject high cost individuals. This paper seeks to review recent studies presenting risk-adjustment models.
Design/methodology/approach– This paper presents a brief discussion of two commonly reported statistics used for evaluating the accuracy of risk adjustment models and concludes with recommendations for increasing the predictive accuracy and usefulness of risk-adjustment models in the context of predicting future healthcare costs.
Findings– Over the last decade, many advances in risk-adjustment methodology have been made. There has been a focus on the part of researchers to transition away from including only demographic data in their risk-adjustment models to incorporating patient data that are more predictive of healthcare costs. This transition has resulted in more accurate risk-adjustment models and models that can better identify high cost patients with chronic medical conditions.
Originality/value– The paper shows that the transition has resulted in more accurate risk-adjustment models and models that can better identify high cost patients with chronic medical conditions.
KeywordsHealth services sector, Cost analysis, Risk assessment
Paper typeResearch paper
In 2003, healthcare costs grew at a rate of six times inflation (Armour and Appleby, 2003). This was not an isolated increase but represented a multi-year trend of healthcare prices rising at multiples of inflation over most of the last decade. Important contributors to rising healthcare costs are high utilizers or when a relatively small subset of patients uses a disproportionate amount of healthcare resources (Cucciare and O’Donohue, 2003; Cummingset al., 1997). Berk and Monheit (2001) reported that in 1996, the most expensive 10 percent of healthcare consumers accounted for approximately 70 percent of total healthcare costs. Furthermore, who becomes and remains a high utilizer can vary significantly over time, with different individuals moving in and out of high cost groups from year to year (Ashet al., 2001).
As a result of rising healthcare costs, there is an increasing interest on the part of healthcare payers and providers to develop statistical models that can accurately predict future healthcare expenditures. For example, the Health Care Financing Administration has historically overpaid health maintenance organizations for the care of Medicare beneficiaries and as a result sponsored much research in developing
The current issue and full text archive of this journal is available at www.emeraldinsight.com/1477-7266.htm
JHOM
20,2
150
Journal of Health Organization and Management
Vol. 20 No. 2, 2006 pp. 150-162
qEmerald Group Publishing Limited 1477-7266
risk-adjustment models that can help better match capitated payments to the
healthcare needs of beneficiaries (Ash et al., 2000). Healthcare providers receiving
capitated payments are also interested in risk-adjustment to the extent they are better able to predict enrollees’ future expenditures (Kapuret al., 2000). Briefly, capitated payments are prospective payments that are made to a healthcare provider for each insured individual. This type of payment strategy puts the healthcare provider at risk since the payment for services is made before any care has been received (Moon, 1997). Furthermore, in this type of payment system, the healthcare provider (e.g. hospital or mental health clinic) assumes the risk of potential high costs that may be incurred by the individual in exchange for a stream of predictable revenue for services that may or may not be used. Thus, in a capitated payment system, the more accurately a provider is able to predict future costs the less risk they assume.
In capitated systems of payment, high utilizing (and high cost) individuals are at risk for under treatment and/or disenrollment of healthcare services due to
providers attempting to reduce healthcare costs (Kapur et al., 2000; Young et al.,
1998). However, theoretically, risk-adjustment methodology is designed to remedy these disadvantages by aligning the prospective or capitated payment made to a provider with an individual’s expected healthcare costs, thus reducing incentives to under treat or reject high cost individuals (Kapuret al., 2000). Risk-adjustment is a statistical tool designed to predict healthcare costs incurred by individuals and/or groups (Greenwald, 2000). It is also used by some researchers to identify high costs patients such as those with chronic disease conditions such as diabetes and asthma (Ash et al., 2000). Generally, patient variables such as age, gender, and diagnosis gathered from a base year and analyzed using some form of multiple regression (see Howell, 1995 for a discussion of multiple regression) to predict year two healthcare costs, however, these variables can also be used to predict costs in the same year (these are known as concurrent or retrospective as opposed to prospective models) (Greenwald, 2000). Some of the most common variables used to predict costs include: age, gender, prior healthcare and pharmaceutical costs, functional status (e.g. difficulties with activities of daily living or ADLs) and diagnoses (Ash et al., 2000; Kapur et al., 2000; Pope et al., 2000).
Over the last five years, there have been several articles published that present a variety of approaches to risk-adjustment (Ash et al., 2000, 2001; Carteret al., 2000; Kapuret al., 2000; Popeet al., 2000). This paper will review these studies with a special emphasis on elucidating the strengths and limitations of these extant risk-adjustment models. This paper will begin with a brief discussion of two common statistics used for evaluating the performance of risk-models. We will then conclude with recommendations for increasing the predictive accuracy and usefulness of risk-adjustment models in the context of predicting future healthcare costs.
Commonly reported statistics for evaluating the performance of risk-adjustment models
The most common statistics used for examining a risk-adjustment model’s ability to account for variance in expected costs and accurately predict future costs are the
r-squared (or the proportion of variance in future costs explained by the model) and the predictive ratio (PR) (Ashet al., 2000 for a discussion of PR; Carteret al., 2000; Kapur
et al., 2000). It is beyond the scope of this paper to provide a thorough discussion of
Predicting future
healthcare costs
these statistical procedures, however, given that these statistics are commonly used to examine risk models’ performance, a brief discussion is warranted.
R-squared
In the behavioral sciences, the r2statistic is used as a measure of variability, with values ranging from 0 to 1 (0 indicates no correlation and 1 indicates a perfect correlation). The purpose of ther2statistic is to describe the extent to which two (or more) variables correlate with one another and therefore how much of the variation in one variable is explained (or accounted for) by the other. For example, if the correlation between healthcare utilization and age were 0.30, we could say that 0.302¼9 percent of the variability in healthcare utilization is directly explained by the variability in age (see Howell, 1995 for a more thorough discussion of ther2statistic). In the context of risk-adjustment,r2represents the percentage of total variance in medical expenditures that is predicted (or accounted for) by the risk-adjustment model (Popeet al., 2000).
Predictive ratios
PRs are used to examine the accuracy of a risk-adjustment model. The PR is designed to compare the predicted costs produced by a given risk-adjustment model with average actual costs of a target patient subgroup. The PR is calculated by dividing predicted costs by actual costs (Ashet al., 2000). For example, if model A predicts that a subgroup of individuals will cost $200 each per year and their actual cost is $200, the PR is 1. In this case, model A predicts healthcare costs perfectly (when compared to actual average costs of the target patient group). In contrast, if model B predicts the cost to be $200 and the average actual cost is $150, the PR is 1.33. Using the same example, if the average actual cost for patients in the subgroup is $250, the PR is 0.8. Both of these analyses demonstrate the model B is inaccurate, with the former overpredicting and the latter underpredicting costs for the target patient population.
Risk-adjustment models: a brief review of the literature
A primary goal of risk-adjustment is to align an individual’s future expected healthcare costs with the prospective payment made to those financially responsible for healthcare services. Historically, researchers have used variables such as age and gender from a base year (e.g. year one) to estimate or predict individuals’ healthcare costs for the following year (or year two), however, increasingly, some form of diagnostic data (i.e. patient diagnosis) is being incorporated into models to predict future healthcare costs. The following is a brief review of the risk-adjustment literature, which is divided into two sections:
(1) prospective; and
(2) combined (i.e. prospective and retrospective) risk-adjustment models.
The former models include information (e.g. diagnoses that occurred in a base year) that occurred before the beginning of the year for which one is interested in predicting costs (e.g. year two). Retrospective models include health information (e.g. the occurrence of a birth) that occurred during the year for which one is interested in predicting costs (Carteret al., 2000).
JHOM
20,2
Prospective models
Kapur et al.(2000) examined the ability of five risk-adjustment models to estimate
future healthcare costs for a group of high cost mental health service utilizers. The following is a listing of the variables included in each of the five models. Model one included only demographic information – age, gender, ethnicity, education level, marital status, type of residence and living arrangements, and primary language; model two included the previously mentioned demographics, insurance indicators (i.e. whether or not the individual was covered by Medicaid, Medicare, or a private organization in year one), and homelessness; model three included model two variables plus indicators for diagnoses, and global assessment of functioning scores; model four included model three variables plus indicators for inpatient and outpatient costs for the previous year; and model five included model four variables plus inpatient and outpatient costs for the two previous years. The results of the analyses showed that models one through three each predicted less than 5 percent of the variation in costs, while models four and five predicted 14 and 16 percent of the variation in future costs, respectively.
Pope et al. (2000) examined the explanatory power of the Principal Inpatient Diagnostic Cost Group Model (PIPDCG) using data from a Medicare sample
(N¼approx. 1.3 million individuals). The PIPDCG in this study incorporated data
such as age, sex, disabled status, and inpatient diagnoses, which were taken from prior year inpatient hospital records to predict patients’ year two healthcare expenditures. The authors report that diagnoses used to predict year two costs were primarily high cost chronic illnesses such as lung cancer, diabetes, HIV/AIDS. The authors reported both ther2statistic and PRs for the model. Anr2of 6.2 percent was reported for the
PIPDCG model using a Medicare sample. The authors also reported anr2of 1.5 percent
using only demographic factors to predict year two expenditures. PRs were reported for both the PIPDCG and a demographic model. Although the PIPDCG more accurately predicted year two costs when compared to the demographic model, the PIPDCG model had a tendency to under predict (e.g. PRs as low as 0.61 for lung cancer) actual year two costs for a wide variety of diagnoses.
Ashet al.(2000) evaluated the risk-adjustment accuracy of the diagnostic cost group hierarchical condition category (DCG/HCC) model to predict future healthcare costs in three patient samples – Medicaid, Medicare, and privately insured. DCG/HCC models use age, sex, and most (utilization data such as number of hospitalizations is left out to limit gaming) diagnoses listed for an individual to predict future healthcare expenditures. This study used data from 1992 (year one) to predict individuals’ future healthcare costs in 1993 (year two). The dependent variable in this study was total covered expenses (e.g. all deductibles, third-party payments, and copayments) in year two. In contrast to the DCG model (also called the PIPDCG), which employs an individual’s primary diagnoses (or primary reason for an inpatient stay), the DCG/HCC model incorporates almost all diagnoses generated for a particular individual during all contacts with healthcare providers during a base year (Ashet al., 2000, 2001; Pope
et al., 2000; Zhaoet al., 2002 for a more thorough discussion of these models). Diagnoses are then organized into hierarchical groupings (e.g. all infections and anxiety disorders) to arrive at a general “summary” of the individual’s health. These summaries are then used to predict an individual’s future healthcare costs. Results of this study showed
that ther2value for the Medicaid sample using the DCG/HCC model was 23 percent,
Predicting future
healthcare costs
9 percent for Medicare, and 9 percent for the privately insured sample. The researchers
note that the higher r2 value reported for the Medicaid sample was due to less
variability in year two costs (e.g. less extreme high costs individuals) and the increased predictability of year two costs due to the fact that many individuals eligible for Medicaid services are disabled, resulting in most individuals using medical services.
In similar study, Ashet al.(2001) compared the ability of two risk models to identify a small group of future high cost future individuals. The researchers used data from 1997 to identify the most expensive 0.5 percent of individuals in 1998. Two models were compared. Model one (i.e. DCG/HCC) predicted future healthcare costs by using age, gender, and a range of costly medical problems encountered during 1997. Model two used prior costs from 1997 to predict costs in 1998. Results showed that both models were relatively equal in their ability to predict future high cost individuals. Unfortunately, the authors did not report the percentage of variation accounted for in costs by each of the models (e.g. traditionalr2), however, previous studies have shown that these two models account for approximately 9 and 6 percent of the variation in expected costs, respectively, (Ash and Byrne-Logan, 1998).
Both the DCG and Prior-Cost models were able to predict a small subset of high cost individuals. For example, 43 percent of the DCG high cost group and 41 percent of the prior cost top group accrued healthcare costs of $10,000 or more in year two. However, 44 percent of the DCG top group and 46 percent of the prior cost top group accrued healthcare costs between $0 and $4,999 in year two, demonstrating that both risk-adjustment methods predict:
. a small of portion of individuals who will be high utilizers; and
. a relatively equal size of individuals who fail to become among the top utilizing groups of patients.
When compared to the Prior-Cost model, the DCG model predicted more expensive individuals (on average, individuals in the DCG top group cost $1,983 more than those in the prior cost group). Also, model one was better able to identify high cost individuals with chronic illnesses such as diabetes. For example, 28 and 20 percent of individuals in the DCG and prior cost groups, respectively, were diagnosed with diabetes. The authors argue that model one is perhaps more useful (with respect to deciding how to best allocate healthcare resources) given its ability to predict high cost individuals with a higher prevalence of chronic illnesses. The ability to predict high cost individuals with chronic illnesses has the added advantage of identifying medical problems that can be managed medically and psychologically, which can potentially result in lower future healthcare costs.
Riley (2000) examined both the ability of demographic and diagnoses-based risk-adjustment models (e.g. PIPDCG and DCGHCC) to predict costs for a Medicare sample of individuals with difficulties conducting activities of daily living (ADLs) (e.g. difficulties with transportation or cooking). PRs were used to measure each model’s accuracy. Results of the study showed that both demographic and diagnoses-based risk-adjustment models under and overpredicted future healthcare costs for individuals displaying difficulties with a wide variety of ADLs. For example, demographic and diagnoses-based models (i.e. PIPDCG and DCGHCC) underpredicted costs for individuals reporting problems with five or six ADLs, with PRs of 0.46, 0.55, and 0.64, for each respective model. Riley also found that diagnosis-based models
JHOM
20,2
(i.e. PIPDCG and DCGHCC) overpredicted costs for institutionalized individuals (e.g. living in a long-term care setting), with PRs of 1.35 and 1.57, for each respective model. In conclusion, demographic and diagnoses-based models underpredict healthcare costs for individuals with severe functional disabilities by 50 percent, while diagnosis based models (PIPDCG and DCG/HCC) were found to overpredict costs for institutionalized persons by approximately 46 percent.
Combined (i.e. prospective and retrospective) models
Carteret al.(2000) examined the risk-adjustment accuracy of the clinically detailed risk information system for cost (CD-RISC) using samples from Medicaid and two Health Maintenance Organizations. CD-RISC procedures include a selection of risk – adjustment models that incorporate different patient variables. For example, CD-RISC models offer both prospective and retrospective risk models. All of the CD-RISC models incorporate demographic (i.e. age and sex) and diagnoses (i.e. inpatient records) information to predict future healthcare costs. However, some of the models included additional clinical information such as the occurrence of high cost episodes of illness or selected clinical events (e.g. bone marrow transplant or child birth) that occurred during the payment year. Finally, some of the CD-RISC models combine both prospective and retrospective data to predict payment year expenditures.
Results of the study showed that the CD-RISC models varied in their ability to
predict future healthcare costs, with r2 values ranging from 9 to 37 percent. These
values tended to increase as additional clinical information was included in model. For example, the prospective CD-RISC model that included both birth episodes and health condition of the newborn during the payment year accounted for approximately 17 percent of the variance intotal payment year expenditures, while models that included the occurrence of selected clinical conditions, such as lung cancer or blood disorders, accounted for as much as 29 percent. Finally, purely retrospective models using age, sex, and diagnoses accounted for 37 percent of the variance in expenditures for year two.
The CD-RISC retrospective models presented in this study account for a substantially larger portion of variation in expected costs than the prospective models. This larger explanatory power in retrospective models may be partly due to the inclusion of patient information such as acute and chronic diseases that are diagnosed for the first time during the year for which the risk-adjusted payment is to be estimated. For example, illnesses such as ischemic heart disease are associated with intense acute episodes that can require relatively expensive medical care (e.g. inpatient hospitalization). Furthermore, certain high cost medical situations such as the birth of a child are often associated with relatively predictable medical costs, which can increase the risk-adjustment accuracy of the retrospective risk-adjustment model. In addition, the CD-RISC retrospective model utilizes data such as pregnancy and high cost illnesses such as lung cancer that can be relatively straight forward with respect to their associated healthcare costs. For example, the costs associated with child birth for the year that the birth occurred are often stable and therefore predictable. Thus, healthcare organizations that are attempting to identify unnecessary (i.e. patients that present with symptoms that are not amenable to medical treatments) may find the CD-RISC retrospective model less useful than models that use data that may indicate a patient is having difficulty managing a chronic illness.
Predicting future
healthcare costs
Strengths and limitations of diagnosis-based risk-adjustment models
At the time this paper was written, there were several companies such as DxCG (www. dxcg.com/) and 3M (www.3m.com/us/healthcare/his/products/coding/clinical_risk. jhtml) that provide diagnosis-based risk-adjustment models to healthcare providers and payors.
Therefore, the following section presents and discusses some of the strengths and limitations of risk-adjustment methodology in order to better orient organizations considering the implementation of risk-adjustment to some important issues.
Strengths
Diagnoses-based risk models are more accurate than demographic models. Several studies have shown that risk-models that employ only a patient’s demographic information account for a relatively small portion of variation in future medical costs (Ashet al., 2001; Carteret al., 2000; Kapuret al., 2000). For example, a study conducted
by Kapur et al.found that using demographic data such as a patient’s gender, age,
ethnicity, marital status, and primary language (i.e. Spanish or English) accounted for less than 1 percent of the variance in expected future medical costs. Similarly, Ashet al.
(2000) applied a demographic model (using patient’s gender and age) to a private and Medicaid population and found that it only accounted for 7 percent of future costs. In contrast, risk-adjustment models that incorporate data such as patient diagnoses can account for more variation in future healthcare care (i.e. 23 percent as in the DCG/HCC model presented by Ashet al., 2000).
Perhaps these models can be further improved when full accurate diagnostic status is used. For example, there is evidence to suggest that many mental health
diagnoses such as depression and substance abuse are often missed (Katon et al.,
1999).
Diagnosis-based risk models can more accurately reflect expenses of patients within disease categories when compared to demographic models. In their study, Ash et al.
(2000) found that models that include only patient demographics such as age and sex, severely underpredict healthcare costs associated with a wide variety of disease conditions (e.g. asthma, hypertension, diabetes, and depression). For example, Ashet al.
calculated PRs for several disease conditions using both demographic and DCG/HCC (i.e. diagnosis-based) models and found that the PRs for the demographic model were never over 0.50, meaning that the demographic model never predicted more than one-half the actual costs for any of these conditions. In contrast, the authors reported PRs using the DCG/HCC model (for the same conditions) between 0.95 and 1.05, thus demonstrating the added accuracy of using patient diagnoses when predicting healthcare costs.
Diagnosis-based risk models can identify some patients that may be at risk for future healthcare problems. Diagnostic risk models can provide information such as the number of high cost patients identified by the model with a specific disease condition (e.g. diabetes) and comorbidities associated with specific conditions (e.g. vascular disease). This information may be useful for making:
. predictions such as which patients are most likely to consume future healthcare
resources; and
. decisions regarding how to best allocate healthcare resources (i.e. disease
management programs) (Zhaoet al., 2002).
JHOM
20,2
Limitations
In general, past risk-adjustment models (both demographic and diagnosis-based) account for relatively little variance in expected healthcare costs. One difficulty with using a statistic such asr2to measure a risk model’s performance is interpreting the significance of the statistic. Nevertheless, the majority of prospective risk-adjustment models reviewed in this paper can account for between 2 and 16 percent of the variation in future healthcare costs. Some researchers interpret these values as low, meaning these models account for a relatively small portion of the variation in future healthcare costs (Kapuret al., 2000; Popeet al., 2000). Popeet al.argue that until more clinically detailed and precise data are used, risk-adjustment models will probably remain relatively weak in predicting future costs for individual patients. An organization considering the use of risk-adjustment methodology should consider the following question, “how much variation in future expenditures accounted for by a risk-adjustment model warrants its implementation?”
Past risk-adjustment models under- and overpredict healthcare expenditures. One economic consequence of under and/or overpredicting healthcare costs is making inappropriate risk-adjusted payments to providers. For example, the DCG/HCC model severely underpredicts healthcare costs for patients diagnosed with arthritis (Ashet al.,
2000). In a study conducted by Ashet al., the DCG/HCC model predicted that roughly
4,000 arthritics would cost $4,300 each in year two, where there actual costs were approximately $5,800. If this model were used to provide risk-adjusted payments to healthcare providers (e.g. hospital), the organization would receive $6,000,000 less in capitated payments than if the model were 100 percent accurate.
The “generalizability” of a risk-adjustment model’s performance can vary across samples of individuals. The term “generalizability” in this context means the extent to which the predictive accuracy of a risk-adjustment model for one sample of individuals can be assumed to exist for a sample of individuals from a different population. There are several factors that may limit the generalizability of risk models. First, using information such as demographic data (e.g. age) to predict expected costs may be useful for some samples of individuals and less useful for others. For example, when predicting costs for a sample of older adults (e.g. 70 and over), age may be a useful for predicting which adults are more likely than others to exhibit functional disabilities (i.e. difficulties completing activities of daily living) in the next year. However, age may be a less useful factor when predicting the same difficulties in a population of younger individuals due to the fact that the prevalence of functional disabilities in younger individuals is likely to be lower. A second factor that may reduce the generalizability of a risk-model’s performance is the variability in expected costs for a given sample of individuals. For example, Ashet al.(2000) found that their DCG/HCC model was able to account for a greater portion of variation in a Medicaid sample than in a privately insured and Medicare sample due to less variability in expected costs (e.g. lower frequency of expensive cases).
Developers of risk-adjustment methodologies tend to include data in their models that reduce unwanted incentives, instead of using data that identify eliminable medical and behavioral health problems. Depending on the predictors used, risk-adjustment models can set-up a variety of unwanted, perverse incentives for providers, such as making their enrollees look sicker for the purpose of gaining larger risk-adjusted payments, and providing unnecessary medical care such as inpatient hospitalization. For example,
Predicting future
healthcare costs
some researchers argue that using utilization data such as prior inpatient costs as a predictor variable, may give healthcare providers incentives to unnecessarily
hospitalize some individuals (Pope et al., 2000; Kapur et al., 2000). As a result,
researchers attempt to prevent or reduce these incentives by avoiding the inclusion of certain types of medical data such as utilization records (e.g. frequency of hospitalizations and some diagnostic categories) (Ash et al., 2000; Pope et al., 2000). One such model, the PIPDCG model, uses an individuals’ primary or most costly diagnosis from an inpatient stay to predict future healthcare costs. All other inpatient stays (and reasons or diagnoses for the stays) are ignored to avoid unwanted incentives. However, as noted by Popeet al.this problem is not avoided by using this strategy. As healthcare providers familiar with a model’s criteria for diagnosis inclusion could unnecessarily diagnose a patient with a more “severe” diagnosis to have it counted in the risk-adjusted payment.
Not only does the methodological decision to leave out utilization data have the potential for reducing the predictive accuracy of a risk adjustment model, it limits the usefulness of a model to identify medical or behavioral health problems such as a patient’s difficulties managing a chronic medical condition (e.g. diabetes) or the presence of a substance use disorder. Said differently, by failing to incorporate utilization data, little is understood about “why” an individual might be utilizing disproportionate amounts of healthcare services, which may have important implications for making treatment decisions. For example, pathways to medical utilization can include: lack of information on how to best use the medical system; the presence of unmanaged stress; unhealthy lifestyle habits; lack of social support; somatization; negative values and beliefs about the healthcare system; and
undiagnosed psychological problems (Friedman et al., 1995; O’Donohue and
Cucciare, 2005). Specifically, an individual who is frequently presenting to primary care or the emergency room with reoccurring injuries related to falling, car accidents, or fights may be suffering from a substance use disorder.
In addition, although using a patient’s primary or principle diagnosis to predict future healthcare costs may provide some information about the medical or behavioral health problems they may have experienced during a base year, it provides little information about the types or frequency of services used. These pieces of information are important because they can provide “indicators” (i.e. diabetes related complications, feelings of sadness and suicidal ideation, and functional disabilities such as occupation difficulties) of the types of problems an individual might be suffering from such as difficulties managing weight or blood glucose, depression, and substance use, which can have important implications for how a healthcare organization might most effectively allocate healthcare resources.
Implementing risk-adjustment methodology can be complicated and expensive. Axelrod and Vogel (2003) outline seven steps involved in the implementation of risk-adjustment methodology that span from data collection to data analyses. The following is a brief outline of these steps. The first step involves collecting data. Data can be collected from databases such as computerized or hand written medical charts and computerized programs containing claims information. Choosing a method of data collection depends on several factors such as cost of implementing a data collection methodology and ease of access to necessary data (e.g. diagnosis). The second step in the risk adjustment implementation process is organizing the data. This involves
JHOM
20,2
“cleaning” (e.g. discarding outliers) the data to make the data more useable and inputting the data into a data analysis software program. The third step involves creating new variables and/or modifying variables from existing data sets to prepare for data analysis. Steps four and five are called information mining and modeling. The former involves interpreting data in an intuitive and clinically meaningful way, while the latter involves understanding the relationships among the variables and looking for more useful predictors within data sets. The final step includes characterizing the relationship between independent variables (or predictors) and dependent variables or (outcomes) (e.g. linear or curvilinear).
Healthcare organizations interested in implementing risk-adjustment methodology may be confronted with a variety of barriers with respect to preparing and successfully completing the above steps. For example, it can be expensive and time consuming to implement new data collection methods (e.g. computerized medical records), devising methods for data collection, and hiring statistical consultants to analyze software outputs and interpret statistical analyses. Berman (2003) reports that total costs associated with implementing DCG methodology (marketed and sold by DxCG) starts at $25,000, with these costs presumably rising as a function of a healthcare organization’s needs with respect to the implementation steps discussed above.
In addition to cost, implementation of risk-adjustment methodology can also be complicated, depending on the organization’s current infrastructure (e.g. current
information systems). Kapuret al.(2000) reported that among the data required by
their most accurate model were inpatient and outpatient costs for the two years prior to the payment year. Depending on the information available to an organization, the acquisition of these data may require complex software, the hiring of statisticians to assist in data collection, analyses, and interpretation. Also, if data are being collected from patient charts, an organization may need to shift needed company resources toward the collection and organizing of data from handwritten files to software databases. Therefore, before implementing risk-adjustment, healthcare organization should consider the following questions:
. What is the probability that useable, valid, or accurate data are currently
available (e.g. how often are diagnoses updated in patient charts, are comorbidities systematically collected and updated, are these data verified for accuracy)?
. How difficult/expensive is it to gather the necessary data?
. Are expensive statistical programs and employee training needed to complete
the analyses and interpretation?
. Does the organization need to hire a statistician to figure out how to transform, analyze, and interpret the data? and
. What are the costs per increases in model accuracy?
Summary and recommendations
The purpose of this paper was to provide a review of the risk-adjustment literature, with a special emphasis on presenting the strengths and limitations of extant risk-adjustment models. The following section discusses recommendations for increasing the predictive accuracy and usefulness of risk-adjustment models in the context of predicting healthcare costs.
Predicting future
healthcare costs
It is clear that over the last decade, many advances in risk-adjustment methodology have been made. For example, there has been a focus on the part of researchers to transition away from including only demographic data in their risk-adjustment models to incorporating patient data (e.g. diagnosis) that are more predictive of healthcare costs. This transition has resulted in more accurate risk-adjustment models and models that can better identify high cost patients with chronic medical conditions (e.g. diabetes). The latter advancement has important implications for healthcare providers when deciding how to most effectively allocate scarce healthcare resources to patients. However, despite these advances, past risk-adjustment models have failed to demonstrate that they can accurately and reliably predict healthcare costs. This is evident in the minimal amount of variance accounted for by most models reported in this paper (i.e. 2-16 percent) and their inability to accurately estimate future costs associated with medical conditions such as arthritis. For example, the DCG/HCC model
presented by Ash et al.(2000) underpredicts healthcare costs for patients diagnosed
with arthritis by as much as 36 percent.
This is not to say that risk-adjustment methodology should be abandoned, but it needs to be improved to warrant the costs (e.g. financial and time) of implementation. There are several ways that may strengthen the predictive accuracy of past risk-adjustment models. First, data should be used that help predict the likelihood of a patient developing preventable and potentially expensive disease conditions (e.g. coronary heart disease and diabetes). Health risk appraisals (HRAs) have been used for years to predict both mortality and patients’ risk of developing a disease by including information such as family history and lifestyle habits (e.g. tobacco and alcohol use, medical history, blood pressure, physical activity, weight, and sleep habits). HRAs use information obtained from epidemiologic studies and mortality statistics to predict the incidence of disease and mortality. Studies have shown that HRAs can be useful in predicting both incidence of disease and mortality for a variety of patient samples (Foxman and Edington, 1987; Gazmararianet al., 1991; Smithet al., 1987, 1991). What might be needed is to articulate and measure specific pathways to high utilization
instead of focusing on correlated proxy variables. Friedmanet al.(1995) for example
have articulated some pathways (e.g. decision support needs, undiagnosed mental illness, and poor lifestyle habits) while O’Donohue and Cucciare (2005) have articulated others (e.g. mental health problem presenting as a physical problem). Thus, risk-adjustment researchers may want to gather and include data in their models at still a more molecular level.
It is important to note that adding additional data such as those found in HRAs may not be financially wise for all parties interested in predicting healthcare costs or identifying potential high costs individuals. Issues such as the cost and incremental usefulness associated with collecting additional data needs to be considered. For example, it might make little or no financial sense for a healthcare provider employing inexpensive disease management interventions to diabetic patients such as brochures, emails, and telephone contact to spend additional money to identify who is likely to develop diabetes, when it may be less expensive (when compared to the cost of implementing risk-adjustment methodology) to send all covered individuals the same materials.
The second recommendation to improve the predictive accuracy of risk-adjustment methodology is to include all data that are useful in predicting future healthcare costs,
JHOM
20,2
regardless of the unwanted incentives that might be developed. For example, frequency of inpatient hospitalizations and “reasons” (e.g. physical symptoms with no identifiable medical etiology) or diagnoses for each patient medical visit have been ignored by some researchers to avoid developing unwanted incentives (Ashet al., 2000; Popeet al., 2000). However, the decision to leave out such data may have limited both the predictive accuracy and as a result the usefulness of their respective models. This is not to say that the development of unwanted incentives should be ignored but not at the expense of developing accurate and useful risk-adjustment models.
References
Armour, S. and Appleby, J. (2003), “As health care costs rise, workers shoulder burden”,USA Today, available at: www.usatoday.com/news/nation/2003-10-20-healthcare-cover_x.htm (accessed December 2, 2003).
Ash, A.S. and Byrne-Logan, S. (1998), “How well do models work? Predicting health care costs”,
Proceedings of the Section on Statistics in Epidemiology of the American Statistical Association, Dallas, TX.
Ash, A.S., Ellis, R., Pope, G.C., Ayanian, J.Z., Bates, D.W. and Burstin, H.et al., (2000), “Using diagnoses to describe populations and predict costs”, Health Care Financing Review, Vol. 21 No. 3, pp. 7-28.
Ash, A.S., Zhao, Y., Ellis, R.P. and Kramer, M.S. (2001), “Finding future high-cost cases: comparing prior cost versus diagnosis-based methods”,Health Services Research, Vol. 36 No. 6, pp. 194-206.
Axelrod, R.C. and Vogel, D. (2003), “Risk-adjustment modeling in health plans”, Disease Management and Health Outcomes, Vol. 11 No. 12, pp. 779-87.
Berk, M.L. and Monheit, A.C. (2001), “The concentration of health care expenditures, revisited”,
Health Affairs, Vol. 20, pp. 9-18.
Berman, J. (2004), “DxCG’s predictive modeling can project patient costs”,Health-IT World, availabl at: www.health-itworld.com/enewsarchive/e_article000176106.cfm (accessed November 15, 2004).
Carter, G.M., Bell, R.M., DuBois, R.W., Goldberg, G.A., Keeler, E.B. and McAlearney, J.S.et al., (2000), “A clinically detailed risk information system for cost”,Health Care Financing Review, Vol. 21 No. 3, pp. 65-91.
Cucciare, M.A. and O’Donohue, W.T. (2003), “Integrated care and the high utilizer: an explication of medical usage patterns and the role in the healthcare crisis”, in Cummings, N.A., O’Donohue, W.T. and Ferguson, K.E. (Eds),Behavioral Health as Primary Care: Beyond Efficacy to Effectiveness, Context Press, Reno, NV.
Cummings, N.A., Cummings, J.L. and Johnson, J.N. (Eds) (1997),Behavioral Health in Primary Care: A Guide for Clinical Integration, Psychosocial Press, Madison, CT.
Foxman, B. and Edington, D.W. (1987), “The accuracy of health risk appraisal in predicting mortality”,American Journal of Public Health, Vol. 77 No. 8, pp. 971-4.
Friedman, R., Sobel, D., Myers, P., Caudill, M. and Benson, H. (1995), “Behavioral medicine, clinical health psychology, and cost offset”,Health Psychology, Vol. 14, pp. 509-18. Gazmararian, J.A., Foxman, B., Tze-Ching Yen, L., Morgenstern, H. and Edington, D.W. (1991),
“Comparing the predictive accuracy of health risk appraisals: the centers for disease control versus carter center program”,American Journal of Public Health, Vol. 8, pp. 1296-301. Greenwald, L.M. (2000), “Medicare risk-adjusted capitation payments: from research to
implementation”,Health Care Financing Review, Vol. 21 No. 3, pp. 1-5.
Predicting future
healthcare costs
Howell, D.C. (1995),Fundamental Statistics for the Behavioral Sciences, 3rd ed., Duxberry Press, Belmont, CA.
Kapur, K., Young, A.S. and Murata, D. (2000), “Risk adjustment for high utilizers of public mental health care”,The Journal of Mental Health Policy and Economics, Vol. 3, pp. 129-37. Katon, W., Von Korff, M., Lin, E., Simon, G., Walker, E. and Unutzer, J.et al., (1999), “Stepped collaborative care for primary care patients with persistent symptoms of depression”,
Archives of General Psychiatry, Vol. 56, pp. 1109-15.
Moon, J.R. (1997), “Partnerships with primary care physicians: reinventing gatekeeper relationships”, in Cummings, N.A., Cummings, J.L. and Johnson, J.N. (Eds), Behavioral Health in Primary Care: A Guide for Clinical Integration, Psychosocial Press, Madison, CT, pp. 305-23.
O’Donohue, W.T. and Cucciare, M.A. (2005), “Pathways to medical service utilization”,Journal of Clinical Psychology in Medical Settings, Vol. 12 No. 2, pp. 185-97.
Pope, G.C., Ellis, R.P., Ash, A.S., Liu, C.F., Ayanian, J.Z. and Bates, D.W.et al., (2000), “Principal inpatient diagnostic cost group model for medicare risk adjustment”, Health Care Financing Review, Vol. 21 No. 3, pp. 93-118.
Riley, G.F. (2000), “Risk adjustment for health plans disproportionately enrolling frail medicare beneficiaries”,Health Care Financing Review, Vol. 21 No. 3, pp. 135-48.
Smith, K.W., McKinlay, S.M. and McKinlay, J.B. (1991), “The validity of health risk appraisals for coronary heart disease: results from a randomized field trial”,American Journal of Public Health, Vol. 81 No. 4, pp. 466-70.
Smith, K.W., McKinlay, S.M. and Thorington, B.D. (1987), “The validity of health risk appraisal instruments for assessing coronary heart disease risk”,American Journal of Public Health, Vol. 77 No. 4, pp. 419-24.
Young, A.S., Sullivan, G., Burnham, M.A. and Brook, R.H. (1998), “Measuring the quality of outpatient treatment for schizophrenia”, Archives of General Psychiatry, Vol. 55 No. 7, pp. 611-7.
Zhao, Y., Ash, A.S., Ellis, R.P. and Slaughter, J.P. (2002), “Disease burden profiles: an emerging tool for managing managed care”,Health Care Management Science, Vol. 5, pp. 211-9.
Further reading
Greenwald, L.M., Esposito, A., Ingber, M.J. and Levy, J.M. (1998), “Risk adjustment for the medicare program: lessons learned from research and demonstrations”,Inquiry, Vol. 35 No. 2, pp. 193-209.
Corresponding author
Michael A. Cucciare can be contacted at: [email protected]
JHOM
20,2
162
To purchase reprints of this article please e-mail:[email protected]