Performance Profiles: The Influence of Patient Satisfaction Data on
Physicians’ Practice
Elizabeth A. Rider, MD, MSW*‡§, and James M. Perrin, MD*§
ABSTRACT. Objective. Health maintenance organi-zations and other payers increasingly use patient satis-faction data to profile physician performance. Little is known about physicians’ use of patient satisfaction in-formation or how profiles affect individual physician behaviors. The objective of this study was to examine primary care physicians’ perceptions of performance pro-files based on patient satisfaction data, whether physi-cians use profiles to change practice behaviors, and which profile components physicians think are impor-tant for assessing quality of care.
Methods. A written survey was conducted in 1998 in Massachusetts with 810 primary care physicians (304 pe-diatricians, 201 family practitioners, 305 internists) who had at least 100 patients in a large managed care plan and had received 1 or more profiles based on patient satis-faction data. Physicians in training were excluded. Phy-sicians’ perceptions of profiles and their reported use to change practice behaviors were measured.
Results. The response rate was 68%. Twenty-three percent reported that profiles were very or extremely useful for improving care. Only 7% reported using pro-files often or always to change care. Although specific profile components related to interpersonal aspects of care were rated more useful,<11% reported using pro-files often or always to make changes on any individual component. A majority, 67% to 89%, reported making no or minor changes on profile components. Responses did not vary by specialty, demographics, or practice charac-teristics. Physicians rated interpersonal factors (eg, abil-ity to communicate with patients, abilabil-ity to show caring and empathy) as the most important indicators of quality of care; they report having the most control over these factors. Office factors (eg, staying on schedule, ease of scheduling appointments) were ranked as least impor-tant for assessing quality of care.
Conclusion. Although health maintenance organiza-tions and other payers increasingly use patient satisfac-tion reports to profile individual physicians and guide physician compensation and health plan participation,
<25% of primary care physicians find profiles useful for improving patient care and even fewer report using pro-files to change practice. Propro-files likely have limited in-fluence on behavior changes. Payers who invest in pro-files may find it advantageous to focus on health plans and practice facilities rather than on individual
physicians.Pediatrics2002;109:752–757;performance pro-files, patient satisfaction, physician report cards.
A
s health care delivery systems experience un-paralleled economic and competitive chal-lenges, they have adopted quality improve-ment principles to improve their economic situation, with a particular focus on patient-centered care and consumer satisfaction. Patient satisfaction, along with outcomes and costs, has become an important measure of health system performance.Recent use of patient satisfaction surveys has stirred controversy.1,2 New reporting systems rate individual health care providers, with insurance companies and other payers using patient satisfac-tion data to profile individual physician perfor-mance—a form of “report card.” Many health care systems profile providers.1,3,4 Plans use profiles to guide physician enrollment and compensation (eg, bonuses, withholds, capitations, base salaries).5–7 Controversy exists regarding individual physician profiling. If patients with special health care needs find more fault with the health care system, then the use of physician profiles may introduce a barrier to care for these patients by limiting physicians’ will-ingness to care for them8unless they are adequately case-mixed.9,10 Others think that profiles are not a reliable measure of physician quality of care.11,12
Feedback of satisfaction survey results is an im-portant component of performance evaluation and quality improvement and a tool for improving care.13–15 From a patient care perspective, feedback can help physicians to identify problems in the pro-cesses of care (not necessarily attributable to the phy-sician) and stimulate review and improvement of practice behaviors, which may ultimately result in improved quality of care.
Few data exist on how physicians perceive perfor-mance profiles. One previous study16surveyed aca-demic physicians about their attitudes concerning potential uses of patients’ reports about their care. Most agreed with using patient judgments to change specific physician behavior, in physician education, to evaluate students and house staff, and for feed-back from patients. Few agreed with using patient judgments to influence physician compensation or publishing reports to help patients select physicians. Also, little is known about provider characteristics that may predict which physicians will use perfor-mance profiles to improve their practices. Physician groups with longer histories and a greater
percent-From the *Department of Pediatrics and ‡Office of Educational Develop-ment, Harvard Medical School, Boston, Massachusetts; and the §Division of General Pediatrics, MassGeneral Hospital for Children, Boston, Massachu-setts.
Received for publication Feb 13, 2001; accepted Nov 21, 2001.
Reprint requests to (E.A.R.) Office of Educational Development, Harvard Medical School, 260 Longwood Ave, MEC 384, Boston, MA 02115. E-mail: elizabeth㛭[email protected]
age of capitated patients engage in higher levels of quality assurance activities.17
Although performance profiles may have the po-tential to enhance the care that physicians provide for their patients, many questions remain about how physicians perceive these profiles or their usefulness for improving patient care and whether physicians use them to change practice. Our study had the objective of examining physicians’ perceptions of performance profiles based on patient satisfaction data and how such data affect individual physician practice behaviors. We assessed 1) how primary care physicians view performance profiles based on pa-tient satisfaction data, 2) whether physicians report use of profiles to change practice, and 3) which pro-file components physicians report are important for assessing quality of care.
METHODS Sample
We obtained a mailing list of all primary care physicians who were providers for a major Massachusetts managed care plan, had at least 100 patients in the managed care plan, and had received at least 1 performance profile based on patient satisfaction data from the plan. Physicians in training were excluded. All eligible pedi-atricians and family practitioners in the plan were included. As
⬎700 internists were eligible, we randomly selected 305 internists. The final sample included 810 primary care physicians (304 pedi-atricians, 201 family practitioners, and 305 internists).
Survey
We developed a written questionnaire to survey physicians’ perceptions and use of the individual performance profiles that they receive from insurance companies, other payers, and employ-ers. Questions were generated after reviewing the literature and previous surveys,8,16consultation with experts, and pilot testing.
The survey asked physicians to rate the usefulness of profiles (using 5-point Likert scales), based on patient satisfaction data, for helping them to improve patient care and how much they have used such data to change their patient care. To create the “useful” variable for these analyses, we dichotomized 5-point Likert ratings into “useful” (very or extremely useful) and “not useful” (not, minimally, or moderately useful). The “change” variable was sim-ilarly dichotomized into “change” (often or always) and “no change” (never, rarely, or sometimes). Additional queries exam-ined similar physician views of specific components of profiles (office factors, technical care, and interpersonal care). Physicians were also asked which components were the most and least im-portant indicators of quality of care and over which they had the most and least control.
We also obtained physician demographic and practice informa-tion, including specialty (pediatrics, family practice, or internal medicine), gender, years in practice, board certification, practice
situation (group, solo, community health center, or academic cen-ter or hospital), percentage of time spent in direct patient care, and percentage of managed care and capitated patients. Actual patient satisfaction ratings for the physicians were not available.
Questionnaires were mailed to physicians between January and May 1998. One week after the first mailing, participants were sent a reminder postcard. Nonresponders received up to 4 subsequent mailings 3 weeks apart. DataStat (Ann Arbor, MI) conducted all mailings and received all questionnaires.
Analysis
We initially described demographic and practice characteristics of participants and their perceptions of the extent of the profiles’ usefulness and their reported use of profiles to make changes.2
andttests were used to assess differences in usefulness, change, and important profile components by specialty and physician demographics. We used linear regression to determine indepen-dent associations of physician demographics and number of re-ports received with the useful and change variables.
RESULTS Sample
The final sample included 802 primary care phy-sicians. Of the 810 eligible physicians, 8 could not participate because they had moved out of state or were hospitalized (5), had no known forwarding address (2), or had died (1). The overall response rate was 68% (545 of 802). Pediatricians had the highest response rate (74%), followed by family practitioners (62%) and internists (58%).
Respondent characteristics are shown in Table 1. Overall, respondents were predominantly male (71%), were board certified (92%), and had a mean of 18 years (range: 2–52) in practice. These physicians spend a majority (92%) of their time in direct patient care. Sixty-two percent of their patients were in a managed care plan, and 21% were under a capitated contract. Mean number of report cards received from the health plan sampled was 2. Ninety-one percent of the respondents recalled having received a report card from the health plan sampled. Forty-seven per-cent reported having received a report card from a second major health plan and 37% from a third plan. Sixty-nine percent had received report cards from at least 1 health plan other than the plan sampled. Three percent did not recall having received report cards from any health plan.
Usefulness and Use to Change Practice
Overall, 23% of primary care physicians found profiles very or extremely useful for improving
pa-TABLE 1. Demographics and Practice Characteristics of Physicians
Variables Pediatricians
(N⫽224)
Family Practitioners (N⫽124)
Internists (N⫽175)
Male 66.7% 77.4% 69.1%
Board certified* 98.2% 87.1% 88.0%
Years in practice (mean)* 19.5 17.2 16.0
Practice situation†
Group practice 66.5% 55.5% 50.9%
Solo practice 25.1% 37.8% 34.3%
Community or academic center 8.4% 6.7% 14.8%
Professional time spent in direct patient care* 94.4% 90.0% 90.3%
Patients in any managed care plan* 66.8% 61.5% 57.2%
Patients under a capitated care contract‡ 20.2% 21.0% 23.3%
*P⬍.001. †P⫽.002.
tient care (Fig 1). Seven percent reported using pro-files often or always to change patient care. Although internists were less likely to say that profiles were very or extremely useful (17%) than were family practitioners (24%) or pediatricians (27%) and less often used them to make changes in patient care (4.6%) than did family practitioners (8.1%) or pedia-tricians (7.9%), these differences were not statistically significant.
Regression analyses showed that physicians with a lower percentage of managed care patients are more likely to use profiles to make practice changes (P ⫽
.001). There were no significant associations of the usefulness and change variables with specialty, gen-der, years in practice, board certification, percentage of capitated patients, or number of profiles received.
Profile Components
More physicians found specific components of profiles useful than the overall profile (Fig 2), al-though⬍11% reported using patient satisfaction re-ports often or always on any component to make changes. A majority found specific profile compo-nents related to interpersonal care very or extremely useful for improving patient care (courtesy and help-fulness of office staff, 67%; physician’s ability to com-municate effectively with patients, 65%; and physi-cian’s ability to show caring, concern, and empathy, 63%). Fewer than 41% found office factors (ease of scheduling appointments, ability to keep appoint-ments on schedule) very or extremely useful for im-proving patient care. Comparative group ratings, which allow physicians to compare their individual ratings with a similar group of physicians, were least often found to be very or extremely useful (35%).
Table 2 shows the percentage of physicians who make no or minor changes on components of patient satisfaction reports. Although physicians found some profile components useful, a majority, 67% to
89%, reported making no or minor changes on the specific profile components.
Quality of Care
Figure 3 lists the factors that physicians rated as most important for assessing quality of care, in order of most to least important. Physicians were also asked to indicate over which factor they had the most control.
Physicians rated the factors of keeping appoint-ments on schedule, ease of scheduling appointappoint-ments, and comparative group ratings as least important for assessing quality of care, and they reported having the least control over these variables in the same order.
Physicians rated physician–patient factors (physi-cians’ ability to communicate with patients, to show caring, concern, and empathy) as the most important indicators of quality of care and also reported having the most control over these factors.
DISCUSSION Usefulness and Use to Change Practice
Fewer than a quarter of primary care physicians reported that profiles based on patient satisfaction data were very or extremely useful for improving patient care, and⬍10% reported using profiles often or always to change patient care. Specialty, gender, years in practice, percentage of capitated patients, and number of profiles received were not significant predictors of these results.
In most instances, profiles are unsolicited and may be perceived as a threat to physicians’ sense of com-petence, autonomy, and professional judgment.18 Physicians may question profiles’ credibility12 and prefer to rely on their own experience or colleagues’ recommendations rather than payers’.19
Disagreement exists about which aspects of physi-cian performance to measure. Physiphysi-cians tend to
cus on technical quality, appropriateness of care, and patient satisfaction, whereas payers tend to focus on accessibility of care, distribution of resources, and cost.2,20Physicians may think that payers who do not share their professional concerns impose profiles. Therefore, they may discount profile data. Physicians also may perceive profiles as an effort at cost con-tainment and doubt that payers’ use of profiles are really directed toward improving patients’ health.2 Berwick21 noted that the development and imple-mentation of consumer satisfaction surveys, guide-lines, critical paths, and computerized information systems have coincided with the “commercialization of the medical marketplace.” Some health plans use such methods to reduce the costs of care and increase profits.2,22 Physicians might question whether the present focus on profiling is a sincere effort to im-prove their patients’ health and the quality of care.22 Although Kerr et al17found physicians who have a greater percentage of capitated patients to be more likely to engage in higher levels of quality assurance activities, we found that percentage of capitated pa-tients was not related to use of profiles for changing practice. The physicians we surveyed with a lower percentage of managed care patients were more likely to use profiles to make practice changes, al-though the frequency of reported change remained small. Although an earlier study16 found that aca-demic physicians agreed with the potential use of
patient judgments to change specific physician be-havior, most of our primary care physician sample reported that they did not actually use profiles to change behavior.
What motivates physicians to change the way they practice? Oxman et al23evaluated 102 trials of inter-ventions to improve practice and found that pro-grams that combine interventions have more impact on clinical practice change than do single interven-tions. Dissemination-only strategies and mailing of unsolicited materials demonstrate little or no change. Local development and implementation of commu-nications and directives with support from medical leaders and feedback promote change.18 Successful feedback is actionable and relevant for physicians. Physician involvement in developing practice crite-ria, along with feedback, increases the likelihood of change in clinical care.24
Profile Components and Quality of Care
Although many primary care physicians did not find profiles useful in general, they reported ratings on specific profile components to be more useful, in particular those having to do with the physician– patient relationship and interpersonal aspects of care. Data on interpersonal aspects of care, such as the ability to communicate effectively, the ability to show caring, concern, and empathy with patients, and “courtesy and helpfulness of office staff,” were most frequently cited as useful for improving patient care, for assessing quality of care, and as factors over which physicians have the most control. Physicians view office management factors (appointments kept on schedule, ease of scheduling appointments) as the least important indicators of quality of care, and they report having the least control over these variables.
Patients hold similar values. The quality and na-ture of the physician–patient relationship and the physician’s interpersonal skills most highly correlate Fig 2. Percentage of physicians finding patient satisfaction profiles useful and percentage reporting changing patient care by profile components (N⫽541).
TABLE 2. Percentage of Physicians Reporting Making No or Minor Changes on Profile Components (N⫽534)
Profile Components Percentage
Physician’s technical skill 89.4
Physician’s ability to show caring, empathy 86.5 Physician’s ability to communicate with patients 84.2
Physical office environment 79.9
Ease of scheduling appointments 68.9
Appointments kept on schedule 68.9
with patient satisfaction, not access, availability, or service provision.25These results hold across differ-ent cultures and health care systems.26,27
The quality of physician–patient interaction
strongly relates to health benefits, including im-proved patient compliance and symptom improve-ment,28 –30 improved recovery from surgery, short-ened hospital stays,31,32 and better management of chronic conditions.33 Effective physician–patient communication also correlates with greater overall patient satisfaction with care.34 Complaints about physicians by the public sector usually concern com-munication problems rather than competency is-sues,35 and the majority of malpractice claims arise from communication errors.36
Many patient satisfaction reports used for physi-cian profiles do not or only superficially address physician–patient factors such as the physician’s ability to communicate effectively with patients or to show caring and empathy. Research suggests that physician–patient factors, as well as other interper-sonal aspects of care, should be included in all pa-tient satisfaction surveys.37 Our participants found these components to be most useful for improving and assessing patient care and ones over which they have the most control.
Our study has several limitations. First, we studied primary care physicians in 1 state. Second, we relied on self-reports of participants’ perceptions of useful-ness and change in practice. Physicians may have made changes in their care of patients in areas other than those surveyed. Actual patient satisfaction rat-ings for the physicians in our study were unavail-able; hence, we do not know whether specific patient ratings may be associated with use of profiles to change practice. Although we had a good response rate, physicians with stronger viewpoints may have been more likely to respond to the survey.
Implications and Conclusions
Rapidly increasing use of profiles based on patient satisfaction data, especially in managed care settings, makes understanding physicians’ perceptions and use of individual performance profiles important and timely. Use of individual profiles to guide phy-sician compensation7 and determine contracts,6 health plan participation,5 and hospital privileges38 increases the importance of physician input regard-ing performance measurement.
increase profiles’ usefulness for improvement in quality of care.
The most important implication of this study may be to curtail the profiling of individual physicians. Individual physician profiles are often unreliable measures of physician quality of care,11,12 are af-fected by case mix, and if not analyzed appropriately could create barriers to care for patients with special health care needs.8 Although primary care physi-cians report as useful specific profile components related to interpersonal aspects of care, it seems that profiles do not motivate or influence behavior changes. Office factors are seen as even less useful and less important for assessing quality of care. Whereas individual physicians report that they do not or believe that they cannot act to change office factors, practice facilities may be able to make such change. Given our results and those of other studies and the financial investment involved, profiling may better focus on the health plan and practice facility rather than on the individual physician.
ACKNOWLEDGMENTS
This work was supported, in part, by the David and Lucile Packard Foundation.
We thank Paul Cleary, PhD, and Thomas Inui, MD, for helpful comments on earlier drafts of the manuscript.
REFERENCES
1. Kassirer JP. The use and abuse of practice profiles. N Engl J Med. 1994;330:634 – 636
2. Blumenthal D, Epstein AM. The role of physicians in the future of quality management.N Engl J Med. 1996;335:1328 –1331
3. Wennberg DE. Variation in the delivery of health care: the stakes are high.Ann Intern Med. 1998;128:866 – 868
4. Spoeri RK, Ullman R. Measuring and reporting managed care performance: lessons learned and new initiatives.Ann Intern Med. 1997; 127:726 –732
5. Golembesky HE. How to integrate your practice into the new health care system.Pediatrics. 1995;96(suppl):866 – 872
6. Denton TA, Matloff JM. Optimizing cardiothoracic surgery information for a managed care environment.Ann Thorac Surg. 1995;60:1522–1525 7. Micklitsch CN, Ryan-Mitlyng TA. Physician Performance Management:
Tool for Survival and Success.Englewood, CO: Medical Group Manage-ment Association; 1996
8. Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. N Engl J Med. 1996;335: 251–256
9. Hargraves JL, Wilson IB, Zaslavsky A, et al. Adjusting for patient characteristics when analyzing reports from patients about hospital care.Med Care.2001;39:635– 641
10. Zaslavsky AM, Zaborski LB, Ding L, Shaul JA, Cioffi MJ, Cleary PD. Adjusting performance measures to ensure equitable plan comparisons.
Health Care Financ Rev. 2001;22:109 –126
11. Hofer TP, Hayward RA, Greenfield S, Wagner EH, Kaplan SH, Man-ning WG. The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease.JAMA. 1999;281:2098 –2105
12. Bindman AB. Can physician profiles be trusted? JAMA. 1999;281: 2142–2143
13. Rider EA, Longmaid HE. Guidelines for clinical teachers receiving feedback from learners.JAMA. 1995;274:996
14. Longmaid HE, Rider EA. Feedback and performance improvement in
clinical medical education.JAMA. 1995;274:1092
15. Nelson EC, Wasson JH. Using patient-based information to rapidly redesign care.Healthcare Forum J. 1994;July/August:25–29
16. Covinsky KE, Bates CK, Davis RB, Delbanco TL. Physicians’ attitudes toward using patient reports to assess quality of care.Acad Med. 1996; 121:1353–1356
17. Kerr EA, Mittman BS, Hays RD, Leake B, Brook RH. Quality assurance in capitated physician groups: where is the emphasis?JAMA. 1996;276: 1236 –1239
18. Greco PJ, Eisenberg JM. Changing physicians’ practices.N Engl J Med. 1993;329:1271–1274
19. Greer AL. The state of the art versus the state of the science: the diffusion of new medical technologies into practice.Int J Technol Assess Health Care. 1988;4:5–26
20. Hanlon CR. Quality assessment and tracking results of cardiac surgery.
Ann Thorac Surg. 1997;64:1569 –1573
21. Berwick DM. Payment by capitation and quality of care.N Engl J Med. 1996;335:1227–1231
22. Chasin MR. Improving the quality of care. N Engl J Med. 1996;335: 1060 –1063
23. Oxman AD, Thomson MA, Davis DA, Haynes B. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice.Can Med Assoc J. 1995;153:142
24. Spiegel JS, Shapiro MF, Berman B, Greenfield S. Changing physician test ordering in a university hospital: an intervention of physician participation, explicit criteria, and feedback.Arch Intern Med. 1989;149: 549 –553
25. Williams SJ, Calnan M. Key determinants of consumer satisfaction with general practice.Fam Pract. 1991;8:237–242
26. Haigh-Smith C, Armstrong D. Comparison of criteria derived by gov-ernments and patients for evaluating general practitioner services.Br Med J. 1989;299:494 – 496
27. Scarpaci JL. Help-seeking behavior, use, and satisfaction among fre-quent primary care users in Santiago de Chile.J Health Soc Behav. 1988;29:199 –213
28. Brody DS, Miller SM, Lerman CE, Smith DG, Caputo GC. Patient perception of involvement in medical care: relationship to illness atti-tudes and outcomes.J Gen Intern Med. 1989;4:506 –511
29. Orth JE, Stiles WB, Scherwitz K, Hiennrikus D, Vallbona C. Patient exposition and provider explanation in routine interviews and hyper-tensive patients’ blood pressure control.Health Psychol. 1987;6:29 – 42 30. Uhlmann RF, Inui TS, Peconaro RE, Carter WB. Relationship of patient
request fulfillment to compliance, glycemic control and other health care outcomes in insulin-dependent diabetes.J Gen Intern Med. 1988;3: 458 – 463
31. Mumford E, Schlesinger HJ, Glass GV. The effects of psychological intervention on recovery from surgery and heart attacks: an analysis of the literature.Am J Public Health. 1982;72:141–151
32. Egbert LD, Battit GE, Welch CE, Bartlett MK. Reduction of post-operative pain by encouragement and instruction of patients: a study of doctor-patient rapport.N Engl J Med. 1964;270:825– 827
33. Greenfield S, Kaplan SH, Ware JE Jr, Yano EM, Frank HJL. Patients’ participation in medical care: effects on blood sugar control and quality of life in diabetes.J Gen Intern Med. 1988;3:448 – 457
34. DiMatteo MR, Taranta A, Friedman HS, Prince LM. Predicting patient satisfaction from physicians’ nonverbal communication skills.Med Care. 1980;18:376 –387
35. Richards T. Chasms in communication.BMJ. 1991;303:1385–1387 36. Shapiro RS, Simpson DE, Lawrence SL, Talsky AM, Sobocinski KA,
Scheidermayer DL. A survey of sued and nonsued physicians and suing patients.Arch Intern Med. 1989;149:2190 –2196
37. Cleary PD, Edgman-Levitan S. Health care quality: incorporating con-sumer perspectives.JAMA. 1997;278:1608 –1612
38. Blum JD. The evolution of physician credentialing into managed care selective contracting.Am J Law Med. 1996;22:173–203
DOI: 10.1542/peds.109.5.752
2002;109;752
Pediatrics
Elizabeth A. Rider and James M. Perrin
Practice
Performance Profiles: The Influence of Patient Satisfaction Data on Physicians'
Services
Updated Information &
http://pediatrics.aappublications.org/content/109/5/752 including high resolution figures, can be found at:
References
http://pediatrics.aappublications.org/content/109/5/752#BIBL This article cites 37 articles, 3 of which you can access for free at:
Permissions & Licensing
http://www.aappublications.org/site/misc/Permissions.xhtml in its entirety can be found online at:
Information about reproducing this article in parts (figures, tables) or
Reprints
DOI: 10.1542/peds.109.5.752
2002;109;752
Pediatrics
Elizabeth A. Rider and James M. Perrin
Practice
Performance Profiles: The Influence of Patient Satisfaction Data on Physicians'
http://pediatrics.aappublications.org/content/109/5/752
located on the World Wide Web at:
The online version of this article, along with updated information and services, is
by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.
