Survey Finds Physicians Very Wary of Doctor Ratings By Carrie Johnson






Full text


6 PEJ January•February/2013

Special Report: Do Doctors Measure Up?

Survey Finds Physicians Very Wary

of Doctor Ratings

By Carrie Johnson

Patient use of online ratings is low

Of those who participated in the poll, 31 percent said they worked for health systems, while 24 percent worked at hospitals and 20 percent at group practices. Other answers included private practice (12 percent), academic health cen-ters (9 percent) and managed care/insurance (1 percent).

The largest number of respondents said their primary job title was medical director (147), followed by chief medi-cal officer (115). Other titles included staff physician (103), department chair (47), vice president of medical affairs (27) and CEO (22).

Although most of the respondents (69 percent) acknowledged they had checked their profiles on an online consumer website that rates physicians, a majority said they believed use of these sites among patients is low.

About 55 percent believed 25 percent or fewer patients have used an online physician rating site, and an additional 35 percent put the number between 25 and 50 percent. Only 1 percent of respondents believed more than 75 percent of patients had used the sites.

Among the physician leaders who said they had checked their online profile, most consulted (89 percent). Also popular were (33 percent) and Angie’s List (19 percent). Other sites listed by respondents were (13 percent), (9 per-cent), (9 percent) and (1 percent).

Of the physicians who checked their online profiles, 39 percent said they agreed with their rating, and 42 percent said they partially agreed. Nineteen percent of the respon-dents said they didn’t agree.

Of the physician leaders who said they hadn’t checked their online profile, 28 percent said their reason was because they were too busy. Another 21 percent said they didn’t think patients used them. The majority —51 per-cent—listed “other” and filled in their own reasons. Among them:

In this article…

Check out the results of an ACPE survey that looks at

physician leaders’ views of measuring performance

and the quality of individual doctors.

A majority of physician leaders view online physician ratings as inaccurate, unreliable and not widely used by the patients they serve.

Those are among the findings of a new American College of Physician Executives survey about physician rat-ing systems that was sent to 5,624 ACPE members primarily working in hospitals, health systems and group practices. About 730 responded to the 21-question survey which was conducted in October and November, and many also elected to post comments about the ratings.

Results showed that many physician leaders view online reviews as little more than popularity contests. Complaints of sampling bias, invalid measurements of competency and frustration steered many respondents away from checking the rating sites.

Survey results also showed that physicians were skepti-cal of external ratings performed by agencies such as Press Ganey, the National Committee for Quality Assurance (NCQA) and The Joint Commission, although they are viewed more favorably than online consumer sites. The survey also gauged feelings about internal ratings, which earned much more favorable reviews.

But despite any misgivings physician leaders may have about the accuracy of ratings systems, most agreed on one thing: As value replaces volume as the measurement of health care outcomes, ratings are here to stay.

As one participant summarized: “Making accurate and timely health care organization and provider ratings readily available to the public and to payers and regulators is both essential to moving to patient-centered care and, ultimately, inevitable.”


“I think most patients who take the time to complete it have not had a good experience. Sampling bias.”

“No way to influence them. Why bother?”

“I deal with so many drug addicts and psychiatric patients that have the time and motivation to hurt me online that it’s not worth the aggravation to even look. It will just upset me.”

“It’s ridiculous to treat physicians as salespeople.”

“They don’t accurately reflect the competency of a physician.”

Skeptical of external ratings

Overall, most of the physi-cian leaders who participated in the survey said they were very wary of outside attempts to rate physicians. This was especially true if the rank-ings were done by patients online, but some also expressed reserva-tions about agencies that collect and compile quality ratings of physicians, such as health plan report cards or data to fulfill The Joint Commission Ongoing Professional Practice Evaluation mandate.

Asked to give their opinion about online consumer websites, only 12 percent of respondents believed they are helpful and should be made more available. A far greater number (29 per-cent) said they are not used very much by patients and don’t affect their orga-nization, while 26 percent called them a nuisance that provides no benefit. A majority of respondents answered “other” to the question and filled in their own responses, most of them questioning the validity of the sites.

“They have numerous mistakes about specialty, location and contact information,” one participant wrote. “They do not offer a valid polling methodology and operate often as purveyors of legalized slander.”

Survey Finds Physicians Very Wary

of Doctor Ratings

In your estimation, what percentage of patients in the United States have used a consumer website that rates physicians?

Answer Response % 1 0-25% 394 55% 2 26-50% 249 35% 3 51-75% 72 10% 4 76-100% 4 1% Total 719 100%

Have you ever checked your profile on a consumer website that rates physicians?

Answer Response % 1 Yes 498 69% 2 No 223 31% Total 721 100% Why not? Answer Response %

1 I’m just too busy. 62 28%

2 I don’t think patients use them. 45 21%

3 Other, please specify: 112 51%

Total 219 100%

If yes, which websites?

Answer Response % 1 159 33% 2 431 89% 3 42 9% 4 63 13% 5 91 19% 6 43 9% 7 4 1%

8 Other(s), please list: 37 8%

Did you agree with your ratings?

Answer Response %

1 Yes 185 39%

2 No 91 19%

3 Partially 203 42%


8 PEJ January•February/2013

Several other participants faulted the sites for using unscientific methods.

“They lack validity, but are advertised as valid,” one person wrote.

“Garbage in equals garbage out,” wrote another. “Patients that trust them often get what they deserve.” Others expressed frustration about their inability to influence the ratings.

“Good idea in theory but they allow angry people to do a lot of damage with no option for us to know what the issue is or how to address it,” one person wrote.

When asked to assign a numerical value to the value of online consumer websites that rate physicians on a scale of 0 to 10, with 0 being no value and 10 being extremely valuable, the physi-cian leaders who participated in the survey rated them at 3.22.

The quality ratings compiled by organizations such as Press Ganey and The Joint Commission fared a little better than consumer web-sites, although they don’t appear to be widely used by physician leaders. About 60 percent said they were aware of such ratings, while 40 per-cent said they were not.

Of those who were aware of these external rating systems, most (41 percent) described their feelings about them as neutral. Another 29 percent said the systems were help-ful and supported their use, while 14 percent said they were a nuisance and a waste of time that measure items that are not meaningful.

Among the comments respon-dents made about external ratings systems, most questioned the methods by which they collected their data:

“They are not accurate. Very small sample can give a poor quality physician four stars and an excel-lent physician half of a star.”

Generally speaking, what is your opinion of consumer websites that rate physicians?

Answer Response %

1 They are a nuisance and

provide no benefit.

180 26%

2 They are not used very much by patients and don’t really affect our organization.

203 29%

3 They are helpful and should be made more available.

80 12%

4 Other, please specify: 232 33%

Total 695 100%

Do you review your organization's internal ratings?

Answer Response %

1 Yes 462 93%

2 No 37 7%

Total 499 100%

Does your health care organization internally gather and evaluate quality ratings of physicians? (For example: individual quality dashboards created by your hospital’s quality management department.)

Answer Response %

1 Yes 516 75%

2 No 172 25%

Total 688 100%

On a scale of 0 to 10, how valuable are consumer websites that rate physicians? (0=No value, 10 = Extremely valuable)

Answer Minimum Value Maximum Value Average Value Standard Deviation Responses 1 Use cursor to adjust scale. 0 10 3.22 2.24 620

What is your opinion of your organization's internal quality ratings of physicians?

Answer Response %

1 They are a nuisance; they waste time and/or measure items that are not meaningful.

34 7%

2 I feel neutral about them. 64 12%

3 I think they are helpful and support their use.

370 71%

4 Other, please specify: 52 10%


The future


ours to shape.

Better Health 2020 sets the stage for excellence as we evolve healthcare together.

You’re at an extraordinary crossroads, experiencing challenges like never before. To help you take on the here and now and what’s to come, McKesson proudly introduces Better Health 2020, our initiative that positions you to use IT more strategically — for better business, better care and better connectivity.

We are ready to address your critical success factors: optimizing performance and quality, coordinating care, maximizing technology and navigating evolving payment models. All for better health, through 2020 and beyond.

Learn how information can shape your future


10 PEJ January•February/2013

“It shows us where we need to place emphasis with better care coordination and proactive out-reach. It has also motivated us to modify certain aspects of our EHR so as to support quality care more.”

“They help us improve care by pointing out areas of clinical weakness or patient satisfaction.”

Looking to the future

The comments left by physicians regarding their general opinions about physician ratings reflect the strong feelings this subject appears to provoke. Some were deeply pes-simistic and predicted a future where a physician’s livelihood would be dependent on the whims of patients

“There are limitations to their usefulness because of how the data are obtained and the questions asked of patients.”

“I don’t believe one can rely on their data. Many organizations say they measure physician per-formance but few consider valid issues such as patient compliance, practice demographics, etc.”

“They need to be standardized in some shape or form, and people interpreting them need to under-stand their limitations.”

The majority of respondents (55 percent) said they did not review external ratings of their organization’s physicians. Of the 45 percent who did review the ratings, many gave a mixed review of what they found.

“It is really variable,” one physi-cian leader wrote. “The data collec-tion is flawed in many cases. What is attributed to our physicians may actually be the responsibility of spe-cialists or other providers. I use all of the ratings as a general guide and take them with a grain of salt.”

Added another: “I gain insight into the dog and pony show.”

However, some of the physicians who participated in the survey said they found the ratings useful and they helped them determine where to focus improvement efforts.

“It is really mostly a broad per-spective that allows me to determine if there is a system-wide issue within our network,” one respondent wrote. Another said the ratings provided insight into “opportunities for organi-zational and individual improvement,” as well as marketing information.

Internal ratings much

more popular

Physician leaders viewed internal quality ratings far more favorably than external ratings. Seventy-five percent of the respondents said their organization internally gathered

Are you familiar with external organizations that collect and compile quality rat-ings of physicians? (For example, health plan report cards or data to fulfill the Joint Commission Ongoing Professional Practice Evaluation mandate.)

Answer Response %

1 Yes 407 60%

2 No 268 40%

Total 675 100%

What is your opinion of these external rating systems?

Answer Response %

1 They are a nuisance; they waste time and/or measure items that are not meaningful.

90 14%

2 I feel neutral about them. 263 41%

3 I think they are helpful and support their use.

185 29%

4 Other please specify: 107 17%

Total 645 100%

Do you review your organization's external ratings of physicians?

Answer Response %

1 Yes 297 45%

2 No 364 55%

Total 661 100%

and evaluated quality ratings of physicians, and the vast majority of those who gathered the ratings also reviewed them (93 percent).

Most of the physician leaders (71 percent) said the ratings were help-ful and supported their use. Just 12 percent said they felt neutral about them while 7 percent said they were a waste of time.

The main reason respondents liked the internal ratings: account-ability.

“Physicians tend to overestimate their compliance with guidelines, for example. The data are valuable in creating realistic self-awareness.”

“We often delude ourselves about how we are performing. It is good to see real information.”


McKenna on Communications

Michael McKenna, MD • VP, Medical Management & Chief Medical Officer, Advocate Lutheran General Hospital

• Centrally involved in the hospital’s strategic partnership with PerfectServe • Here, from a recent interview,

are a few jewels from Dr. McKenna.

PerfectServe: Communication Smart Enough for Medicine

On the care continuum:

“Communication, or lack thereof, has been a major cause of breakdowns and failures on a regular basis as we deliver care. It isn’t ‘how do you find out where the physician is,’ it’s really about communication among all the caregivers with the physician being the leader. How does that team deliver care and how do they communicate among each other to make sure the care is seamless across the continuum? The continuum could be before they get in the hospital, while they’re in the hospital or after discharge, but we’re going to be responsible for caring for that patient across the continuum.”

On care transitions:

“Timely transmission of information is really important when you think about all the different transitions that patients go through. All of them require a handoff, which would allow the next caregiver to have a seamless transition in their ability to care for a patient.

We don’t think about discharges anymore, it’s really how do you transition to a different level of care. And when that happens, we have to make sure the information is communicated in a timely manner to prevent any gaps in their care. ”

On healthcare reform & ACOs:

“Under health care reform, we’re going to be under the gun to deliver care in a more timely and efficient fashion, and one thing we can’t do is waste time or effort by not having appropriate communications. That’s something we have to fix, because it’s a low hanging fruit. So, if there is a way we can improve communications and take that out of the equation as a cause for waste, that’s really going to help us as we get into ACOs. So, then the challenge is, how many different ways can you use

PerfectServe to help speed up and make timely and correct, accurate transfer of information? Whether it’s information between physicians or other entities, that’s where the grab is going to be.”

On patient satisfaction:

“We know that if a patient has a concern and the nurse is asked to get ahold of the physician to see if something could be done, and it takes hours for them to get a response, that clearly affects patient satisfaction. With PerfectServe’s rules-based logic, it allows us to get ahold of the physician in a more reliable way and get an answer more quickly, which would lead to improved patient satisfaction.”

On clinician-to-physician communication:

“When you get down to it, there are virtually no areas where we’re taking care of patients today that communication isn’t extremely important. Whether you’re trying to communicate with the nurse about a change in status, trying to get a critical result from the lab or from an x-ray or you need to talk to a social worker about placement, we need to be able to help our physicians complete communications in a timely and efficient manner, and in a way that’s fundamentally different than perhaps what we’ve been doing for the last 30 years.”

On PerfectServe:

“I don’t want to waste my time, so if I have to do something extra to try to

communicate with someone, that’s not efficient. So what I like about PerfectServe is the ease of getting ahold of the people I need to communicate with. Because of the rules-bases system, I get the right physician who’s covering for the practice, and it just makes it so much simpler and more efficient to go through that type of system. PerfectServe also has a platform that could be adapted in a very reliable way to have a more tightly integrated communication system.” 1-877-844-7728


12 PEJ January•February/2013

No matter how they felt about ratings, both internal and external, most agreed they are the wave of the future and more transparency is inevitable. Instead of fighting this trend, many advocated for better, more accurate measurements and a system of checks and balances that will give physicians more ability to influence what is being said about them publicly.

“I believe consumer ratings of providers will be a strong market driver in years to come as we are bet-ter able to collect and evaluate valid quality measures across the board,” one respondent wrote. “It’s a little too soon for that to be a reality, but it makes good sense all around.”

Carrie Johnson is associate editor of PEJ.

Others believed too much com-plaining about ratings could ultimately damage physicians’ credibility. “Most physicians choose to be unaware of the ratings and spend more time discredit-ing the process than actually review-ing their data,” wrote one respondent. “Physicians don’t offer any alterna-tive rating methodologies, thus los-ing some credibility from payers and employers.”

who care more about ease of parking than quality of care.

A few suggested physicians should start taking matters into their own hands.

“We should set up sites where peers rate each other,” wrote one respondent. “Consumer rating sites tend to attract either unhappy con-sumers or shills giving themselves great reviews.”

Which best describes your career stage in management?

Question Student / resident Early-career clinician Part-time clini-cian / part-time management Full-time management Retiring / retired Total responses Mean 1 Your CURRENT management career stage 3 51 394 192 21 661 0.27

What type of organization best describes your current employer?

Answer Response %

1 Academic health center 56 9%

2 Ambulatory care center 5 1%

3 Consulting firm 3 0%

4 Government 6 1%

5 Group practice 119 20%

6 Hospital 145 24%

7 Health system 186 31%

8 Industry (pharma, medi-cal devices, biotech)

1 0%

9 Managed care / insur-ance

9 1%

10 Military 0 0%

12 Private practice 72 12%

Total 602 100%

See related column by ACPE CEO Peter Angood, MD, page 95.





Related subjects :