Cost-effective clinical decision making

(1)

atrics, which states that “usually infants requiring oxygen therapy do not need an arterial Po2 higher

than 50 torr and do well with an arterial Po2

between 50 and 70 torr.”5

WILLIAM H. TOOLEY, M.D.

J

ULIEN I. E. HOFFMAN, M.D.

Cardiovascular Research Institute, University of California, San

Francisco

San Francisco, California

REFERENCES

1. James LS, Lanman JT: History of oxygen therapy and

retrolental fibroplasia. Pediatrics 57(suppl):591, 1976.

2. Huch R, Huch A, Albani M, et at: Transcutaneous Po monitoring in routine management of infants and children with cardiorespiratory problems. Pediat-rics 57:681, 1976.

3. Conway M, Durbin GM, Ingram D, et at: Continuous monitoring of arterial oxygen tension using a cath-eter-tip polarographic electrode in infants.

Pediat-rics 57:244, 1976.

4. Wilkinson AR, Gregory GA, Phibbs RH: A new fiber

optic umbilical arterial catheter for continuous oxyhemoglobin saturation measurement. Pediatr Res 11:544, 1977.

5. Standards and Recommendations for Hospital Care of Newborn Infants, ed 6. Evanston, Ill, American

Academy of Pediatrics, 1977, pp 91-92.

Massive studies, minimal progress

Large, expensive, multidisciplined, multicenter studies present special problems to journals. The study by Kinsey et al. in this issue (p. 655) is a

good example. This study was designed by a

distinguished peer group. It was carried out in five university centers by 27 investigators, written

up for publication by ten authors, evaluated by

statisticians, and was ready for publication nearly nine years after the research was first begun. The

goal of the study was a very good one. The

authors hoped to define the level of Pao2 and the duration of exposure which could result in retro-lental fibroplasia. Nearly 10,000 blood gas studies were carried out on 589 low-birth-weight infants. No answer was really found. The sad fact is we

now realize that intermittent, infrequent Pao2

measurements do not reflect the true Po2 of an infant. Other basic flaws in the study are pointed out in the accompanying commentaries.

It is virtually impossible to reject such a study

for publication. It is also difficult to even suggest revisions in such an article. Multiple-authorship

papers

take a long

time to collectively write. If a

decade has gone by, many of these authors are no

longer in their original hospitals. Some revisions suggested by reviewers would require redoing parts of the study. This is usually impossible. The

funds have been exhausted and few members of

the original study groups are still working at the same hospitals!

It’s sad to realize that large studies, no matter how well intentioned or controlled, often don’t give useful answers.

Cost-effective

clinical

decision

making

J.F.L.

Are pediatric preoperative chest x-ray exami-nations worth doing? Sane et al. (p. 669) conclude that they are “medically and economically

justi-fled

and

essential.” To answer this question

requires at least the following information. What is the cost of the examination (dollars, radiation

exposure, cost of further testing and perhaps

inappropriate therapy resulting therefrom, and

possible prolongation of hospital stay)? What is the benefit? What percent of patients have

unsus-pected findings; of these, the percent with

modified treatment; and of these, what percent

benefited therefrom? What was the cost per unit

of benefit received? Is there some other use of

these scarce resources that would have yielded

greater benefits?

Cost-effective clinical decision making is a

quantitative and systematic approach to making

clinical decisions derived from statistics,

epidemi-ology,

economics, and decision analysis. Here is

how

one might analyze the data of Sane et al.

using this approach. Structuring the problem

comes first, then data collection. Usually critical

data are absent and approximations must be

made. I have attempted to reconstruct their data

and made some additional data assumptions that

may be very far from accurate.

This problem seems best structured by consid-ering two classes of benefits separately. These are

the detection of (1) abnormalities leading to

postponing surgery and modifying anesthesia and

(2)

Resultant Benefits

1,389 No Abnormalities 0

Patients 40 Previously Suspected 0

111

Abnormalities

Unsuspected

{

14 No Change in Treatment 0

12 Consultation 0

Surgery Anyway

57

Treatment Changed

10 Surgery Later Positive

Decision tree for patients benefiting from preoperative chest x-ray examination in the form of

modification of surgery. Tree is read from left to right. Numbers of patients are indicated in each path.

COMMENTARIES 757

.Positive

1 Lost to . Negative

Follow-up

Abnormalities Modifying Surgery

Sane et al. report 1,500 consecutive preopera-tive patients. Out of these, 111 had abnormalities and therefore 1,389 had no abnormalities. This is

shown on the first branch of the decision tree

(Figure). Of

these

1 1 1

patients, 71 had previously unsuspected abnormalities. This group defines the population benefited by these x-ray examinations.

Of these 71 patients, 57 had their treatment

modified as a result; presumably for 14 treatment

was not modified and therefore no benefit is

accrued. Of the 57 patients with modified treat-ment, 12 had surgery anyway after consultation. This is an additional cost and because treatment

was unchanged there was no benefit. For 34

patients anesthesia was modified and for 11

surgery

was postponed. Of these 11, one was lost to follow-up: If one assumes that the benefits to

surgery

are positive, then the child who was lost

tQ follow-up and did not have surgery received a

negative benefit (a loss) associated with x-ray

screening. Of the remaining ten patients with

, postponed surgery, all ten had surgery at a later

date.

Thus there are 44 patients with previously

undetected abnormalities whose treatment was modified favorably as a result of the x-ray exami-nations. This is as far as Sane et al. go. Let is make some assumptions to show how their analysis could have proceeded. If the reader objects to

these assumed numbers, please substitute

numbers more of your liking, and redo the analy-sis.

False Positives

Let us assume that 10% of these 44 patients had

false-positive

x-ray readings. Therefore, these 4.4 patients receive no benefit.

Operative Mortality

Let us translate the benefits achieved from

treatment modification into a percentage reduc-tion in operative mortality. Assume an average

reduction of 5% in operative mortality which, in

the absence of x-ray screening, we will assume to be five deaths

per 1,000

operations.

Change in Life Expectancy

We will assume these patients have a mean age of 10 years and an expected survival of 60 more

years of life. Reducing operative mOrtality is

saving

those years of life. However, since those years of life occur in the future, we will discount them to present value under the assumption that

a present benefit is more highly valued than a

benefit in the distant future. The general idea of discounting is a simple one made complicat#{231}d by the mathematics. The idea is that a child would

prefer one candy now to two candies an hour

from now. A politician would rather vote for

legislation with immediate benefit and distant

cost than immediate cost and distant benefit. A

banker is so pleased to take your dollar today that he will give you back a dollar and five cents at the end of a year. There is no consensus as to what the

discount rate should be. We have chosen a 4%

discount rate. Using the following formula, we

at Viet Nam:AAP Sponsored on September 8, 2020

www.aappublications.org/news

(3)

can obtain the present value of 60 future years of life saved:

P =.i

(i-n r ., (1+r)

where P is the present value, measured in years of life, r is the discount rate (0.04), and n is the number of years saved (60). This formula yields a present value of 22.62 years of life.

Calculating Costs and Benefits

Sane et al. give the costs of 1,500 chest x-ray examinations as $22,500. This excludes the costs

of additional diagnostic testing and modified

treatment.

The one patient lost to follow-up, we will

assume, has a loss of benefit equal to the gain in benefit to one true-positive patient with modified treatment. This is a guess made in the face of a

total absence of data. If one had access to the

authors’ records we might be able to make a

better guess. Therefore, the number of patients benefiting is [44 - 44(0.10)] - 1 = 38.6

The 5% reduction in an operative mortality

of 5 per 1,000 is on a per patient basis:

0.005 x 0.05 = 0.00025 reduction in operative

mortality per patient. Translating this into present value years of life saved per patient gives

0.00025

x 22.62

= 0.005655. Multiplying this by

the 38.6 benefited patients gives 0.218 total

present value years of life saved.

Given the $22,500 cost of x-ray examination, this translates into $22,500 + 0.218 = $103,077

cost per present value year of life saved.

Before interpreting this, let us consider the

other major class of benefits.

Unsuspected Abnormalities Unrelated to Surgery

Sane et al. list these in their Tables I and II.

One could take each one of these patients who

were treated as a result of these findings. One

could calculate the costs of treatment and the

expected benefits achieved, add them up, and

include them in an overall calculation of costs per year of life saved.

Note that some of these abnormalities may

have been discovered sooner or later without

these x-ray examinations. The benefits attached to the x-ray examinations should be the added bene-fits accruing to this earlier detection.

How much per year of life saved is too much? If money is no object, then one might do x-ray examinations up to the point where the benefits exceeding the risks of radiation are fractionally

more than the next best use of x-ray studies. If

money is scarce and doing these x-ray studies

means not doing something else, then the decision about x-ray studies should be based on the return associated with the most cost-effective activity foregone (economists call this opportunity cost).

Let us assume that if this money were not spent

on preoperative x-ray examination it could be

committed to an activity with a return of $15,000 per year of life saved (say, increased polio immu-nization or urging the surgical residents to use a stethoscope).

If one is willing to assume that any return

greater than $15,000 per year of life saved is too high, then one can return to the data and ask how great do the benefits associated with

abnormali-ties not associated with surgery have to be to

justify preoperative x-ray examination.

Let us assume that treatment costs for these

abnormalities are $50,000 since some major

surgery is involved. How many present value

years of life (z) must be saved here to yield a

return of $15,000 per year of life saved? The calculation is as follows:

$22,500 + $50,000 $15 000

0.218+z

Solving for z gives 4.615 present years of life

saved.

If these undetected abnormalities result in

more than a total of 4.615 (present ‘alue) years of life saved, then by our

$15,000-per-year-of-life-saved criteria (for this institution and this popula-tion), preoperative chest x-ray examinations are worthwhile.

Notes on the Data

The Institution. One is inclined to ask what

kind of primary care is being provided to this

population when the first time dextrocardia is

detected is on the preoperative chest x-ray film. There are two important issues here. First, the decision may be institution-specific. Second, the

question may be too narrowly defined. Perhaps

more is to be gained by reorganizing the surgical outpatient department.

The Population. What kind of population is this

that

has eight dextrocardia/dextroversions per

1,500 patients operated on? A decision

appro-priate for this population may be wrong for

another population.

Sample Size. If the decision is critically

depen-dent (sensitive) on finding a small number of

(4)

COMMENTARIES 759 a large sample size is critical. The following is not

unusual. Article A reports on 1,000 patients with

no positives, article B reports two positives on

1,500 patients, and article C reports no positives

on 500 patients. The authors of articles A and C

conclude that the effort is unnecessary, while the author of article B has clearly shown the great value of the effort. This is in spite of the possi-bility that they are drawing too small samples from the same population.

Baseline Data. Sane et al. argue that one value of having these x-ray examinations is that they are

baselines for possible future comparison. This

potential

benefit

could

also be subjected

to anal-ysis. Of 1,000 preoperative x-ray examinations

done ten years ago, how many were used for

baselines, leading to what modification in treat-ment and resultant added benefit?

Subclassifying the Patient Population

Sane et a!. presume an all-or-none decision.

Breaking down these 1,500 patients into

subgroups such as emergency/nonemergency,

below and above 10 years old, or patients with

and without x-ray examinations in the last year may allow for capturing almost all the benefits by using x-ray examination on only part of the total

population.

Missing Costs and Benefits

One could spend hundreds of hours calculating

all the possible benefits not subsumed in this

analysis. The most obvious one here is the risk of

radiation exposure. One could make more

esti-mates, or one could do a limited numerical

analysis and use judgment to decide if the

unquantified factors will shift the decision one way or another. This approach is no different than

the clinician’s using research findings as part of the basis for a clinical decision.

All Those Guesstimates

This decision must be made now, either

explic-itly or by default. It cannot wait for perfect

information. The reader must decide not that

perfection has been achieved, but rather whether

Sane et al. have used a satisfactory approach or

that

the type

of analysis described here is more useful.

As scientists, physicians demand accurate

research data. As clinicians, they must daily make hundreds of decisions about particular patients with even less information than Sane et al. have provided. There are ways to carefully think about these clinical decisions. That there are analytic processes is the important message here.

In the past, physicians could explicitly ignore the costs of what they do. This is one major reason why medical care costs so much. Soon the federal

government may set a limit (“cap”) on medical

care expenditure. Rising medical care costs are

leading to such concerns in other countries. That

is presumably why Sane et al. addressed this

problem of economics in the first place. Money

vill be limited, and someone is going to decide

how

it is used. Either physicians will do this, or

someone else will. Who should make these

deci-sions?

DUNCAN NEUHAUSER, Ph.D. Harvard School of Public Health

677 Huntington Avenue Boston, Massachusetts

For a discussion of these questions and other analytic examples, including appendicitis and tonsillectomies, see Bunker J, Barnes B, Mosteller F; Costs, Risks and Benefits of Surgery. New York, Oxford University Press, 1977.

(5)

1977;60;756

Pediatrics

Duncan Neuhauser