A Belated Randomized Control Trial

(1)

The number specified, four, was sufficient to provide what the investigators regarded as reason-able power against an extreme alternative to the null hypothesis-one in which mortality was pos-tulated as fourfold less for extracorporeal mem-brane oxygenation therapy than for conventional medical therapy. The stopping rule used meant that randomization would be terminated as soon as the fourth death was recorded in either treatment group, even if that number was only one more than the number observed in the other treatment group. There were no deaths in the extracorporeal mem-brane oxygenation-treated group (in the random-ized portion of the study); hence, the difference was the maximum possible with the rule used.

I have reservations about any trial that is de-signed to detect a treatment difference of “miracle” proportions, such as the one postulated in this trial. Generally, use of such a difference for sample size rationale means either that the investigators “shopped” for a set of specifications to yield a desired sample size or that the trial is being done to demonstrate or “prove” the value of a treatment.

One can argue that there is an ethical obligation to make patients aware of the limited value of an underpowered trial before enrollment. Even stronger arguments can be made for a full disclosure of the rationale for a trial prior to randomization when investigators are no longer in a state of equi-poise regarding the merits of the treatments being evaluated. The method of consent developed by Zelen,8 if used at all, should be reserved for settings in which the state prevails. When it does not, patients should be so informed before randomiza-tion. The desire to shield patients from upsetting discussions by using nontraditional approaches to consent is a natural one, but its pursuit should be reserved for settings in which there is a legitimate state of equipoise.

One obvious consequence of testing a treatment after it is “established” has to do with the con-straints that fact places on evaluations of the treat-ment. In the case ofthis trial, the constraints meant focusing simply on mortality in the first few days of life, and then within narrow limits on the number of deaths “allowed.” As a result little can be learned about the broader question of how babies given extracorporeal membrane oxygenation therapy fare in the long run.

In retrospect, one needs merely to be reminded of the scenario regarding the introduction of an-other “high tech” therapy in caring for premature babies -the use of high concentrations of oxygen starting in the 1940s.9 The use of high concentra-tions of oxygen, while beneficial in saving lives, was ultimately associated with blindness caused by ret-rolental fibroplasia. Years after the treatment was

administered, this association gave rise to claims of damage in suits brought by patients blinded by the treatment. The fact that they might not have been alive to bring suit had it not been for the treatment did not dissuade courts from trying such claims. One can only hope that history is not repeated with extracorporeal membrane oxygenation.

Before extracorporeal membrane oxygenation is declared a winner, keep in mind that the sample sizes of the two trials of extracorporeal membrane oxygenation reported so far have a combined sam-ple size of only 31. Let us hope that this fact, along with the embarrassing paucity of useful long-term follow-up data on extracorporeal membrane oxy-genation-treated babies, is sufficient to stem a rush to establish centers to deliver the treatment until more is known about its virtues and limitations.

REFERENCES

CURTIS L. MEINERT, PHD

Department of Epidemiology

School of Hygiene and Public Health The Johns Hopkins University Baltimore, MD

O’Rourke PP, Crone RK, Vacanti JP, et al. Extracorporeal

membrane oxygenation and conventional medical therapy

in neonates with persistent pulmonary hypertension of the

newborn: a prospective randomized study. Pediatrics.

1989;84:957-963

2. Bartlett RH, Gazzaniga AB, Jefferies MR, et al.

Extracor-poreal membrane oxygenation (ECMO) cardiopulmonary

support in infancy. Trans Am Soc Artif Intern Organs.

1976;22:80-93

3. Bartlett RH, Roloff DW, Cornell RG, et al. Extracorporeal

circulation in neonatal respiratory failure: A prospective

randomized study. Pediatrics. 1985;76:479-487

4. Chalmers, TC. Randomization of the first patient. Med Clin

North Am. 1975;59:1035-1038

5. Day HW. An intensive coronary care area. Dis Chest.

1963;44:423-427

6. Flynn RL, Samuel MF, III. Coronary care programs in the

United States. Isr J Med Sci. 1967;3:279-286

7. Freedman B. Eguipoise and the ethics of clinical research.

N EngI J Med. 1987;317:141-145

8. Zelen M. A new design for randomized clinical trials. N EngI

J Med. 1979;300:1242-1245

9. Silverman WA. Human Experimentation: A Guided Step

Into the Unknown. Oxford, England: Oxford University Press; 1985

A Belated

Randomized

Control

Trial

When medical scientists and/or practitioners are asked to comment on an article that may have

(2)

COMMENTARIES

367

already created a controversy of both a scientific and ethical nature in the public press, the tempta-tion is irresistible to abandon the language of sci-entific articles and to call a spade a spade. I bow to that irresistible temptation. A plague on all houses except those who reported and conducted the study in this issue of Pediatrics.’

When the study in question was designed and executed (1985 to 1988), an extremely difficult sit-uation had been created by others. An elaborate, expensive, and potentially both effective and dan-gerous new technology had been allowed to prolif-erate without the gathering of reliable information about its relative efficacy and safety. Those answers should have been obtained years before the concep-tion of the study carried out by the Boston Chil-dren’s Hospital Group. The causes of the delay are two serious misconceptions with both scientific and ethical roots: reluctance to randomize patients at the earliest stages of new technology development2 and a serious double standard with regard to ran-domized vs nonrandomized trials of unproven ther-apies.57

I apologize ahead of time to the developers of extracorporeal membrane oxygenation for using them as an example of these grievous errors of modern medicine. From my biased and evangelistic viewpoint, they cannot be criticized because they were conforming to normal practice and “knew not what they were doing.” If they can be convinced of what I am trying to sell, they will be pleased to have supplied such an excellent lesson.

Bartlett and his group deserve the most credit for developing and popularizing what has now proven to be a good way to save the lives of thou-sands of infants.8 Their first reported successful application of extracorporeal membrane oxygena-tion was published in 1974, however, a full 15 years before the appearance of the best evidence so far that it seems to work.’ Bartlett et al’#{176}reported a randomized trial in 1985, 11 years after his first enthusiastic report, but this trial was an extreme example of adaptive designs, and only one patient received conventional therapy. Acceptance of the outcome then rested primarily on the validity of historical controls. The study was a useful experi-ment in the evaluation of adaptive designs,’2 but it did not establish the relative value of extracorporeal membrane oxygenation and left open the need for the study to be carried out by O’Rourke et al.’

Suppose that Bartlett et al’3 had randomized the study of the first patients in 1972 and continued through 1976 when the treatments of 13 consecu-tive patients were reported. Because “the learning curve” might well have been improving outcome in both the experimental and control groups, no final answer would have been obtained by then-except

that the chance for half the patients to receive the more conservative conventional therapy might have doubled the number of patients studied. Although the Bartlett group from both Irvine, CA and Ann Arbor, MI reported the most uses of extracorporeal membrane oxygenation in infants during the 1970s, a number of other studies reported by Bartlett et al’3 were active at the time. Important answers could have been found in multiple randomized con-trol trials by the early l980s when the therapy was thought effective enough to warrant the establish-ment of a registry.’4 Had the first eligible patients been randomized in a traditional 50-50 study, it would never have been necessary to start the con-tortions of the adaptive design studies reported in

198510 and 1989.’

The author of this commentary has recom-mended randomizing the first patient since 1968 and knows very few investigators other than him-self who have used it. Obviously there must be something seriously wrong with the idea.

The most common argument against the idea is that new therapies require a lot of tinkering and improvements before they can be compared justi-fiably with standard therapy. Entirely true! There would be a real danger of a false rejection, because those patients randomized to the new treatment apparently did miserably at first, although that may have been a reflection of selection rather than treatment. However, premature cessation could have been avoided by an understanding data mon-itoring committee wielding a steady hand at the helm. Cooperative trials would also have had to start early, but that should not be an obstacle in these days of modern communication methods.

The second major reason for rejection of the idea is the double standard of ethics and peer review applied to practice, pilot, and randomized trials. The report of 13 patients treated by 1976 by Barlett et al’3 mentions obtaining both approval of the university’s human subjects research committee and informed consent of the family after a detailed explanation. At a time when the new procedure could well have caused more deaths than it saved lives-9 of the 13 died, and 9 of the other 12 reviewed by Bartlett-was the reason for not ran-domizing explained to the peer review board and the parents? If the parents were told that random-ization was being postponed because a worse out-come was expected from the new procedure, they would have been ahead of their doctors in calling for a proper study. One of the best documentations of “the learning curve” in a lethal situation is in table 6 of the report ofthe registry of extracorporeal membrane oxygenation patients.’4 Survival was better when the first 10 patients in each clinic were excluded from the calculations (P < .004) and again

at Viet Nam:AAP Sponsored on September 7, 2020

www.aappublications.org/news

(3)

when the first 20 patients were excluded (P < .012).

Would some of these first 10 or 20 patients have been better off if they had been randomized to conventional therapy?

A third reason for postponing the rigors of pre-senting an randomized control trial to an

institu-tional review board and parents was undoubtedly

the hope that the new treatment would be so over-whelmingly successful that historical controls would suffice to prove its efficacy. But in what percentage of new therapy evaluations has that situation applied? Many a new moon will come and go before another penicillin, insulin, or vitamin B12 is discovered. Extracorporeal membrane oxygena-tion is also a good example of the difficulty of using historical controls, because the outcome of standard therapy has improved, or at best, earlier diagnosis of transient cases has made the use of historical controls unreliable.’5

What should be done now to avoid the extracor-poreal membrane oxygenation mistake that has been duplicated so many times with other dramatic therapies? How can one minimize the number of new technologies that are abandoned or adopted after the sacrifice of too many patients, treated or not treated, as the case may be?

1) Whenever clinicians have a bright idea that a new treatment might be more beneficial than

harm-ful, they should prepare a protocol, present it to their institutional review board, and start to ran-domize.

2) Institutional review boards should recognize the ethical and scientific deficiencies of uncon-trolled pilot trials and innovative practice. They should refrain from placing roadblocks in front of advancing medical care by demanding that clini-cians document the evidence for all that they do-not just for their randomized trials.

3) Professional ethicists should get off the backs of people performing good clinical research and recognize that practitioners also have potential con-flicts of interest when they make clinical decisions without data. In the absence of evidence of efficacy, both the withholding and the delivery of a new treatment are equally less ethical than randomized assignment under controlled conditions.

4) Third parties, governmental and private,

should recognize that they are harming people by

slowing progress when they pay for new procedures,

such as extracorporeal membrane oxygenation, be-fore they are established, or refusing reimburse-ment for innovative therapy when they should be reimbursing only when the patient is part of a properly designed and executed randomized control trial.

If these apparently revolutionary concepts were

adopted, a number of other changes would have to occur. Institutional review boards would have to require registration of all randomized control trials under way.’6”7 Data monitoring and peer review boards would have to be established to determine both the scientific validity and ethics of new studies and the proper time to stop randomizing and de-dare the innovation a winner or loser. Electronic means of constant communication would have to be devised to keep abreast of rapid changes. A portion of the enormous sums now spent on inno-vative and standard care would have to be diverted to proper experimentation.

Were such a system in place by 1980, a lot of babies might have been saved by extracorporeal membrane oxygenation in the last 9 years. Con-versely, in institutions just starting to use the new procedure, half of the babies might have been saved from extracorporeal membrane oxygenation if as-signed to standard therapy.

ACKNOWLEDGMENTS

This study was supported in part by grants

RO1HS-05936 and -05523 from the National Center for Health

Services Research and Health Care Technology

Assess-ment and by a gift from the Janssen Foundation.

REFERENCES

THOMAS C. CHALMERS, MD Harvard School of Public Health

and Boston Veterans Adminis-tration Medical Center Boston, MA

Mount Sinai School of Medicine

New York, NY

1. O’Rourke PR, Crone RK, Vacanti JP, et al. A prospective

randomized study of extracorporeal membrane oxygenation

(ECMO) and conventional medical therapy in neonates with

persistent pulmonary hypertension of the newborn.

Pediat-rics. 1989;84:957-963

2. Chalmers TC. When should randomization begin? Lancet.

1968;1:858

3. Chalmers TC. Randomization of the first patient. Med Clin

North Am. 1975;59:1035-1038

4. Chalmers TC. Randomize the first patient! N EngI J Med.

1977;296:107

5. Chalmers TC. Informed consent, clinical research and the

practice of medicine. Trans Am Clin Climatol Assoc.

1982;94:202-212

6. Chalmers I, Silverman WA. Professional and public double standards on clinical experimentation. Controlled Clin Trials. 1987;8:388-391

7. Silverman WA. The myth of informed consent: in daily

practice and in clinical trials. J Med Ethics. 1989;15:6-11

8. Bartlett RH, Andrews AF, Toomasian JM, Haiduc NJ,

Gazzaniga AB. Extracorporeal membrane oxygenation for

newborn respiratory failure: forty-five cases. Surgery.

(4)

COMMENTARIES

369

9. Bartlett RH, Gazzaniga AB, Burns NE, Fong SW. Prolonged

extracorporeal cardiopulmonary support in man. J Thorac

Cardiovasc Surp. 1974:68:918

10. Bartlett RH, Roloff DW, Cornell RG, et al. Extracorporeal

circulation in neonatal respiratory failure: a prospective

randomized study. Pediatrics. 1985;76:479-487

11. Zelen M. Play the winner rule and the controlled clinical

trial. J Am Stat Assoc. 1969;64:131-146

12. Ware JH, Epstein MF. Extracorporeal circulation in

neo-natal respiratory failure: a prospective randomized study.

Pediatrics. 1985;76;849-851

13. Bartlett RH, Gazzaniga AB, Jefferies MR, et al.

Extracor-poreal membrane oxygenation (ECMO) cardiopulmonary

support in infancy. Trans Am Soc Artif Intern Organs.

1976;22:80-93

Am Soc Artif Intern Organs. 1976;22:80-93

14. Toomasian JM, Snedecor SM, Cornell RG, Cilley RE,

Bar-tlett RH. National experience with extracorporeal

mem-brane oxygenation for newborn respiratory failure: data

from 715 cases. Trans Am Soc Artif Intern Organs.

1983;34:140-147

15. Dworetz AR, Moya FR, Sabo B, Gladstone I, Gross I.

Sur-vival of infants with persistent pulmonary hypertension

without extracorporeal membrane oxygenation. Pediatrics.

1989;84:1-6

16. Dickersin K. Report from the panel on the case for registers ofclinical trials at the Eighth Annual Meeting ofthe Society for Clinical Trials. Controlled Clin Trials. 1988;9:76-81

17. Simes RI. Publication bias: the case for an international

registry of clinical trials. J Clin Oncol. 1986;4:1529

DEMONSTRATION

OF MOTHER-TO-INFANT

TRANSMISSION

OF

HEPATITIS

B VIRUS

BY MEANS

OF POLYMERASE

CHAIN

REACTION

Department of Pediatrics, Yokohama City University School of Medicine, 3-9, Fukuura, Kanazawa-Ku, Yokohama, Japan

Summary

To investigate the failure of vaccines to prevent mother-to-infant transmis-sion of hepatitis B virus (HBV), serum, cord blood, and colostrum samples from eleven mothers, known to be carriers of hepatitis B surface antigen, and their infants were examined by means of a highly sensitive polymerase chain reaction (PCR) method. HBV-specific DNA was detected in ten maternal serum samples, eight samples of colostral whey, eight samples of colostral cells, and one cord blood sample. Four infants of mothers with HBV-DNA-positive colostrum showed low responsiveness to hepatitis B vaccine. The infant whose cord blood was positive for HBV DNA showed low responsiveness to hepatitis B vaccine and subsequently became an HBV carrier. These results suggest the need for further study to evaluate whether breastfeeding is advisable for HBV carriers.

From Mitsuda T, Mon T, Ookawa N, et al. Demonstration of mother-to-infant transmission of

hepatitis B virus by means of polymerase chain reaction. Lancet. 1989;2:886-887

ERRATUM

Noted by J.F.L., MD

In the article “Pediatric Problems in a Suburban Shelter for Homeless Families” by Bass et al (Pediatrics 1990;85:33-38), there is an error in the abstract on page 33. In the sentence, “Of 67 children screened before entrance, 3% had Gardia lamblia detected in the stool,” the percent value should be 13%, not 3%.

(5)

1990;85;366

Pediatrics

THOMAS C. CHALMERS