Interobserver Variability in the Radiographic Diagnosis of Necrotizing Enterocolitis

(1)

Interobserver

Variability

in the Radiographic

Diagnosis

of Necrotizing

Enterocolitis

Antonio G. Mata, MD, and Ronald M. Rosengart, MD

From the Department of Pediatrics, Southern California Permanente Medical Group and Kaiser Foundation Hospital, 486 7 Sunset Blvd, Los Angeles

ABSTRACT. A total of 17radiographs from seven babies

with clinical evidence of necrotizing enterocolitis were examined by six pediatric radiologits and four

neonatol-ogists from the Southern california area. The participants were asked to interpret each radiograph as to the presence or absence of necrotizing enterocolitis and to indicate whether pneumoperitoneum was present. A substantial

degree of interobserver variability was observed, with

only one of the 17 films yielding universal agreement. We

propose that this interobserver variability in interpreting abdominal films of neonates with clinically suspected necrotizing enterocolitis may in fact contribute to the reported difference in incidence of this disease among

centers. Pediatrics 66:68-71, 1980; necrotizing enteroco-litis, radiographic diagnosis, prematurity.

Although necrotizing enterocolitis (NEC) is seen

worldwide, the reported incidence from various

neo-natal centers has varied from less than 1% to 14% of premature infants.’5 Speculations as to the etiology of NEC have been extensive.’3 Variations in prac-tice, such as the use of an umbilical vessel catheter,

the positioning of such a catheter, early vs late feedings, the use of oral hypertonic feeding solu-tions, and the use or lack of use of breast milk have been postulated to explain the differences in

mci-dence from center to center.

This study examines the interobserver variability

in interpreting radiographs of neonates with

clini-cally suspected NEC. We propose that such inter-observer variability may in fact contribute to the

reported difference in incidence of this disease

among centers.

Received for publication March 19, 1979; accepted Oct 24, 1979. Reprint requests to (A.G.M.) Department ofNeonatology,

MATERIALS AND METHODS

The records of all babies cared for at our neonatal

intensive care unit with a discharge diagnosis of NEC over the past five years were reviewed. Our institution serves a population of 300,000 health

plan members, with approximately 4,000 deliveries

per year during these five years. Of these, about 250

babies per year have a birth weight less than 2,500

gm.

Twenty-three cases of NEC were found. Twenty of these were born in this institution and three were

transfers. Those babies included in the study all

had the usual clinical findings of NEC: vomiting, bilious gastric residuals, abdominal distention, and hematest-positive stools. At least one of the

radio-graphic criteria for the diagnosis of NEC (ileus,

pneumatosis intestinalis, “foamy” appearance or

evidence of free air within the peritoneal cavity)

was also present in each baby. All infants had multiple radiographs usually obtained in the

an-teroposterior and cross-table lateral positions.

Seven infants were randomly chosen for this study. From these seven cases, 17 radiographs were selected. Of these 17 films, 12 corresponding

cross-table lateral views were available. All the films were

taken at a distance of 40 in from the patient by a portable GE 15 MA machine on Gevaert brand film

and were developed by a Kodak M-6 automatic

processor.

A panel of six Southern California area pediatric radiologists, three staff neonatologists, and a senior neonatology fellow agreed to participate in the study and interpret the radiographs without clinical information. High quality copies of the radiographs

were forwarded to each participating physician

after the name of the patient was removed and each

film was color coded. The Table summarizes the

(2)

TABLE. Pertinent Clinical Data of Seven Infants*

ARTICLES

69

. Patient Film Set Birth Weight

(kg)

X-Ray Projection Infant’s

Age (days) When Film Was Ob-tamed Outcome B.B. A B 0.8 AP-CTLt AP 9 59

Perforation ileum and ascending colon,

died B.J. C D E F 1.5 AP AP-CTL AP AP-CTL 8 18 19 29

Successful medical management, survived

B.T. G H 2.3 AP-CTL AP-CTL 1 2

Perforation ileum and sigmoid colon, DIC,

died B.S. I J 2.1 AP-CTL AP-CTL 3 14

B.K. K L 2.5 AP-CTL AP-CTL 2 5

Perforation greater curvature of stomach,

DIC, survived B.C. M N 1.7 AP-CTL AP 6 7

B.M. 0 P

Q

2.0 AP AP-CTL AP-CTL 2 4 21

* Data are derived from seven patients using 17 film sets (29 films). Abbreviations used are: AP, anteroposterior view; CTL, cross-table lateral view; DIC, disseminated intravascular coagulation.

eventual outcome are displayed. Each physician was asked to decide whether each film (an antero-posterior film) or set of films (an anteroposterior

and accompanying cross-table lateral film) (1) was diagnostic of NEC, (2) compatible with but not diagnostic of NEC, or (3) had no evidence of NEC (normal film), and (4) to state whether or not pneu-moperitoneum was present. Individual comments

were solicited. The fihns were felt to be technically

of good quality by all reviewers and, when specifi-cally asked, stated the copies were excellent and did not affect the final interpretations.

RESULTS

A substantial degree of interobserver variability

is depicted in the Figure. Only one (G) of the 17

films yielded universal agreement. This film was interpreted as showing no evidence of NEC. Two of

17 films (12%) (H, P) were felt by all observers to

be abnormal. Nevertheless, even on these abnormal

studies there were striking differences: on set H, two observers felt perforation was present, while eight diagnosed NEC without perforation. On both films A and K, nine of the ten observers felt NEC

was present, but one observer felt film K was

nor-mal despite the nine others finding evidence of NEC with perforation. Similarly, film A was interpreted as showing perforation by eight of the observers, while two others did not come to this conclusion. Eight other films show even greater variation in the

interpretations. There was no consistent interpre-tation of these films (B, D, E, F, I, J, L, and

Q).

On

these films, approximately half the observers (range 4/10 to 7/10) felt NEC was present (compatible or

diagnostic) while the others read these films as

normal. Although the other three films (C, N, 0)

were interpreted by most to be normal, even here some felt NEC was present. No one observer was found to be either consistently over-reading or Un-der-reading the radiographs. In general, there was slight over-reading by the clinicians, but this was, again, not consistent.

DISCUSSION AND IMPLICATIONS

Interobserver variability in interpreting chest

ra-diographs,’4 as well as coronary arteriograms’5 has

been documented in the literature. In this report,

substantial interobserver variability in the

radio-graphic diagnosis of NEC among six experienced

pediatric radiologists and four neonatologists is de-scribed.

The diagnosis of NEC is usually made from din-ical signs suggesting bowel obstruction in the

pres-ence of abnormal radiographic findings. These

ra-diographic findings are in fact nonspecific and

in-dude the presence of ileus,’ the presence of bowel

wall edema,’6”7 or the foamy intraluminal appear-ance which may be confused with meconium or stool.’ ‘ Pneumatosis intestinalis, a manifestation of

intramural air, is a finding that some feel must be

at Viet Nam:AAP Sponsored on September 7, 2020

www.aappublications.org/news

(3)

1ABJ1C D,E FJ,GH,,IJ,,KL,,MNJ,OP Q

B.B.

B.J.

B.T.

B.S.

BK.

B.C.

B.M.

INFANTS

AND FILMS

(I) z

0

w

a-w

z

w

>

w

C’)

0

= NEGATIVE

COMPATIBLE-NOT DIAGNOSTIC N.EC

N E.C -PERFORATION

.3 COMPATIBLE-PERFORATION

Figure. Interobserver variability of radiologic interpretations of 17 sets of films obtained

in seven neonates with NEC. Note the wide range of interpretations between observers,

eg, in set F, six observers interpreted the films as negative, two compatible, and two diagnostic for NEC.

present to diagnose NEC.’8 However, this is prob-ably a relatively late finding, and in fact may be seen in other entities.’9 Likewise, the demonstration of air in the portal system and perforation of a viscus are also late findings and also may occur in other disease states.2#{176}

The early diagnosis of NEC therefore remains a very difficult one, especially in the extremely im-mature infant. We maintain that the diagnosis is too often, by necessity, based on clinical and non-specific radiographic signs. We contend that inter-observer variability makes this diagnosis even less

of an objective one. Therefore, in certain centers,

films may be interpreted as showing evidence of NEC when in fact NEC is not present.

Such over-reading of films, or a false-positive study, would tend to overestimate the incidence of such cases. This assumes that the same observer would tend to always over-read the films. However, this intraobserver variability, ie, the consistency of each observer when reviewing the identical films, was not examined in this study. Since the diagnosis of NEC is suspected primarily from clinical signs, over-reading the abdominal radiographs may be preferable to under-reading. Conversely, under-reading of the film in the presence of clinical signs of NEC may prove misleading.

The early diagnosis of NEC remains difficult but highly desirable. More sensitive methods to allow for early and accurate diagnosis are needed before various therapeutic modalities can be evaluated.

SUMMARY

Seventeen abdominal roentgenograms of

neo-nates with suspected clinical NEC were sent to six

pediatric radiologists and four staff neonatologists

in the Southern Californa area for their interpre-tation. Participants were asked to read each film

and to ascertain the absence or presence of NEC on a radiologic basis. A wide spectrum of

interob-server variability was observed with only one film

yielding universal agreement. It is proposed that

this interobserver variability may contribute to the wide range of incidence reported for NEC among

centers. Intraobserver variability was not tested in

this presentation.

ACKNOWLEDGMENTS

The authors are indebted to Ms Cheryl Tsieprati for

the preparation of the manuscript, to the six participating radiologists (Dr John Gwinn, MD, Children’s Hospital of

Los Angeles, Hooshang Kangarloo, MD, UCLA Medical

Center, Ralph Lachman, MD, Harbor General Hospital,

Victor Mikity, MD, Los Angeles County/University of

Southern California Medical Center, Arnold Veinstein, MD, Cedars-Sinai Medical Center, Los Angeles, and

Mi-chael Weller, MD, Long Beach Memorial Medical Cen-ter), and to Doctors Patrick Robbie and Gale Gordon for their film interpretations.

REFERENCES

(4)

ARTICLES

71

2. Santulli TV, Schullinger JN, Heird WC, et al: Acute NEC in infancy: A review of 64 cases. Pediatrics 55:376, 1975 3. Bell RS, Graham B, Stevenson JK: Roentgenological and

clinical manifestations of neonatal NEC. Am J Roentgenol

Radium Ther Nucl Med 112:123, 1971

4. Touloukian RJ, Posch JN, Spencer R: The pathogenesis of ischemic gastroenterocolitis of the neonate: Selective gut mucosal ischemia in asphyxiated neonatal piglets. JPediatr Surg 7:194, 1972

5. Mizrahi A, Barlow B, Beroon W, et al: NEC in premature infants. J Pediatr 66:697, 1965

6. Uoyd JR: The etiology of gastrointestinal perforations in the newborn. J Pediatr Surg 4:77, 1969

7. Book iS, Herbst JJ, Jung AL: Comparison of fast- and slow-feeding rate schedules to the development ofNEC. JPediatr

89:463, 1976

8. Krouskop RW, Brown EG, Sweet AY: The relationship of feeding to NEC. Pediatr Res 8:383/109, 1974

9. Book LS, Herbst JJ, Atherton SO, et a!: NEC in low-birth-weight infants fed an elemental formula. J Pediatr 87:602,

1975

10. Willis DM, Chabot J, Radde IC, et al: Unsuspected hyper-osmolality oforal solutions contributing to NEC in

very-low-birth-weight infants. Pediatrics 60:535, 1977

11. Santulli TV: Acute NEC: Recognition and management.

Hosp Practice November 1974, pp 129-135

12. Livaditis A, Wallgreen G, Faxelius G: NEC after catheteri-zation of the umbilical vessels. Acta Pediatr Scand 63:277, 1974

13. Mokrohisky ST, Levine RL, Blumhagen JD, et al: Low umbilical-artery catheters increase associated complications in infants N Engi J Med 299:561, 1978

14. Birkelo CC, Chamberlain WE, Phelps PS, et al: Tuberculosis case finding-a comparison of the effectiveness of various

roentgenographic and photofluorographic methods. JAMA

133:359, 1947

15. Zir LM, Miller SW, Dinsmore RE, et al: Interobserver

van-ability in coronary angiography. Circulation 53:627, 1976 16. Wayne ER, Burrington JD, Hutter J: Neonatal NEC:

Evo-lution of new principles in management. Arch Surg 110:476,

1975

17. Reid WD, Shannon MP: NEC-a medical approach to treat-ment. Can Med Assoc J 108:573, 1973

18. Dudgeon DL, Coran AG, Lauppe FA, et al: Surgical man-agement of acute NEC in infancy. J Pediatr Surg 8:607,

1973

19. Robinson AE, Grossman H, Brumley GW: Pneumatosis in-testinalis in the neonate. Am J Roentgenol Radium Ther Nuci Med 120:333, 1974

20. Arson RG, Fishbein JF: Portal venous gas in the pediatric

age group. J Pediatr 79:255, 1979

WHAT KIND OF EVIDENCE IS NEEDED?

“ . . .there was rapid growth [of coronary care units] in the United States without

benefit of controlled investigations that could have determined efficacy, effec-tiveness, and cost. CCUs were accepted unquestioningly for the treatment of acute heart disease. They have now increased to a level where approximately one half of US acute care hospitals have such units . . .. The randomized clinical

trials done in Britain have had little effect here. Perhaps skepticism of CCU effectiveness has increased, but CCU use has also increased since the first

random clinical trial . ..was published in 1971. Other studies of output and cost, cost-effectiveness, and oversupply, rational planning, and regionalization have also had little impact.

This is an unfortunate, but all too common occurrence in our health and medical care system. Therapies are often devised, accepted and proliferate

rapidly on the weakest of evidence .. . .“

Submitted by Student

From Bloom BS: Stretching ideology to the utmost. Am J Public Health 69:1269-1271, 1979.

(5)

1980;66;68

Pediatrics

Antonio G. Mata and Ronald M. Rosengart