Impact
of an Education
Program
on Perinatal
Care
Practices
William
R. Harlan,
MD,
George
E. Hess,
MA,
Robert
C. Borer,
MD,
and
Roland
G. Hiss,
MD
From the Department of Postgraduate Medicine and Health Professions Education, and
Department of Pediatrics, University of Michigan Medical School, Ann Arbor
ABSTRACT. Education of health professionals has an
important role in improving health care. A media-based,
self-instructional, perinatal education program was de-veloped and field tested in rural and urban regions of Michigan. Cognitive tests, chart audits, and consultation/ referral times were used to measure the impact on edu-cation and patient care. The program effectively in-creased knowledge and improved patient care practices
by physicians and nurses. This study presents evidence
that a targeted educational program in a media-based format can significantly improve perinatal care.
Pediat-rics 66:893-899, 1980; education program, patient care
practices.
Important progress has been made in reducing
perinatal mortality in the United States during the past decade, but perinatal care practices often fall
short of the level that could be achieved.
Respira-tory failure remains the most common cause of death in newborn infants, and many deaths could be avoided by prevention of prematurity and by early recognition and treatment of neonatal prob-lems.’ Development of regional perinatal centers and technological advances are important recent improvements in management of the disorder.2” However, recognition, initial management, and
re-ferral remain important responsibilities for
corn-munity hospitals, and these activities require con-tinuing professional education for improvement.
To have a broad impact, an educational program must be widely disseminated and capable of effect-ing change in care practices as well as increasing knowledge. This goal has proven deceptively diffi-cult. Changes induced by continuing education in the practice behavior of health professionals are
Received for publication Oct 29, 1979; accepted April 14, 1980.
Reprint requests to (W.R.H.) G-1209 Towsley, University of
Michigan, Ann Arbor, MI 48109.
PEDIATRICS (ISSN 0031 4005). Copyright © 1980 by the
American Academy of Pediatrics.
difficult to achieve and to document.57 However, the components of a successful educational program are clear and include: valid and acceptable learning materials, an effective process for diffusion to po-tential learners, and an evaluation of the changes in health care practices. We combined these ele-rnents to develop a perinatal education program for community health professionals. The unique as-pects of this program are the use of self-prescribed, media-based learning units and the careful assess-rnent of changes in the process of patient care. This report documents the effectiveness of the program in improving knowledge and in changing practice behavior. Although the program was addressed to neonatal care and was tested in a single state (Mich-igan), the educational and evaluational paradigm has applicability in other disciplines and other re-gions.
MATERIALS
AND METHODS
Educational Program
A media-based independent study program coy-ering knowledge and skills necessary to improve
prevention, early detection, and management of neonatal respiratory distress was developed and validated using national experts as reviewers.8 The program content included information regarding
identification and handling (including referral) of
high risk pregnancies; identification and initial
management of neonates at high risk of respiratory
all materials were generally appropriate for study by both physicians and nurses. To permit selection of study materials specific to individual needs, a
series of videotape pretests was developed. The
tests were problem-oriented and designed to
moti-vate the learner to use project materials. To
eval-uate learner knowledge of perinatal topics, a corn-prehensive series of written tests was prepared and
administered after the dissemination of the pro-gram. Items for both tests (videotape and written) were developed from the learning objectives of the media-based units and reflected the content of the units. Because the videotape pretest was designed primarily as a motivational, self-prescriptive test, and the format differed from the written test, no attempt was made to use comparison of scores on pretests and posttests to determine learning gain.
Dissemination of Program
The dissemination model was designed to be applicable to all areas of the United States and utilize state and regional resources for perinatal education. The unit for dissemination and study
was the regional perinatal center and referring
hos-pitals. Each selected regional perinatal center had previously received center designation through the
organized system established by the Department of
Public Health of the State of Michigan. In this
study, the Michigan Department of Public Health,
Bureau of Personal and Child Health, distributed the materials to regional perinatal centers and to the participating referring hospitals. Distribution through the department of health was chosen for the following reasons: (1) the model, if successful, would be reproducible in any state and was not dependent upon the presence of a medical school with continuing medical and nursing education
pro-grams; (2) the state department of public health is
responsible for designation of the perinatal referral
system, and the department is involved in the
qua!-ity of care and professional education; (3) the state department of public health in most states is the licensing authority for obstetric units and is
influ-ential in determining professional fee structures,
and its offerings (the educational program) should, therefore, be attentively received.
The physician-directors of the regional neonatal intensive care centers and the principal perinata! nurses were asked to assume responsibility for man-aging study within their regions. The physician-nurse team from the regional neonatology center visited the medical and nursing staffs of each refer-ring hospital to explain the education program, to recruit interest in participation, and to initiate or-ganization of the community hospitals to use the program and to participate in the evaluation. The
state department staff (a health educator) encour-aged each participating referring hospital to develop a “local coordinating council” composed of physi-cians, nurses, laboratory technicians, and adminis-trators for the specific purpose of participating in this program. The council disseminated informa-tion, arranged logistical details, and handled oper-ational problems. For many small hospitals, this was the first such organized educational activity for the medical staff, and for all hospitals it was the first cooperative educational effort involving both medical and nursing staffs.
Physicians, nurses, and allied health professionals responsible for perinatal care in the local hospitals completed the video pretests and were given study materials. Each hospital maintained a study area equipped with videocassette and slide-tape play-back equipment and a complete set of project study materials. Individual utilization of study materials was recorded by the hospital coordinators, but ed-ucational needs were individually determined and
no participant was expected or required to study all 38 programs. Continuing education credits were provided to nurses and physicians for study of ma-terials. This served as a major motivational factor and was the primary means of documenting utii-zation. The University of Michigan was not directly identified as responsible for the dissemination pro-cess. Further details of this process and useful strat-egies will be reported separately.
Evaluation Strategy
A field test was designed to evaluate the effec-tiveness of the programs and the dissemination process in both rural and urban settings. The 13 perinatal referral regions in southern Michigan were divided into two groups, one predominantly rural, the other urban. Regions whose centers were affili-ated with a medical school or that were currently conducting an extensive regional education
care practices, and promptness of consultations/ referrals.
Process Measurements
A written cognitive test was given to each indi-vidual participant on completion of the project. This constituted a posttest for the intervention group and a pretest or test of equivalency for the nonintervention or control group. An experimental design with pretest and posttest administration to both groups would have been preferable, but carried
a greater risk of stifling participation.
Changes in practice behavior were determined through retrospective chart audits and prospective recording of consultations and referrals for care. For the chart audit, records of all low birth weight (<2,500 gm) infants and a 20% sample of those weighing more than 2,500 gm were selected during equivalent time periods (January through April) before and after the educational program was insti-tuted and during the same periods in the control region. Record keeping and management practices were incorporated into a set of criteria for care of newborn infants.9 Criteria were developed initially by project staff and then reviewed and endorsed by a panel composed of the three participating
inten-sive care unit directors and a national consultant in
neonatology. Data were abstracted from patient
records and a computer program developed to corn-pare this abstract with the care criteria. This pro-gram lists all management programs encountered in each case and provides a score that expresses the percentage of criteria achieved. (The data forms and computer program are available from the
au-thors at reproduction cost.)
The computer program was validated by process-ing 25 patient records, selected to range from simple to complex. The records were abstracted by a
non-physician evaluator and processed by the computer program. The same records were reviewed by a board-certified neonatologist who compared them to the criteria generated by the pane!. In each case the computer-generated problem list matched that of the reviewer and the extent of compliance with criteria was correctly expressed. Our record abstrac-tor was used throughout this study, but in other
studies we have found a high interrater reliability
of
Promptness of Referral
To determine changes in consultation and refer-ral, a standardized telephone log of all infant con-su!tation/referral requests was kept in each regional center before and after the intervention. Data were collected from regional hospitals receiving the
ed-ucational program (intervention) and from hospi-tals in the same region not having educational
in-put. It was hypothesized that hospitals receiving the educational program would reduce the intervals between birth, recognition of a problem, and seek-ing consultation, whereas other hospitals within the same region would demonstrate no change in these intervals.
RESULTS
Participation and Cognitive Gain
A participant was defined as an individual who was involved in perinatal care and whose use of project materials was documented. No attempt was made to categorize participation according to the
amount of material studied, because not all individ-uals received the same study recommendations from the pretest. Approximately 50% of the esti-mated potential physician population used project materials in the rural region, but there was no measurable participation by physicians in the urban region. Nursing participation in the educational program was more extensive; 91% of registered ii-censed practical nurses responsible for perinatal care in the two regions participated in the study.
Four tests covering obstetric, pediatric, nursing,
and resuscitation care were administered to those providing those types of care. Participants were asked to take only those tests relevant to their practice activities. Physician performance is re-ported as a score representing the percentage of correct answers (Figure). The differences in scores between rural physicians and physicians in the
con-tro! region were statistically significant (P < .001).
The mean scores for nurses on the three relevant tests were significantly higher in the urban and
rural intervention regions than in the control region
(Figure).
Test performance by nursing personnel was also assessed by comparing the scores for community nurses to criterion performance. The criterion
group was the nursing staff of a neonatal intensive care unit who had been provided with project ma-terials. Criterion scores for each of the three nursing tests were set at the 20th percentile of the neonatal intensive care unit staff scores. The criterion was set at this level because the criterion group corn-prised registered nurses only while the intervention group contained practical nurses as well. The per-centage of nurses achieving criterion performance in each intervention region was significantly better
than in the control region (P < .001) with rural
P< Oat control vs P< 001 control vs either rural intervention rural or urban intervention use of project materials and professional educa-tional level were the independent variables that best predicted scores.
Change in Care Practices
Changes in practice behavior were measured by retrospective chart audit. Three of the intervention hospitals refused to release patient data and one control hospital was eliminated because of staff and patient changes resulting from a prolonged strike. The audit was comprised of 390 records from three rural, two urban, and three control hospitals. Audit scores following educational intervention improved in the experimental regions, both rural and urban, and did not change in the control region (Table 1). The audit figures are percentages representing the proportional attainment of ideal care, which would
be 100%. The scores combine subcategories of data recording and management decisions. Scores of 60% are generally considered to represent good care in adult patients.9 Analysis of variance with orthogo-nal contrasts was used to test the differences be-tween rural/urban intervention scores and control scores. The differences were significant in each case (P< .01).
0
0
0
Physicians
COntrOl region rurol intervention : urban intervention
Nurses
Figure. Cognitive test performance for physicians and
nurses. Comparisons are between postmtervention
per-formances in rural and urban regions and performance in
the control (nonintervention) region. No physicians took the cognitive test in the urban region and, thus, this region is not represented.
The improvement in patient care following edu-cational intervention was related to improved col-lection and recording of data and to improved man-agement of patient problems. When the patients
were divided into preterm (<2,500 gm) and term infants (>2,500), improved management of low birth weight infants in both rural and urban regions was evident (Table 2). Although improvement in
data recording for low birth weight infants was not evident in the urban region, it should be noted that data recording scores in urban hospitals were
sig-nificantly higher for the preterm group before in-tervention. To test for the possibility that the dif-ferences in scores were the result of educational intervention rather than baseline differences, prein-tervention and postintervention chart audit scores were correlated with treatment (experimental as-signment, extent of participation within hospitals, and mean number of units studied within
hospi-1J)’2 This analysis supported the hypothesis of an
effective educational intervention.
Promptness of consultation/referral was corn-pared between intervention and nonintervention hospitals in the same region for two intervals, prein-tervention and postintervention (Table 3). The in-tervals were not normally distributed, and a loga-rithmic transformation was used for analysis. The difference between preintervention and postinter-vention referral intervals in the rural intervention hospitals was significant by analysis of variance, as well as by nonparametric analysis of the raw data. In urban hospitals there was a trend to decreased referral interval, but the difference between prein-tervention and postintervention periods was not significant (P = .23).
DISCUSSION
The media-based approach to a perinatal educa-tional program was capable of improving the knowl-edge level and of changing the practice behavior of health professionals, although problems were encountered in the dissemination process. The changes in patient care were related to the content of the educational program, and provider
perform-TABLE 1 . Medical Record Audit of Care Practices in Different Regions Before and
After Intervention
Region Audit Score: % of Optimal Care Differences
Preintervention Postintervention
N % N %
Rural 75 48.8 44 56.7 79*
Urban 70 51.6 51 56.7 5.lf
Control 73 55.4 77 53.3 -2.1
C p<
.oi.
TABLE 2. Recording and Management Audit Scores for Care of Preterm and Term Infants
Region Infants <2,500 gm Infants >2,500 gm
N Record- Manage- Combined N Record- Manage- Combined
ing ment (%) ing ment (%)
(%) (%) (%) (%)
Rural
Preintervention 34 57.0 43.6 50.6 41 50.3 29.0 47.2
Postintervention 16 67.5* 60.8* 63.8* 568 29.5 52.6
Urban
Preintervention 34 64.0 47.8 59.3 36 48.6 18.8 4’L4
Postintervention 25 63.9 62.6* 63.8 26 55.1* 16.9 49.8*
Control
Preintervention 32 62.8 60.9 61.8 41 53.4 33.7 50.3
Postintervention 30 61.6 56.1 58.9 47 51.5 36.3 49.7
Significance
Rural vs control .013 .02 002 .002 83 .03
Urban vs control .78 .02 .11 .003 63 #{163}34
* Preintervention vs po
stinterven tion significance tests are independent t -tests; differences are significant at .02 level or
less.
t
Post intervention vs control tests of significance are orthogonal comparisons with control region.TABLE 3. Mean Time Respiratory Problems*
Between Birth and Consultation for High Risk Infants with
Region Preintervention Postintervention
N Mean Time (hr) N Mean Time (hr)
Rural
Intervention 33 2.78 ± 41 13 0.61 ± 0.6t
Nonintervention 63 2.73 ± 44 36 2.73 ± 3.0
Urban
Intervention 37 3.62 ± 2.7 26 2.65 ± 24
Nonintervention 51 3.62 ± 4.8 26 3.61 ± 5.1
* Values are means ± SD.
tP= .008.
ance was directly related to the extent of program
participation. The results validate these
conclu-sions, although problems in the design and opera-tion of large field studies necessitate that several
caveats be noted. First, it is difficult to “match” control and intervention regions as each region and hospital tends to differ in many respects. There is no easy solution to this problem, but we attempted as close a match as possible. The base line
charac-teristics of providers in the intervention and control
regions were similar and there were no significant
changes in audited performance in the control
re-gion during the period of study. Rigorous pretesting was avoided because of concern that it would dis-courage potential learners. Therefore, the gains in knowledge could not be assessed, and cognitive gain comparisons were made between intervention and control regions. Despite these design constraints, the data support the hypothesis that the program led to cognitive gains and these were, in turn, as-sociated with improved care by providers.
The use of self-instructional materials in audio-visual and print formats, rather than lectures and
workshops, provided efficiency of faculty time and flexibility for the learners, many of whom had
corn-plex and demanding schedules. Self-instructional
packages have been commonly used in undergrad-uate and medical education, but their effectiveness has not been evaluated extensively for continuing education audiences, despite a high degree of ac-ceptance by this group.’3 Abrahamson and co-work-ers’4 found that physicians using slide-tape pro-grams scored significantly higher than a control group on a cognitive examination of the material. The widespread use of home study courses and purchase of audiovisual materials indicates
accept-ance by practicing health professionals, but there
have been doubts that acceptance and utilization actually lead to changes in practice behavior.’5 Therefore, this project has broader implications for medical educators, although the subject matter was limited to perinatal care. This program affords the first demonstration that media-based seif-instruc-tional materials can increase knowledge and im-prove practice behavior.
edu-cational program, but this is especially true for practicing professionals who often have loose
in-structional affiliations and select educational inputs
based on personal interests and relevance to needs.’6 We sought an approach to diffusion of the program that would be generally applicable in other states and would not depend on the personal in-volvement of a university or professional group which might not have the interest or available manpower. The state department of public health
was selected as a mechanism for distribution of
educational materials for health professionals be-cause it was postulated that the department would have strong influence on the utilization of perinatal educational materials. The state departments of public health have direct involvement in the re-gional process for this aspect of health care and
have a direct mandate to monitor and support the
care given to newborn infants. An additional
con-sideration was the expectation that the educational
program could be disseminated in other states in a similar manner and that utilization of the program would not be dependent upon the presence of major
medical centers or continuing education units with
strong outreach capabilities.
This hierarchy of state and regional authorities
was chosen for dissemination, but the functional
integrity of the system was untested prior to this
project. The assumption that designed referral re-gions were, in fact, the functional relationships was
tested in the course of the project and found to be
true only in the rural regions. Referral logs and subsequent discussions with neonatologists indi-cated that in this large, urban area the designated
regions were often not the functional referral units.
The neonatologists in the rural regions had
educa-tional and consultative relationships with the
refer-ring hospitals in their assigned regions, but the
urban neonatologists did not have’ relationships
that followed their assigned regions. This had a
striking impact on physician use of the learning materials, but appeared to have little influence on
utilization by nursing personnel. Physicians in the rural region had a reasonable participation rate of
53%, but in the urban region there was no physician participation. Using a slightly different approach to influencing perinatal care, others have found phy-sician participation to be a critical element.7 This
experience emphasizes the importance of a linking
process for physicians. The role of “change agent”
can be filled by the regional neonatologist, but only
if his/her professional activities bring him/her into
functional contact with the learners. This need for
a resource person has been described in the
theo-retical construct of educational dissemination, but
its practical importance in physician
continu-ing education is forcefully portrayed in these
stud-17,18
Dissemination of information to nurses is less problematic. Nurses have a single hospital affflia-tion, are provided work time for education, and effective and appropriate educational leaders are available in the hospital and in the regional pen-natal center. In contrast to the irregular physician utilization, nursing personnel had a high use rate (92%), and there was a commensurate gain in
knowledge. Overall, nursing participation was clearly related to improved care practices. This was exemplified by the results in the urban region, where only nursing personnel utilized the programs. Despite this fact, there was improvement in care practices, indicating the important impact that nursing education can have on care in a hospital setting.
A unique aspect of this program was the demon-stration that cognitive gains were paralleled by improved patient care practices. Chart audit before
and after the educational intervention and compar-isons of intervention and nonintervention regions demonstrated significant differences in the process of medical care. This is an important extension of the assessment of education impact. In this field
test, data recording and patient managment were
improved. Recording of data was significantly irn-proved for all infants in the rural region and for term infants (>2,500 gm) in the urban region. Greater gains were not made in the urban region because performance was higher in this region to begin with and was, in fact, similar to that in the rural region after the program. Patient management was significantly improved in both regions for
pre-term (<2,500 gm) but not for term infants. Because
the greatest potential for complications exists in the preterm group, this change is perhaps most note-worthy. The improved management in the rural
region can be attributed to physician and nurse education, but the changes in management in the urban region are less easily explained. No urban physicians recorded participation in the program, and one conventionally assumes that physicians are the primary decision-makers. However, in nurser-ies, nursing personnel often have an important role
in patient management through highlighting prob-lems and suggesting solutions, as well as use of protocols for initial management of urgent situa-tions. These mechanisms may explain the changes observed.
hospi-tals in the same region demonstrated a significantly
shorter time between birth and consultation/refer-ral in hospitals receiving the educational message.
Regional referral and consultation is the backbone
of efforts to improve neonatal care, but efforts at
shortening the interval between problem
recogni-tion and consultation have not been carefully
stud-ied previously. Overall, the educational program
significantly improved three important aspects of neonatal care-data recording, management
deci-sions, and referral.
This project was directed toward perinatal edu-cation, but the outcome deserves discussion in the broader context of continuing professional educa-tion and its evaluation. First and foremost, this project demonstrates that self-learning materials in a media-based format can be used effectively in community sites, and faculty involvement is not required. Such programs can improve knowledge and change practice behavior. However, it is equally important to recognize that an effective dissemina-tion process is crucial to success. This link is vital in physician continuing education in which interests vary widely and practice activities occur in several
sites. Often, education concerning the routine but
important aspects of patient care loses out to the lure of new and exciting technological advances. For nursing personnel, the essential educational link is provided by the hospital and the inservice education program. Many physicians remain skep-tical that education can change patient care in a
measurable way.’9 This skepticism is based on a lack of evidence rather than data to the contrary. Only recently have attempts been made to docu-ment changes in care practices following educa-tional intervention. The present studies indicate that improvement in patient care can be docu-mented following a targeted educational program, and this should encourage physicians and educators in the attainment and measurement of this goal.
ACKNOWLEDGMENTS
This work was supported by contract N01-HR-2960
from the Division of Lung Diseases, National Heart,
Lung, and Blood Institute.
James Gnesen, PhD, Barbara Rothfeder, MN, and Ronald Gentile, MPH, were instrumental in planning and developing this evaluation.
REFERENCES
1. Farrell PM, Wood RE: Epidemiology of hyaline membrane
disease in the United States: Analysis of national mortality statistics. Pediatrics 58:167, 1976
2. Swyer PR: The regional organization of special care for the neonate. Pediatr Clin North Am 17:761, 1970
3. Butterfield U: Regionalization for respiratory care. Pediatr Clin North Am 20:449, 1973
4. Cassady G, Setzer E: The impact of perinatal intensive care
on quality of life, in Brown A, Aladgem S (eds): Clinical Perinatology. St Louis, CV Mosby, 1979, p 472
5. Caplan RM: Measuring the effectiveness of continuing med-ical education. J Med Educ 48:1150, 1973
6. Williamson JW, McGuire C: Consecutive case conference:
An educational evaluation. J Med Educ 43:1068, 1968
7. Wirtschafter DD, Sumners J, Colvin E, et al: Continuing
medical education: An assessment of impact upon neonatal
care in community hositals, in Proceedings ofthe Eighteenth Annual Conference on Research in Medical Education.
Washington DC, Association of American Medical Colleges,
1979, pp 252-257
8. Michigan Perinatal Education Program. Ann Arbor, The University of Michigan Media Library, 1978
9. Payne BC, Lyons TF, Dwarshius L, et al: The Quality of
Medical Care: Evaluation and Improvement. Chicago,
Hos-pit.al Research and Education Trust, 1976
10. Fleiss JL: Measuring agreement between two judges on the
presence or absence of a trait. Biometrics 31:651, 1975
11. Landis JR, Koch GG: The measurement of observer
agree-ment for categorical data. Biometrics 33:159, 1977
12. Cook TD, Campbell DT: The Design and Conduct of
Quasi-Experimental and True Experiments in Field Settings.
Chicago, Rand McNally 6 Co, 1976
13. Neu HC, Howrey SP: Testing the physician’s knowledge of
antibiotic use: Self assessment and learning via videotape. N
Engi JMed 293:1291, 1975
14. Abrahamson 5, Caput DZ, Wolfers WR: The use of
self-instructional materials in continuing education, in Ninth Annual Conference on Research in Medical Education.
Washington DC, Association of American Medical Colleges,
1970
15. Butt HR: Self-assessment of medical knowlege programs. N
EngI J Med 291:791, 1974
16. Strom JK, Harlan WR: The dissemination of new medical
information. JAMA 241:2622, 1979
17. Wenrick VW, Mann FC, Morris WC, et al: Informal
educa-tion for practicing physicians. J Med Educ 46:299, 1971
18. Havelock RG: The Change Agent’s Guide to Innovation in
Education. Englewood Cliffs, NJ, Educational Technology
Publications, 1973
19. Cooper JAD: Continuing education: What do we mean by it
