Development and Evaluation of a CD-ROM Computer Program to Teach Residents Telephone Management

Development and Evaluation of a CD-ROM Computer Program to Teach

Residents Telephone Management

Mary C. Ottolini, MD, MPH, and Larrie Greenberg, MD

ABSTRACT. Objective. Under managed care,

tele-phone management is crucial to pediatric practice, but an effective method is needed to teach residents telephone skills. Our objective was to design an interactive CD-ROM program to teach residents an organized, consistent approach to telephone complaints and to determine whether use of the program was associated with better subsequent telephone management than reading the same information.

Setting. The general pediatric ambulatory center of a tertiary care children’s hospital.

Participants. A total of 24 PL-2 and PL-3 pediatric residents.

Design. A randomized, prospective, controlled com-parison was conducted of resident management of two telephone calls: a 5-year-old with cough and trouble breathing, and a 7-year-old with fever. Thirteen residents were randomized to the computer group and 11 to the reading control group.

Intervention. Scripts, scoring, and feedback for 10 CD-ROM-simulated calls were developed from texts and pediatrician survey using a modified Delphi technique. Volunteers acted out the caller’s role in scenario scripts and were recorded onto a CD-ROM. The computer sim-ulated calls by recognizing questions typed in a free-form free-format and answering with a voice response. Feed-back was provided for omissions in history-taking and errors in assessment, triage, and home management. The computer group worked through the CD-ROM calls while the control group had equal time to read the same information.

Evaluation Measures. A trained, standardized patient acted as the mother in pretest calls placed at the begin-ning of the month and posttest calls at the end. Calls were recorded and scored in a blinded manner using scoring templates and on interpersonal skills using the Patient Perception Questionnaire.

Results. Pretest scores for the two calls were similar in the computer versus the control group (cough, 70.33%

68.36 vs 68.46%66.73; fever, 75.64%69.82 vs 73.59%6 9.06). Posttest scores were significantly higher in the computer group than in the control group on both calls (cough, 79.08%68.17 vs 69613.3; fever: 83.33%69.96 vs 70.35% 6 9.66). Interpersonal skills also were similar pretest (1963.4 vs 2062.7). There was modest

improve-ment in both groups without a statistically significant difference in posttest scores (24.262.9 vs 22.563.1).

Conclusions. Use of this CD-ROM telephone man-agement program was associated with better postinter-vention telephone management. The program augments faculty instruction by teaching a consistent, general ap-proach to telephone management.Pediatrics1998;101(3). URL: http://www.pediatrics.org/cgi/content/full/101/3/e2;

telephone management, resident education, computer-aided instruction.

ABBREVIATION. PL, postgraduate level.

T

elephone management traditionally has been

an integral component of pediatric practice.

Today, as primary care physicians serve as

gatekeepers for health care services under managed

care, effective telephone management is crucial.

De-spite the significance of telephone encounters in

pe-diatric practice, fewer than half of pepe-diatric

resi-dency programs provide telephone training.

_The

pediatric telephone call is especially challenging

be-cause visual assessment of a child’s appearance and

interaction with the environment are not available to

gauge the severity of illness. Failure to ask

appropri-ate questions to obtain essential information without

visual cues can lead to inappropriate assessment

and management,

_{with potentially life-threatening}

sequelae.

Few telephone instructional programs have

re-ported effectiveness in teaching residents telephone

management skills. Working with an expert in

soft-ware design, we developed and evaluated a

CD-ROM interactive telephone management program to

teach residents telephone skills. The CD-ROM allows

the computer to simulate calls and provide pertinent

feedback. Our hypothesis was that guided practice

with feedback using CD-ROM cases would enable

residents to obtain information, assess patients, and

provide advice over the telephone significantly

bet-ter than those who learned by reading similar

mate-rial.

METHODS CD-ROM Development

Scripts for CD-ROM cases representing the 10 most common telephone complaints6 _{were written to simulate common}

tele-phone conversations with parents. The cases are listed in Table 1. To determine the questions that should be asked and feedback that should be provided for the chief complaints depicted in the CD-ROM cases, the historical information essential to forming an appropriate diagnosis and management plan was extracted from

From the Department of General Pediatrics at Children’s National Medical Center and George Washington University School of Medicine, Washing-ton, DC.

This work received the Ray E. Helfer Award for Innovation in Pediatric Medical Education and was presented at the Ambulatory Pediatric Associ-ation Presidential Plenary Session, May 5, 1997, Washington, DC. Received for publication Jul 28, 1997; accepted Nov 7, 1997.

articles, standard pediatrics texts, and telephone advice texts. Questions were then formulated to elicit the essential information, and management options were listed. The lists of questions and management options for each of the 10 scenarios were presented to focus groups consisting of pediatric emergency medicine and general pediatric faculty, pediatric residents, telephone advice nurses, and community pediatricians. Members of the focus groups were asked to score questions as essential, helpful, or unnecessary, and to determine criteria for appropriate triage and home management advice. Responses were analyzed using a Delphi technique.7_{There was very strong agreement among}

mem-bers of the focus groups regarding which questions they rated as essential and which management options they felt were appropri-ate.

In the introductory section of the program, residents were encouraged to use a consistent, organized format for taking a history over the telephone and to learn to use the parent’s descrip-tion of the child to gauge the severity of the complaint. Data-gathering was divided into nine general categories: obtaining identifying information; clarifying the chief complaint with an open-ended question; determining the severity and duration of the chief complaint; ascertaining the child’s overall appearance; previous home therapy; associated symptoms; illness exposure; medical history; and acknowledging the emotional tone of the caller.

Residents began by identifying themselves and obtaining es-sential identifying information about the caller such as parent and child’s names, child’s age, name of primary care provider, tele-phone number, and chief complaint. They were expected to follow up with an open-ended question to learn what was most worri-some about the chief complaint. An open-ended question is em-phasized because Korsch concluded in her classic communication research that failure to address the parent’s main worry resulted in dissatisfaction with the doctor–patient encounter.8 _Residents

then asked directly about the severity and duration of the chief complaint, if these were not evident in response to the open-ended question.

Residents often have difficulty assessing how sick the child is over the telephone, because they have been trained to use visual cues such as the child’s environmental interaction almost intu-itively to assess the overall state of health. Therefore, they were directed to ask questions specifically about the child’s overall appearance and activity level. Previous home therapy was as-sessed to judge the level of sophistication of the caller and to gain information about previous medical advice and current manage-ment. Residents were directed to always ask about medical his-tory, including medication use, whereas a brief review of systems or determination of illness exposure was helpful for some com-plaints.

Residents also were instructed to acknowledge the emotional tone of the caller to calm or reassure parents and to establish rapport. To allow residents to respond to angry, frustrated, and frightened callers, volunteers acted out the caller’s role in scripts written for each of the scenarios and were recorded onto a CD-ROM. Residents and staff were asked how they would phrase individual questions so that different ways of phrasing key ques-tions could be programmed, enabling the computer to recognize questions typed in a free-form format and to answer with a voice response.

This program was designed to encourage a consistent, orga-nized approach to the presenting problem. Data-gathering was divided into general categories as described, followed by the assessment and triage, and concluding with call back and home management advice. Feedback was provided for omissions in

history-taking, and errors in assessment, triage, and home man-agement. The entire program can be read from the computer monitor, with the introduction and interactive data-gathering por-tions having a voice component as well.

CD-ROM Evaluation

To test our hypothesis that use of the program would result in better subsequent telephone management than reading the same information, we conducted a randomized, controlled comparison of the pre- and posttest scores of 24 second- and third-year pedi-atric residents during their month-long ambulatory rotation on two telephone scenarios placed by a trained, standardized patient. The study was conducted over a 9-month period from June 1996 until March 1997.

All second- and third-year residents were asked to participate if they did not have vacation or other scheduling conflicts during their ambulatory rotation. No residents refused. Residents con-sented to allow some of their calls to be tape-recorded, but were not told which calls were from the standardized patient.

Both groups of residents attended a small group session cov-ering the expectations and important aspects of telephone man-agement (Fig 1). Next, each resident spent a morning with the telephone advice nurse, first listening to her answer calls and then answering some calls themselves under supervision of the nurse. Practical aspects such as protocol use, documentation, and inter-personal aspects of telephone management were emphasized dur-ing this one-to-one session. The pretest calls were placed by the standardized patient during this session.

Eleven residents then were randomly assigned to the reading control group, and 13 to the computer group. Control residents were given copies of advice protocols and readings covering the same content as in computer cases. Both groups had two 90-minute sessions to complete the cases or the readings. To ensure that there was no contamination between groups, residents in the reading group were told not to share their readings with those in the computer group, and individual access codes necessary to use access the computer program were given only to those in the computer group.

After completing the readings or the cases, the residents spent two half-day sessions independently answering telephone calls from parents of patients followed in the General Pediatric Ambu-latory Center. The two posttest telephone calls were placed at this time. All were free to refer to advice protocols with pre- and posttest calls.

Both groups received the same two pre- and posttest telephone calls: a 5-year-old with cough and trouble breathing and a 7-year-old with fever. Calls were recorded and scored using standard templates (Table 2) and on interpersonal skills using the Patient Perception Questionnaire9_{(Table 3) by the standardized patient}

who was blinded to the group assignment.

The standard scoring templates for the test cases were devel-oped from the consensus of a survey of 100 pediatricians in the Washington, DC area. Sixty percent of those surveyed responded.

TABLE 1. CD-ROM Telephone Scenarios 5-month-old with trouble breathing

Toddler with vomiting and diarrhea

3-week-old infant with persistent crying and a frustrated mother Anxious mother of a child with fever

Angry father of a 3-year-old with ear pain 6-year-old with abdominal pain

Child with a rash

2-year-old with an accidental ingestion Infant with head trauma

Adolescent with a sore throat

TABLE 2. Test Script: Cough in a 5-Year-Old Child

Resident No: Date of Call: Pretest Posttest Identifying information

2 Dr: Hello, this is Dr from Children’s. How can I help? M: My son is having trouble breathing.

2 Dr: What is your son’s name? 2 Dr: How old is he?

Open-ended question

2 Dr: What worries you most about the trouble breathing?/What kind of trouble breathing?/How much trouble breathing? Severity

Any questions about the severity of cough score as a total of 2 points. Dr: Does the cough wake him up?

Dr: Does the cough seem to be getting worse? Dr: What makes the cough worse?

2 Dr: Is he wheezing?/Making any noises/sounds with breathing? 2 Dr: Is he breathing fast?

2 Dr: Do his ribs suck in when he breaths (retractions)? 2 Dr: Is his nose running?/Nose congested?

Duration

2 Dr: How long has he been sick? Activity

2 Dr: Does he have a fever?

2 Dr: What is his activity level?/How is he acting?/Is he acting sick?/Is he home from school? 2 Dr: Is he drinking/eating OK?

Previous therapy

2 Dr: What have you tried at home? What have you given him?/Have you given him any medicine? Illness exposure

1 Dr: Has he been around anyone sick? ROS

2 Dr: Is anything else bothering him?/Any other symptoms?/Does anything hurt? PMH

2 Dr: Does he have any medical problems?/Has he had asthma/bronchitis/pneumonia?/Has he been sick like this before?/ Has he been healthy?

1 Dr: Is he taking any medicine? (score as 0 if already asked under Previous Therapy) 6 Assessment: It sounds like your son has a cold (URI, virus).

4 Triage: Manage at home. (The resident may offer for the child to be seen, but should not insist or send the patient to the emergency room.)

Call back if (score as 2 points total for this section if any of the following are asked): Persistent symptoms: the cough lasts.1 week.

Changing symptoms: he begins vomiting or running a fever.

Worsening symptoms: the cough is interfering with usual activity or he develops difficulty breathing. Anxiety provoking symptoms: there is something about the way he is acting that worries you. Home therapy (score 2 points total for this section if any of the following are recommended):

Vaporizer Increase fluids

Over-the-counter medicine containing dextromethorphan/pseudoephedrine at bedtime. Elevate head

Other

2 Any other questions? Do you understand?

7-Year-Old With Fever (Anxious Mother) Resident No: Date of call:

Pretest Posttest Identifying information

2 Dr: Hello, I’m Dr from CNMC. How can I help? M: My daughter has a fever.

2 Dr: What is your daughter’s name? 2 Dr: How old is she?

Open-ended question

2 Dr: What concerns/worries you most about her right now? Severity

2 Dr: How high is the fever? Duration

2 Dr: How long has she had a fever? Overall appearance

2 Dr: How is she acting now?/Is she acting normal?/What is her activity level?/Did she go to school? 2 Dr: Is she drinking/eating OK?

Previous therapy

2 Dr: What have you done/given her for the fever? Illness exposure

1 Dr: Has she been around anyone sick? ROS

2 Dr: Are there any other symptoms?

Score 1 for any of the following for a maximum score of 2 points Dr: Sore throat?

Pediatricians were presented with a chief complaint and age of the child and asked to rate a series of questions or recommendations as essential, helpful, or unnecessary.

The standardized caller had previous training as a standard-ized patient. She had an additional 10 hours of training for this study, memorizing the scripts, and in role-play sessions with the investigators to ensure that her answers were correct and consis-tent. Interrater reliability was established at .91 by one of the investigators who listened independently to tapes and scored recorded calls blinded to resident group assignment and to the standardized patient assessment.

RESULTS

Statistical analysis was performed using

x

_{or the}

Mann–Whitney test on Minitab statistical software

(Table 4).

The number of third year residents (PL-3) and the

pretest scores was similar. Posttest scores were

sta-tistically significantly higher in the computer group

than in the reading group on both calls. Some

post-test scores were lower than prepost-test scores in both

groups. All residents reviewed advice protocols with

nurses before pretest calls, and residents in the

read-ing group were provided copies of protocols as part

of their readings. Few residents in the reading group

and no residents in the computer group referred to

protocols when taking the posttest calls.

Interpersonal skills improved modestly in both

groups on the Patient Perception Questionnaire. The

computer group scored higher posttest, but the

dif-ference was not statistically significant.

In an anonymous written evaluation of the

tele-phone training, all of the residents rated the

experi-ence positively. All of the residents in the computer

group rated the program as helpful or very helpful.

A total of 73% of the reading group rated the

pro-gram helpful or very helpful, with the remainder

giving neutral ratings. Residents commented that the

computer program was “fun,” “good cases, pretty

typical,” “nonthreatening learning situation,” “it was

pretty close to being real,” “gave extra time to think,”

“pointed out important questions I should have

asked/advice I should have given,” “easy and fun,”

and “helped me organize my thoughts; very good

learning tool.” The primary criticism was that the

computer did not recognize all ways of phrasing a

question and did not have answers to all questions.

The primary criticism from residents in the reading

group was that they wanted to do the computer

cases.

DISCUSSION

Residents must learn telephone skills because

tele-phone management is an essential part of pediatric

practice. Although institutions such as our own have

implemented telephone advice programs staffed by

nurses using protocols, physicians are ultimately

re-sponsible. Many physicians in smaller communities

still answer the majority of calls, especially

after-hours. Senior residents have the knowledge and skill

necessary to correctly assess and manage most

pa-tients encountered in the clinic or emergency

depart-ment. However, they often fail to gather enough

pertinent information to assess and manage the same

complaints over the telephone. Lack of a consistent,

organized approach to telephone management has

been a problem identified in previous studies.

We emphasized teaching a consistent, general format

that could be applied to a variety of complaints.

Before the development of this computer program,

our residents had little formal training in telephone

management. Fosarelli found that program size and

manpower were predictive of whether pediatric

res-idency programs provided telephone training.

Large residency programs were less likely to provide

telephone training, and reported instruction was

usually by lecturing. Although lecturing is an

inef-fectual means of teaching an interactive skill, it is the

method most commonly reported because it appears

to be an efficient way to deliver the content.

Increas-TABLE 2. Continued

7-Year-Old With Fever (Anxious Mother) Resident No: Date of call:

Pretest Posttest Identifying information

Dr: Abdominal pain?/Vomiting/Diarrhea? Dr: Is she coughing/Having any trouble breathing? Dr: Does she have a rash?

PMH

2 Dr: Does she have any medical problems?

1 Dr: Does she take any medicine? (If they asked about any medicine earlier, score as 0). Assessment

6 Your daughter sounds like she has a mild illness, like a virus that is not serious. Triage

4 Manage at home. (The resident may offer for the child to be seen, but should not insist on it, nor should the resident refer them to the emergency room.)

Call back if (any of the following score as 2 points) Persistent fever.3 days.

Changing symptoms: localizing symptoms like cough, pain, vomiting, or diarrhea. Worsening symptoms: fever.105°F; lethargy.

Anxiety-provoking symptoms: acting in such a way that worries the parent. Home management (score as 2 points if they recommend any of the following)

Reassure the mother that fever is not dangerous. Febrile seizures are unlikely to recur after 6 years of age, especially because she has not had a fever in 6 years.

Continue Tylenol 15 mg/kg q 4–6 hr prn, or Ibuprofen 10 mg/kg q 6 hr. (Both are not needed.) Encourage the child to continue to drink plenty of fluids.

ing demands on faculty to see more patients and

precept trainees in the clinic have limited the time

available to precept telephone calls.

Although interactive programs have been

devel-oped, there is only modest evidence that they are

more effective at teaching telephone management

than a traditional didactic approach. An innovative

program described by Kosower at UCLA called

T.A.L.K. taught telephone communication skills by

allowing residents to analyze recorded calls in group

and individual feedback sessions.

_{Evans used}

pa-tient simulators to teach telephone communication

skills to family practice residents.

_{Although both}

programs were well received by residents, neither

provided an objective evaluation of their

effective-ness at improving skills.

Several researchers did evaluate the effectiveness

of their programs using standardized patients. Curry

and Schwartz studied the effectiveness of a small

group discussion regarding important questions to

be asked for a complaint of vomiting and diarrhea

using five residents in the discussion group and five

in a control group.

_{There was no statistical}

differ-ence in posttest scores between the groups at either 6

days or 28 weeks. Wood found that role-play

ses-sions in which residents simulated parent callers

helped to improve some aspects of telephone

his-tory-taking in a controlled trial.

_{Smith studied the}

effect of feedback given to 12 medical students for a

single call on their performance on a subsequent call

using a standard scoring template. He found that

their proficiency improved significantly between the

first and second calls, but there was no control

group.

We hypothesized that to become proficient at

tele-phone management, skill-modeling using an

orga-nized approach and guided practice was necessary.

The computer program was modeled on previous

studies using standardized patients as simulated

callers.

_{Use of standardized patients both as an}

evaluation and a teaching tool has been well

docu-mented.

_{Standardized patients offer several}

ad-vantages to an educational program: 1) the learner

can be evaluated and/or provided feedback in a

controlled situation; 2) use of standardized patients

allows for more objective assessment because criteria

used to judge each learner’s performance can be

tailored and validated against preset standards; 3)

specific simulated encounters can be used that the

learner may not experience by chance during

train-ing or are of such a sensitive nature that it would be

unethical to allow an inexperienced learner to

inter-act with inter-actual patients/parents; 4) standardized

pa-tients can be instructed to provide feedback to the

learner in place of the faculty; 5) students’ and

resi-dents’ history-taking and physical skills can be

eval-uated better because they are assessed directly.

Although use of standardized patients encourages

active learning, it is a costly educational intervention

for a large residency program. Therefore we used the

computer to simulate a series of standardized

pa-tients. Computers have been used successfully to

teach assessment and management skills in other

areas of medical education.

_{Using a CD-ROM}

program, the computer can simulate a telephone call,

with the resident asking questions and receiving

an-swers without visual cues and responding not only

TABLE 3. Patient Perception Questionnaire

Poor Fair Good Very Good Excellent

Greeting you warmly: being friendly, crabby or rude

Treating you like you’re on the same level; never talking down to you or treating you like a child

Letting you tell your story, listening carefully, asking thoughtful questions, not

interrupting while you are talking Showing interest in you as a person; not

acting bored or ignoring what you have to say.

Encouraging you to ask questions, answering them clearly, never avoiding your questions or lecturing you.

Using words you can understand when explaining your problems; explaining any technical medical terms in plain language. Would you want this physician for your own

child’s doctor

Definitely no Probably no Maybe Probably yes Definitely yes

TABLE 4. Results

Reading Group Mean6SD

Computer Group Mean6SD

PValue

Total number 11 13

PL-2 residents 6 6 .68

PL-3 residents 5 7

Cough

Pretest 68.466.73% 70.3368.36% .55 Posttest 69.0613.3% 79.0868.17% .03 Change (Post–Pre) 0.6614% 8.569.9% .15 Fever

Pretest 73.5969.06% 75.6469.82% .45 Posttest 70.3569.66% 83.3369.96% .004 Change (Post–Pre) 24.1613.2% 7.7616.2% .046 Patient perception scale

Pretest 2062.7 1963.4 .77

Posttest 22.563.1 24.262.9 .41

to the content of the complaint, but also to the

emo-tional tone of the voice. The residents can participate

in self-directed guided practice sessions with the

computer to become proficient in the general

ap-proach to telephone management.

Use of this CD-ROM telephone management

pro-gram was associated with significantly higher

post-test scores on telephone calls placed by a

standard-ized patient in a randomstandard-ized, controlled trial. We feel

this program is most helpful in teaching a consistent,

general approach to telephone management,

includ-ing history-takinclud-ing, providinclud-ing an assessment, triage,

call back, and home-management advice. The

CD-ROM program augments, but does not replace,

fac-ulty involvement. Facfac-ulty can help residents to refine

interpersonal skills once residents have gained

knowledge and practice covering general content

areas.

The limitations of this study are the small sample

size and lack of long-term follow-up. To maintain

proficiency at any skill, continued practice and

feed-back are needed over time. We are currently working

to incorporate an integrated telephone management

experience that is not limited to 1 month, but is

ongoing throughout the residency program. In

addi-tion, although the program was very well received,

some residents in both groups scored lower on the

posttest than on the pretest calls. Unanticipated

achievement declines have been reported by

Swan-son and others on performance-based evaluations.

Residents may have been more highly motivated to

perform well initially, because pretest calls were

taken while reviewing common protocols with the

advice nurse present; however posttest calls were

taken by residents unobserved and unaware of

which calls were recorded. Second- and third-year

residents in our program already were generally

pro-ficient at managing telephone calls, with pretest

scores of 68% to 75% of ideal. More significant gains

may have been seen if first-year residents

partici-pated.

The major costs in developing this program was

for faculty time to develop, record, score, and

pro-vide feedback for the cases, and for the computer

programmer to write the program. Based on the

success of this study, our goal is to share this

pro-gram with other institutions seeking to refine

resi-dent telephone management skills. We also plan to

make use of the computer case-based, interactive

format to develop self-directed learning modules

covering other areas of primary care pediatrics to

help our institution maintain its commitment to

teaching while facing conflicting demands on faculty

time attributable to changes in the health care

envi-ronment.

ACKNOWLEDGMENTS

This work was supported by Bayer Institute for Health Care Communication grants for the development and evaluation of this program.

We thank Ren Lan Loai and Dr Pincetl from the Computer Informatics Department at George Washington University for their help with program design. We also thank the nursing staff and pediatrics faculty at Holy Cross Hospital and Children’s National Medical Center who contributed to the CD-ROM devel-opment and evaluation.

REFERENCES

1. Bergman AB, Dassel SW, Wedgewood RJ. Time-motion study of prac-ticing pediatricians.Pediatrics.1966;38:254 –263

2. Hessel SJ, Haggery RJ. General pediatrics: a study of practice in the mid-1960’s.J Pediatr.1968;73:271–279

3. Wood PR. Pediatric resident training in telephone management: a sur-vey of training programs In the United States. Pediatrics. 1986;77: 822–925

4. Yanovski SZ, Yanovski JA, Malley JD, et al. Telephone triage by primary care physicians.Pediatrics.1992;89:701–706

5. Isaacman DJ, Verdile VP, Kohen FP, et al. Pediatric telephone advice in the emergency department: results of a mock scenario.Pediatrics.1992; 35–39

6. Poole SR, Schmitt BD, Carruth T, Peterson-Smith A, Slusarski M. After-hours telephone coverage: the application of an area-wide telephone triage and advice system for pediatric practices. Pediatrics.1993;92: 670 – 679

7. Milholland AV, Wheeler SG, Heleck JJ. Medical assessment by a Delphi group opinion technique.N Engl J Med.1973;288:1272–1275

8. Korsch BM, Gozzi EK, Francis F. Gaps in doctor-patient communica-tion.Pediatrics.1968;42:855– 871

9. Schnable GK, Hassard TH, Kopelow ML. The assessment of interper-sonal skills using standardized patients.Acad Med.1991;66:534 –536 10. Wood PR, Littlefield JH, Foulds DM. Telephone management curricula

for pediatric interns: a controlled trial.Pediatrics1989;83:925–930 11. Brown SB, Eberle BJ. Use of the telephone by pediatric house staff: a

technique for pediatrics care not taught.J Pediatr.1974;84:117–119 12. Perrin EC, Goodman HC. Telephone management in acute pediatric

illness.N Engl J Med.1978;298:130 –135

13. Greitzer L, Sapleton FB, Wright L, Wedgewood RJ. Telephone assess-ment of illness by practicing pediatricians.Pediatrics.1976;88:880 – 882 14. Fosarelli PD. The emphasis of telephone medicine in pediatric training

programs.Am J Dis Child.1985;138:555–557

15. Kosower E, Inkelis SH, Seidel JS. Telephone T. A. L. K.: a telephone communication program.Pediatr Emerg Care.1991;2:76 –79

16. Evans S, Curtis P. Using patient simulators to teach telephone commu-nication skills.J Med Educ.1983;58:894 – 898

17. Curry TA, Schwartz MW. Telephone assessment of illness: what is being taught and learned?Pediatrics.1978;62:603– 605

18. Smith RS, Fischer PM. Patient management by telephone: a training exercise for medical students.J Fam Pract.1980;10:463– 466

19. Kleflher JH.The Teaching Learning Process. San Antonio, TX: The Uni-versity of Texas Health Science Center at San Antonio; 1989:1–52 20. Dunn EV, Norton PG, Dunn RC. Using simulated patients to teach

family practice residents to manage patients by phone. J Med Educ.

1987;82:524 –526

21. Barrows HS. A review of the uses of standardized patients for teaching and evaluating clinical skills.Acad Med.1993;68:443– 451

22. Anderson MB, Stillman PL, Wang Y. Growing use of standardized patients in teaching and evaluation in medical education.Teach Learn Med.1994;6:15–22

23. Scheidt PC, Lightsey KH, McDaniel D, Labow JC. Evaluation of com-puterized simulations in a pediatric clerkship. Teach Learning Med.

1991;3:108 –111

24. Cohen PA, Dacanay LS. Computer-based instruction and health profes-sions education: a meta-analysis of outcomes.Eval Health Prof.1992;15: 259 –281

25. Lynn HC Jr, Healy JC, Bell JR, et al. Plan Alyzer, an interactive com-puter-assisted program to teach clinical problem solving in diagnosing anemia and coronary artery disease.Acad Med.1992;67:821– 888 26.Enhancing Medical Students Assessing Change in Medical Education. The

Road to Implementation (ACME-TRl Report).Acad Med.1993;68(suppl): S1– 46

DOI: 10.1542/peds.101.3.e2

1998;101;e2

Pediatrics

Mary C. Ottolini and Larrie Greenberg

Residents Telephone Management

Development and Evaluation of a CD-ROM Computer Program to Teach

Services

Updated Information &

http://pediatrics.aappublications.org/content/101/3/e2

including high resolution figures, can be found at:

References

http://pediatrics.aappublications.org/content/101/3/e2#BIBL

This article cites 24 articles, 7 of which you can access for free at:

Subspecialty Collections

_management_sub

http://www.aappublications.org/cgi/collection/administration:practice

Administration/Practice Management

ev_sub

http://www.aappublications.org/cgi/collection/teaching_curriculum_d

Teaching/Curriculum Development

b

http://www.aappublications.org/cgi/collection/medical_education_su

Medical Education

following collection(s):

This article, along with others on similar topics, appears in the

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtml

in its entirety can be found online at:

Information about reproducing this article in parts (figures, tables) or

Reprints

http://www.aappublications.org/site/misc/reprints.xhtml

DOI: 10.1542/peds.101.3.e2

1998;101;e2

Pediatrics

Mary C. Ottolini and Larrie Greenberg

Residents Telephone Management

Development and Evaluation of a CD-ROM Computer Program to Teach

http://pediatrics.aappublications.org/content/101/3/e2

located on the World Wide Web at:

The online version of this article, along with updated information and services, is

by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397.