Effectiveness
of Computer-Generated
Appointment
Reminders
Thomas
G. Quattlebaum,
MD; Paul M. Darden,
MD; and
John
B. Sperry,
MD
From the Department of Pediatrics, The Medical University of South Carolina, Charleston
ABSTRACT. Previous studies of appointment reminders among general pediatric patients have been done exclu-sively among low socioeconomic populations in clinics
with low continuity of care and using block scheduling methods. This study of mailed computer-generated
ap-pointment reminders took place in a setting with patient demographics and practice techniques similar to those of
many private pediatric practices. During a 6-month period, 901 appointments that were made more than 7 days prior to the scheduled date were randomly assigned to receive reminder postcards or to serve as controls. The
overall broken appointment (no-show) rate was reduced
from 19% in the control group to 10% in the reminder group, representing a 48% reduction (P = .0002). The
magnitude of reduction of no-shows was similar for
ap-pointments scheduled more than 14 days prior to the
appointment time (39%) as well as those scheduled more
recently (58%). Equivalent results were seen with
well-child appointments (47%) as with other visits (50%). Both lower and higher socioeconomic groups demon-strated similar results. The mailed postcards cost $0.20
each and were highly cost effective, generating an esti-mated $7.50 for each $1 spent during the study. With
reductions similar to those found in this study, computer-generated appointment reminders are likely to be cost effective in other practices if current no-show rates are greater than 2% to 4%. Pediatrics 1991;88:801-805; pe-diatric practice, appointments.
Broken appointments interfere with patients’ medical care and waste health professionals’ time. Many possible factors that might influence ap-pointment-keeping behavior have been studied, and
Received for publication Jun 4, 1990; accepted Nov 26, 1990. Presented, in part, before a joint session of the American Pedi-atric Society, the Society for Pediatric Research, and the
Am-bulatory Pediatric Association, Anaheim, CA, May 10, 1990.
Reprint requests to (T.G.Q.) 198 Rutledge Aye, Charleston, SC 29403.
PEDIATRICS (ISSN 0031 4005). Copyright ) 1991 by the American Academy of Pediatrics.
several have been found to be very important. Low socioeconomic status has been demonstrated to be a major determinant of broken appointments in
most studies.”2 The patient-physician relationship is an important factor, and a number of studies
have shown that providing continuity of care ne-duces the likelihood of broken appointments.”3’4
Scheduling methods influence patient waiting
time, satisfaction, and the frequency of broken
ap-pointments. Block scheduling, modified block scheduling, and individual scheduling are the three major scheduling techniques used in medical
set-tings.5 In the block scheduling method all morning
or afternoon patients are scheduled for one appoint-ment time, resulting in long waits for patients. The modified block (or “wave”) technique breaks the day into smaller blocks, usually hours, but still leads to substantial waiting for patients in each wave. The use of individually scheduled appointments rather than block scheduling has been shown to lead to important reductions in broken
appoint-ment (no-show) rates.6
Reminders provided near the time of appoint-ments have been shown in several settings to be
effective in reducing the number of broken appoint-ments.7’#{176} Both telephone and mailed reminders
can be useful, but mailed reminders are cheaper and more likely to be cost effective.9
very different populations, continuity, and sched-uling systems.
The use of mailed computer-generated appoint-ment reminders was studied in a setting with pa-tient demographics and practice techniques similar to those of many private pediatric practices. The purpose of this study was to examine the reductions in no-show rates in this population and the cost effectiveness of these reminders.
METHODS
Practice
Setting
The University Pediatric Group of the Medical
University of South Carolina is a primary care
practice consisting of three full-time faculty attend-ing physicians, two behavioral science faculty, a nurse practitioner, and 24 residents. Each faculty member is scheduled to see patients each day during either a morning or afternoon session. Residents spend part of every week in the practice seeing their own panel of patients and are in the office from 1 to 3 half-days each week, depending on their rota-tion for that month. An average of 20% of each house officer’s time over 3 years is scheduled in the practice. One or two of the attending physicians, the nurse practitioner, and four or five residents
see patients during each half-day session.
Behav-ioral science faculty see patients 5 half-days during
the week.
Scheduling
System
Individualized patient appointments are sched-uled by computer. The time allotted for each patient visit is determined automatically according to the type of problems the patient brings to the office as well as the differing time requirements of faculty physicians and residents at various levels of train-ing.’5”6 The schedule is structured so that the length of an appointment is determined by a
pa-tient’s presenting problem, with more complex
problems allotted longer appointments. The length of time scheduled also reflects the skills and profi-ciency of the physician seeing the patient. Patients scheduled for attending physicians and upper-level residents are given shorter appointments than
those seeing a less experienced resident. This
tech-nique leads to shorter waiting times for patients and more efficient use ofphysicians and office staff.
No overbooking of appointments is used to
corn-pensate for expected no-show patients, and the practice’s staffing is designed to provide efficient care for all of the scheduled patients.
Appointment
Reminders
The scheduling system permits the automated printing of reminder postcards each week for
mail-ing to patients who are scheduled for the following
week. During a 6-month period from mid-March to mid-September 1989, patients with appointments made more than 7 days before the scheduled visit were assigned randomly to receive a postcard ap-pointment reminder (the “reminder” group) on to act as a member of a control group who did not receive reminders. To ensure that all members in the same household would be in the same group, assignment was based on the terminal digit of the patient’s family account number.
Postcards were printed once each week for
pa-tients in the reminder group who had appointments scheduled the following week. These cards were mailed so that they would arrive at least 3 days before the scheduled appointment. Fewer than 1% of the reminders were returned undelivered because of address problems.
Each card was addressed to the patient’s parent or guardian and noted the name of the doctor and the date, day of week, and time of the appointment.
If a parent called before the date of an appoint-ment to cancel or reschedule, office personnel made the appropriate changes in the computer-main-tamed schedule. A list of patients scheduled for the
day’s sessions was printed daily for use by the
nursing and business office staff. If a parent called before the appointment time to reschedule, the appointment was marked as “canceled.”
Appoint-ments not kept were noted as “broken.”
Patient
Population
and Continuity
Data
Demographics of patients in the study were de-termined from data maintained in the practice’s computer system. This system provides patient and physician scheduling, management of the residents’ outpatient experiences, and patient billing.’7”8 De-mographic and visit data were updated at each patient visit. The reason for each patient’s visit was determined from the scheduling system.
Socioeconomic status was determined from fi-nancial classification data. Patients who were clas-sified as medically indigent because of low family income and those eligible for medical assistance (Medicaid) were considered to be of lower socioeco-nomic status.
Continuity calculations were performed by deter-mining the percentage of time patients were seen by their assigned physician.19 Visits for laboratory
tests, immunizations, etc, not involving contact
Income and Cost Estimates
Patient
Demographics
Because of the lack of overbooking of
appoint-ments to compensate for expected broken
appoint-ments, an administrative analysis prior to the study estimated that the number of patient visits could increase by 10% to 15% without increasing over-head expenses on salaries. It was then assumed that
these costs would not change if the broken
appoint-ment rate were reduced.
The income generated by each patient visit
con-vented from a no-show to a kept appointment was
estimated by calculating the average fee collected per outpatient visit during the study.
Cost calculations were based on the cost of cards,
staff time involved with printing and handling the
reminders, and postage. The cost of computer hand-ware was excluded.
Statistical
Analysis
Data were tested using
x2
analysis. Statistical significance was set at the .05 level.RESULTS
During the 6-month period, 901 appointments were included in the study. Of these, 701 were kept, 122 were broken, and 78 were canceled in advance. The cancellation rates of patients who did not receive reminders (9.2%) and those who did (8.0%) were not significantly different. Reminders were mailed for 391 of the 823 noncanceled
appoint-ments in the study. No reminders were mailed for
the remaining 432.
Sixty-nine percent ofthe appointments were with patients of higher socioeconomic status. There was no significant difference in age distribution, race, sex, or socioeconomic status between the reminder and control groups.
The no-show rate among patients of lower socio-economic status in the control group (25%) was
significantly greater than that of higher income
patients (17%, P < .05).
Comparison of the subgroups of patients sched-uled for attending and resident physicians revealed a significantly greater proportion of patients in lower socioeconomic status for residents (49.7%) than for attending physicians (9.1%, P < .0001).
Continuity
Patients included in the study were seen by their
assigned physicians 96% of the time, with patients
of faculty having a 99% continuity rate and those
of residents, 94%.
Reductions
in No-Show
Rates
The Table shows the reductions in no-show rates achieved with reminder cards. The overall no-show rate was reduced from 19% in the control group to 10% in the reminder group (P = 0002). This
de-crease represents a 48% drop in the no-show rate. Similar and significant reductions occurred with
both well-child visits (47%) and other visits (50%). The magnitude of reduction of no-shows was simi-lan for both “long interval” appointments scheduled more than 14 days prior to the appointment time (39%) and “short interval” appointments scheduled more recently (58%).
TABLE. Reductions in No-Show Rate s With Appointment Reminders4
Category No Reminder Rem inder Sent Reduction in No-Show
te % No. No-Show
Rate, %
No. No-Show
Rate, %
All visits 432 19 391 10 48t
Well-child appointments 298 16 273 9 47t
Other visits 134 25 118 13 SOt
Interval
Long (>14 d) 220 20 196 12 39t
Short (i4 d) 212 18 195 8 58t
Patient
Offaculty 171 10 160 4 56t
Ofresident 213 26 205 15 41t
Socioeconomic status
Low 118 25 100 15 41
Other 314 17 291 8 Sit
4 No-show rate = broken divided by kept plus broken appointments.
8% a) ‘a
0 ‘a 0 z 0
0.
6%
4%
-‘- 20% convrtsd -e- 40% convrtsd
2%
0%
$10 $16 $20 $25 $30 $36 $40
Income Per Visit
Reminders mailed to patients of faculty resulted in similar reductions in no-shows (56%) to those
mailed to patients of residents (41%). The neduc-tion of broken appointments was similar for
pa-tients of higher (51%) and lower socioeconomic status (41%).
During the study 113 patients received reminders for more than one appointment. The cancellation
rate for the first reminder (9.7%) was not signifi-cantly different from that for reminders for
subse-quent appointments (11.1%). There was no signif-icant difference in no-show rates between first (10%) and subsequent appointments (11%).
Cost Analysis
Each appointment reminder was estimated to
cost $0.200, based on paper, personnel time, and postage. During the study period, the practice
col-lected $22.08 pen patient visit.
Comparison of the kept appointment rate in the reminder and control groups allowed an estimate of the increase in patient visits and practice income
resulting from the use of reminders. The difference
between the actual number of kept appointments
in the reminder group (352) and what would have been expected based on the kept rate of the control
group (315.9) represents the increased patient visits
resulting from the use of appointment reminders.
The cost of all cards mailed, including those to
patients who rescheduled well in advance and were
not included in the study, was $106.50, on $2.95 for each increased patient visit. Inasmuch as the prac-tice collected $22.08 for each $295 spent for ne-minder cards, they generated an estimated $7.50
for each $1 spent.
Figure. Prior no-show rates at which the cost of com-puter-generated appointment reminders is offset by in-creased patient income if reminders cost $0.20 each. The
curves represent the break-even points for reminders that
convert either 20% or 40% of no-shows to keep
appoint-ments. With prior no-show rates above the curve, income generated is greater than the cost of reminders.
Although the cost pen card using this system
should be similar in other practices, the income generated for each no-show converted to a kept appointment probably would be different. Break-even points for other possible combinations of gen-enated income, prior broken appointment rate, and
reductions in no-shows were therefore analyzed.
For example, if a practice generated $20 per visit and each reminder cost $0.20, then the cost of the
reminders would be offset by increased income if 1
of 100 cards resulted in a no-show being converted to a kept appointment. If the reminders reduced the no-show rate by 20% from 5 of 100 to 4 of 100, then the income generated and the cost of the reminders would be equal. Reminder cards, there-fore, would break even in this example if the onigi-nal no-show rate were 5%.
The Figure presents a graphic representation of similar calculations and shows the prior no-show
nate required for appointment reminders to break even at various levels of generated income.
DISCUSSION
These results confirm that computer-generated appointment reminders are highly effective in ne-ducing the number of broken appointments in a practice setting with patient demographics and
practice techniques similar to those of many private
pediatric practices.
The improvements in kept appointment rates occurred in all categories of patients and
appoint-ments. Unlike Levy and Claravall,’#{176} who reported
that reminders resulted in significant reduction in no-show rates only for appointments scheduled
more than 14 days prior to the appointment time, significant reductions were found for both “short
interval” and “long interval” appointments.
Patients of low socioeconomic status were found,
as in other studies, to have a significantly higher
no-show rate than higher income patients.
Although the reduction in no-shows among
pa-tients of low socioeconomic status did not reach
significance (P = .058), this fact is believed to be a
reflection of the much smaller number of visits scheduled for this group during the study compared
with patients in higher socioeconomic status. Nearly three times as many visits for patients of higher socioeconimic status were included in the study.
Although Morse et al’2 suggested in a follow-up to Nazanian and coworkers’ study7 that mailed
ne-mindens may lose effectiveness with time, their results should not be extrapolated to other settings.
As noted in their article, the majority of their patients’ appointments were episodic in nature: 64% were made less than 1 week before the sched-uled visit. During the current study period, by con-trast, only 44% of our patients’ appointments were made within a week. Additionally, Morse et al es-timated that only 80% of their reminders were received by their patients, as opposed 99% in the current study.
As Morse et a! also noted, high-density popula-tion areas close to their facility had been removed by urban renewal, and transportation services for their patients had been reduced. Because of these changes, the effort required from patients to keep appointments would have increased, and it is not surprising that no-show rates increased and mailed reminders were less effective. It is noteworthy that reminders remained effective among their patients who had phones. Patients in this group who ne-ceived reminders had a significantly lower broken appointment rate than those who did not.
Very little has been published about no-show rates found in private pediatric practices. More than 25 years ago Alpert1 included a survey of several solo private pediatricians in the Boston area in his study and found a cancellation rate of 15% and a no-show rate of about 4% in his small sample.
No studies have been published about no-show
rates in pediatric group practices.
In estimating the cost of reminders during the study period, personnel costs were calculated based on the salary of the registered nurse who was des-ignated to handle the cards each week. Subsequent to the completion of the study a lower salaried office
assistant assumed this task, and the current cost
per card is $0.170. Calculations based on the present
costs show that approximately $8.80 is generated
for each $1 spent on reminders.
The computer system used for this study
con-sisted of an IBM AT-type personal computer
run-ning patient and physician information and sched-uling software developed by the first author.’5’8 This system is also being used at practice settings in other states. Although more elaborate hardware
setups can be used (in our office multiple terminals
and printers are attached to a 386 AT clone), the
system requires only a personal computer and a
printer to operate effectively. Several of the
prac-tices using this system have found this minimum
configuration to be adequate, enabling them to use computerized patient scheduling with a minimum of hardware expense.
In summary, computer-generated appointment reminders were found to lead to large reductions in no-show rates among all categories of patients. Estimates based on these findings suggest that such reminders would be cost effective in many practices
if they reduce no-show rates by only 15% to 20%
(much less than found in this study) when the prior rate of broken appointments is greater than ap-proximately 4% to 7%. With reductions similar to those found in this study, computer-generated
ap-pointment reminders are likely to be cost effective if current no-show rates are greaten than 2% to 4%.
REFERENCES
1. Alpert JJ. Broken appointments. Pediatrics. 1964;34:127-132
2. Hurtado AV, Greenlick MR, Colombo TJ. Determinations of medical care utilization: failure to keep appointments.
Med Care. 1973;11:189-198
3. Pearce T, O’Shea JS, Wessen AF. Correlations between
appointment keeping and reorganization of hospital ambu-latory pediatric services. Pediatrics. 1979;64:81-87
4. Hershey CO, Cohen DI, Goldberg HI, et al. Effect of an academic group practice on patient show rates: a randomized controlled trial. Med Care. 1987;25:72-77
5. Callahan N, Redmon W. Effects of problem-based schedul-ing on patient waiting and staff utilization of time in a
pediatric clinic. J App Be/isv Anal. 1987;20:193-199
6. Rochart JF, Hoffmann PB. Physicians and patient behavior under different scheduling systems in a hospital outpatient department. Med Care. 1969;7:463-470
7. Nazarian LF, Mechaber J, Charney E, Coulter MP. Effect
of a mailed appointment reminder on appointment keeping.
Pediatrics. 1974;53:349-352
8. Gates SJ, Colborn DK. Lowering appointment failures in a
neighborhood health center. Med Care. 1976;14:263-267
9. Sheppard DS, Moseley TA. Mailed versus telephoned ap-pointment reminders to reduce broken appointments in a hospital outpatient department. Med Care. 1976;14:268-273 10. Levy R, Claravall V. Differential effects of a phone reminder
on appointment keeping for patients with long and short between-visit intervals. Med Care. 1977;15:435-438
1 1. Meller W, Anderson A. The effect ofappointment reminders
on keeping appointments in a core city pediatric outpatient department. Minn Med. 1976;59:625-626,659
12. Morse DL, Coulter MP, Nazarian LF, Napodano Ri. Wan-ing effectiveness of mailed reminders on reducing broken appointments. Pediatrics. 1981;68:846-849
13. Friman P, Glasscock 5, Finney J, Christophersen E.
Reduc-ing effort with reminders and a parking pass to improve appointment keeping for patients ofpediatric residents. Med Care. 1987;25:83-86
14. Gerson L, McCord G, Wiggins S. A strategy to increase appointment keeping in a pediatric clinic. J Community Health. 1986;11:111-121
15. Quattlebaum TG. A multiuser MUMPS language patient! physician scheduling system for microcomputers. Comput Methods Programs Biomed. 1988;27:287-293
16. Quattlebaum TG. Computerized patient scheduling in out-patient residency practices. AJDC. 1989;143:1333-1336
17. Quattlebaum TG. Microcomputer analysis and management of residency training experiences. Comput Methods Pro-gram.s Biomed. 1985;20:169-172
18. Quattlebaum TG. Implementation of multiuser MUMPS language database systems on microcomputers. Comput Methods Programs Biomed. 1988;26:45-52
19. Steinwachs DM. Measuring provider continuity in
1991;88;801
Pediatrics
Thomas G. Quattlebaum, Paul M. Darden and John B. Sperry
Effectiveness of Computer-Generated Appointment Reminders
Services
Updated Information &
http://pediatrics.aappublications.org/content/88/4/801
including high resolution figures, can be found at:
Permissions & Licensing
http://www.aappublications.org/site/misc/Permissions.xhtml
entirety can be found online at:
Information about reproducing this article in parts (figures, tables) or in its
Reprints
1991;88;801
Pediatrics
Thomas G. Quattlebaum, Paul M. Darden and John B. Sperry
Effectiveness of Computer-Generated Appointment Reminders
http://pediatrics.aappublications.org/content/88/4/801
the World Wide Web at:
The online version of this article, along with updated information and services, is located on
