Patient satisfaction with out-of-hours services;
how do GP co-operatives compare with
deputizing and practice-based arrangements?
Cathy Shipman, Fiona Payne, Richard Hooper and Jeremy Dale
Background Although the rapid growth in general practi-tioner (GP) co-operatives has met with GP satisfaction, little is known about patient satisfaction. This study compares patient satisfaction with co-operative, GP practice-based and deputizing arrangements within one geographical area 15 months after a co-operative had become established; and with telephone, primary care centre and home consultations within the co-operative.
Methods A validated postal questionnaire survey of weighted samples of patients making contact with the co-operative, practice-based and deputizing arrangements was undertaken.
Results A total of 1823 (53.2 per cent) patients responded. There were no significant differences between organizations in terms of overall satisfaction, but patients using practice-based arrangements were significantly more satisfied with the waiting time for telephone consultations (p < 0.001) and more satisfied with waiting times for home visits than deputizing patients (p = 0.020). Within the co-operative, overall satisfaction, satisfaction with the doctor’s manner and with the process of making contact was greater among those attending the primary care centre, and satisfaction with explanation and advice received greater than for patients receiving telephone consultations alone (p < 0.01). Those receiving telephone advice reported increased information needs and help seeking during the following week (p < 0.05).
Conclusions Overall, patients were as satisfied with the co-operative as with practice-based or deputizing service arrangements, although many concerns were expressed about the quality of service provision. Differences in satisfaction were greater between forms of service delivery within the co-operative. Dissatisfaction with telephone consultations needs to be considered, together with issues relating to equity in access to out-of-hours’ primary care centre consultations and the potential impact of NHS Direct. Keywords: out-of-hours, patient satisfaction, general practice, GP co-operatives
New forms of out-of-hours general practitioner (GP) arrange-ments have encouraged a shift of care from within the home to
primary care centre or telephone consultations.1–3 Although
some evidence suggests that these changes have led to improved
quality of life for GPs,4the impact that this has had on patients
Salisbury found few differences between patient
satisfac-tion with co-operative or deputizing services.5,6Higher levels
of dissatisfaction than previously reported were in part attribu-ted to changing patient expectations, lack of valid comparative questionnaires in previous studies, and the age structure and ethnicity of the study population. Satisfaction was highest amongst patients visited at home and lowest for those receiving telephone advice. However, the findings were compromised because most patients using the deputizing company were resident outside the district served by the co-operative and the study occurred in 1995 before organizations could adapt to new national terms and conditions for out-of-hours services. In
a study of smaller GP co-operatives, Hallam and Henthorne7
found little difference in patient satisfaction between home visits or primary care centre consultations, although less satis-faction was reported with telephone consultations.
Patients have indicated greater satisfaction with
practice-based rather than deputizing services,8 but direct comparison
between practice-based and co-operative arrangements has been lacking.
The present study aimed to compare patient satisfaction with a GP co-operative, a deputizing service and GP practice-based arrangements in one geographical area and to compare satisfaction with the different forms of service delivery pro-vided by the co-operative. We replicated the methods used by
Salisbury, to allow comparison.5
qFaculty of Public Health Medicine 2000 Printed in Great Britain
King’s Out of Hours Project, Department of General Practice and Primary Care, Guy’s, King’s and St Thomas’ School of Medicine, Bessemer Road, London SE5 9PJ.
Cathy Shipman, Research Fellow Fiona Payne, Research Officer
Department of Public Health and Epidemiology, Guy’s, King’s and St Thomas’ School of Medicine, (Bessemer Road), London SE5 9PJ.
Richard Hooper, Lecturer in Medical Statistics
Primary Care Unit, University of Warwick, Coventry CV4 7AL.
Jeremy Dale, Professor of Primary Care
The study was conducted in an inner London Health Authority with a socially deprived population of approximately 730 000. The co-operative had been established for 15 months and had a membership of 290 GPs. In addition, 110 GPs used practice-based arrangements (rota or own on call) at varying times, but also directed calls to a deputizing doctor service. Patient calls to practice-based arrangements were almost all received by the deputizing service, which provided an answering service. Only about 5 per cent of GPs used neither
the co-operative nor the deputizing answering service.4
We used a validated questionnaire developed by McKinley
et al.,9 and subsequently refined by Salisbury.5 It comprises an overall measure of satisfaction with questions grouped into sub-components shown in Tables 2 and 3 (below).
Questions were scored on a five-point Likert scale, with one indicating very dissatisfied and five very satisfied. Socio-economic data (age, sex and postcode) were gained both from information received by the co-operative and deputizing service and from the questionnaires.
Patients were sampled from contacts with the GP co-operative and deputizing company between 21 April and 25 May 1997 (Table 1).
Certain patients were excluded to avoid potential distress to the very ill (Appendix 1); we also excluded those who made a second contact during the study period, patients whose contact was through a third party (except for children), and temporary residents.
Sampling was stratified according to organization and, within the co-operative, the form of service delivery. Under each of the five headings in Table 1 a different sampling proportion was used, to achieve roughly equal sample sizes based on predicted contact rates, subject to an anticipated non-response rate of 50 per cent. For a comparison of any two
categories, 250 patients in each would give 80 per cent power at the 5 per cent significance level to detect a difference of quarter of a scale point, assuming a standard deviation of one
scale point (as found by Salisbury).5
Names and contact information were obtained from the co-operative and deputizing service, and code numbers were allocated to each sampled contact. Each sampled patient received a questionnaire 1 week after their out-of-hours con-sultation, and two reminders were sent at weekly intervals.
Satisfaction sub-scales were calculated from questionnaire responses using Salisbury’s scoring system (personal commu-nication). Whereas the individual rating scores comprised Likert satisfaction scores, aggregation into sub-components of satisfaction allowed parametric analysis. Multiple regression analyses were undertaken on the overall satisfaction sub-scale to investigate differences between organizations and, for data from the co-operative, between types of consultation after allowing for the effects of potentially confounding socio-demographic variables. When comparing organizations, data on contacts with the co-operative were weighted to adjust for the stratification by form of service delivery. (Each form of service delivery at the co-operative was given a weight pro-portional to the total number of contacts of that form, divided by the number of such contacts actually sampled, with the average weight across all contacts for which data was available fixed at 1.0.) Post hoc comparisons of sample means were
undertaken using Tukey’s HSD test.10
In all, 1288 (54.4 per cent) patients responded from the co-operative, 302 (47.8 per cent) from the deputizing service
Table 1 Patient contacts and response rates by service arrangements
. . . .
Base Home Telephone Total Deputizing GP own
Contacts 993 1434 3977 6404 1067 997 (15.5%) (22.4%) (62.1%) (100%) Excluded 128 358 663 1149 218 228* Eligible 865 1076 3314 5255 846 769 Sample 763 648 957 2368 632 427 Responded 444 331 513 1288 302† 233‡ Response rate (%) 58.2 51.1 53.6 54.4 47.8 54.6
*Exclusion criteria are presented in Appendix 1. †Telephone, 121; home visit, 179; base, 4. ‡Telephone, 111; home visit, 122.
and 233 (54.1 per cent) from GPs providing practice-based arrangements (Table 1).
More patients aged over 65 (66.5 per cent, 181/272) responded than younger patients (52.6 per cent, 1628/3094)
(x2¼18.49, df = 1, p < 0.001), and more men (57.4 per cent,
763/1329) responded than women (53.1 per cent, 1050/1976)
(x2= 5.69, df = 1, p = 0.017).
A greater percentage of recorded deputizing contacts (41.0 per cent, 117/285) had been made during night-time hours (22.00–07.59 h) than for practice-based (13.9 per cent, 26/187) or co-operative (27.4 per cent, 345/1261) organizations
(x2= 43.053, df = 2, p < 0.001).
Overall levels of satisfaction did not differ by organization, nor did satisfaction with explanation or advice, doctor’s manner or the receptionist who took the initial call details (Table 2). There were significant differences between the organizations with the wait for a home visit (F = 3.96; df = 2.589; p = 0.020) and satisfaction with practice-based arrangements was signi-ficantly greater than with the deputizing service (post hoc comparisons using Tukey’s HSD tests). Significant differences were found with the wait for a telephone consultation (F = 11.7,
df = 2.688; p < 0.001) when satisfaction with practice-based arrangements was significantly greater than with either the co-operative or deputizing service (post hoc comparisons using Tukey’s HSD tests).
Patients using the co-operative were more satisfied with attendance at the primary care centre (base) than with telephone consultations for all satisfaction sub-scales ( p < 0.01). Satisfaction with attendance at the primary care centre was also significantly greater than with receiving a home visit for overall satisfaction, doctor’s manner and the process of contacting the service ( p < 0.01) (see Table 3).
Multiple regression analysis showed that differences between organizations remained non-significant after adjust-ing for variables that were strongly predictive of satisfaction (Table 4) (parsimonious regression model for overall satisfac-tion resulting from backward eliminasatisfac-tion of variables, selecting from: age and sex, whether patient was child or subject, wanting a home visit, telephone or primary care centre consultation, wanting a prescription; access to public transport, well enough to travel, recovery following consultation, receiving medica-tion or prescripmedica-tion, or referred to hospital). For co-operative patients, differences between the three forms of service delivery remained significant (Table 4).
Fewer primary care centre attenders lived more than 6 km
Table 2 Mean satisfaction scores, with 95 per cent CIs, for the three organizations
Co-operative Deputizing Practice based
Process of making contact with the service 3.53 (3.43–3.64)* 3.30 (3.20–3.41) 3.36 (3.24–3.48)
Receptionist 3.57 (3.46–3.67)* 3.55 (3.45–3.65) 3.49 (3.38–3.61)
Wait for phone consultation 3.08 (2.98–3.18)† 2.77 (2.50–3.04)‡ 3.52 (3.33–3.71)‡
Acceptability of phone consultation 3.22 (3.13–3.31)† 3.23 (3.03–3.43)‡ 3.33 (3.15–3.51)‡
Wait for base attendance 3.76 (3.68–3.84)† – –
Acceptability of base attendance 3.29 (3.21–3.37)† – –
Wait for home visit 2.72 (2.61–2.83)† 2.53 (2.39–2.67)§ 2.86 (2.62–3.10)§
Acceptability of home visit 3.39 (3.28–3.50)† 3.39 (3.25–3.53)§ 3.50 (3.20–3.80)§
Doctor’s manner 3.77 (3.65–3.88)* 3.68 (3.55–3.80) 3.81 (3.67–3.95)
Explanation or advice 3.66 (3.52–3.80)* 3.62 (3.51–3.73) 3.70 (3.56–3.83)
Overall satisfaction 3.26 (3.16–3.36)* 3.17 (3.05–3.28) 3.30 (3.16–3.43)
*Weighted to correct for the different sampling proportions used for the three forms of service delivery. †For those patients who received the relevant form of service delivery.
‡For those patients known to have received a telephone consultation, or who completed the relevant items on the questionnaire but did not complete the home visit items.
§For those patients known to have received a home visit, or who completed the relevant items on the questionnaire.
Table 3 Mean satisfaction scores, with 95 per cent CIs, for the three forms of service delivery at the co-operative
Base Home Telephone
Overall satisfaction 3.65 (3.56–3.74) 3.29 (3.18–3.40) 3.15 (3.05–3.24)
Explanation or advice 3.85 (3.77–3.93) 3.70 (3.59–3.81) 3.57 (3.46–3.68)
Doctor’s manner 4.04 (3.96–4.12) 3.73 (3.62–3..84) 3.71 (3.62–3.80)
Process of making contact with service 3.72 (3.65–3.80) 3.51 (3.43–3.60) 3.49 (3.42–3.57)
from the primary care centre (6.4 per cent) than those receiving home (13.4 per cent) or telephone (11.4 per cent) consultations
(x2 = 11.58, df = 2, p = 0.003). In the multiple regression
analysis, however, distance from primary care centre did not emerge as a predictor of satisfaction.
Outcomes of contacts
More patients with an overall satisfaction score of less than three (neutral) reported going to an A&E department during the following week than did those who indicated greater
Table 4 Results of multiple regression analysis of overall satisfaction
Variable Regression coefficient SE p
Age in years 0.0082 0.0011 <0.001
Received a prescription 0.33 0.05 <0.001
Felt well enough to travel 0.34 0.06 <0.001
Wanted a home visit ¹0.45 0.06 <0.001
Wanted to be seen at the centre ¹0.23 0.10 0.025
Patient’s current perceived condition: <0.001
same 0.00 – worse ¹0.50 0.20 recovered 0.61 0.09 improved 0.35 0.09 don’t know 0.31 0.14 Organization: 0.951 co-operative 0.00 – deputizing ¹0.016 0.072 own GP 0.016 0.085 Co-operative only Age in years 0.0083 0.0013 <0.001 Received a prescription 0.11 0.06 0.072
Able to use public transport ¹0.081 0.063 0.201
Felt well enough to travel 0.42 0.06 <0.001
Wanted a home visit ¹0.47 0.06 <0.001
Patient’s current perceived condition <0.001
same 0.00 –
worse ¹0.63 0.21
recovered 0.55 0.10
improved 0.27 0.10
don’t know 0.32 0.16
Form of service delivery <0.001
base 0.00 –
telephone ¹0.43 0.08
home ¹0.09 0.08
Table 5 Outcomes of out-of-hours contacts for patients using the co-operative
Telephone advice Home visit Base attendance
(%) (%) (%) x2(df¼2) p
Referred to hospital 12.4 8.6 7.0 8.36 0.015
Prescription 17.6 52.5 63.8 226.8 <0.001
During next week
Visit A&E department 13.4 11.5 8.2 6.34 0.042
Called ‘on-call’ doctor 11.6 10.7 6.6 7.23 0.027
Talked to GP at practice 49.5 45.3 35.1 20.2 <0.001
levels of satisfaction (18.5 per cent compared with 8.5 per cent;
x2= 33.2, df = 1, p < 0.001).
Fewer primary care centre attenders reported attendance at A&E department or referral to hospital during the following week than all other patients (Table 5). More primary care centre attenders said that they had improved or recovered, and fewer reported consulting with a doctor at their surgery during the week following their out-of-hours contact. However, more also said that they felt well enough to travel at the time of contact (66.4 per cent) than those receiving home (11.5 per cent) or
telephone (44.8 per cent) consultations (x2= 228.74, df = 2,
p < 0.001), suggesting that they constituted a less ill sample
This study found overall patient satisfaction comparable for GP co-operative, practice-based or deputizing arrangements. However, patients using practice-based arrangements were more satisfied with the waiting times for telephone consulta-tions and home visits, perhaps because practice-based GPs were responding to fewer patients. Co-operative users were more satisfied with the process for making contact with the service,
contrasting with McKinley et al.’s findings8 but comparable
Those attending the primary care centre reported greater
satisfaction than found previously.5This may reflect changing
public expectations and operational differences between co-operatives. Salisbury reported more (32 per cent) home visits and fewer (7.1 per cent) primary care centre and
tele-phone (57.8 per cent) consultations than we found.6[Salisbury
also included 999 or ambulance (0.2 per cent) and patients not contacted (2.4 per cent).] Attending the primary care centre appears to provide an increasingly acceptable form of face-to-face contact. However, primary care centre attenders reported less difficulty in travelling because of illness, less subsequent need for hospital care and increased rates of recovery. They may, therefore, reflect a less ill sample of people than other patients.
Primary care centre attenders tended to live slightly closer to the centre and most (95 per cent) reported arriving by car. This raises questions about equity in access to services, parti-cularly in inner city areas where car ownership and access to private transport can be limited. At the time of the study patients living at the perimeters of the district needed to travel up to 12 km to attend the centre.
Telephone consultations appear to be meeting with greater dissatisfaction than other types of consultation. Difficulty in contacting the service and expecting a home visit were asso-ciated with increased use of the A&E department, suggesting that the increase in telephone consultations may be having an impact on other providers.
These results contrast with the initial findings from the
evaluation of NHS Direct, the 24 hour nurse-led telephone advice line that is currently being piloted in England. The preliminary evaluation of the first three pilot sites suggested high general patient satisfaction with little impact to date on other service providers, although it is too soon to draw firm
conclusions.11 Whether this reflects differences between
the populations using these services and those calling out-of-hours services, a ‘novelty factor’, or differences between the organizational and skill mix arrangements between
NHS Direct and out-of-hours services is unknown.12In many
areas NHS Direct is already becoming the first point of out-of-hours contact, and it will be interesting to see whether this coincides with increasing levels of dissatisfaction emerging.
Our results may not be generalizable to smaller co-operatives, to co-operatives serving other populations, or to co-operatives with different organizational arrangements, but may reflect experiences within other inner city areas. Although we sought to control for population characteristics by undertaking the study within a single geographic area, there were significant difference in distributions between the times of calls received by the three types of organizations, which may have influenced satisfaction levels.
The response rates were lower than those obtained using
almost identical technique in Salisbury’s study.5This reflects
the difficulties of inner-city working and raises the possibility of under-representation of some patient views, particularly given a large minority ethnic population (with language differ-ences) within the district.
This study indicates that within an inner city area, care provided by a co-operative achieves slightly higher levels of accept-ability to patients than do deputizing arrangements. There was little evidence of greater acceptability for practice-based arrangements. Patients appear to prefer face-to-face contact (whether attendance at a primary care centre or receiving a home visit), and failure to provide this may result in increased demand being made on A&E departments.
Little is known as yet about the reasons for dissatisfaction with telephone consultations and whether this reflects unmet expectations, GP consultation skills, delays in calling patients
back, or other organizational issues.13 Strategies to improve
the quality of telephone consultations appear necessary,13
together with operational developments such as providing patient transport to primary care centres, if the quality of out-of-hours services is to be further improved. NHS Direct may have an important role to play through influencing the public’s expectations and increasing the acceptability of telephone advice.
This study was undertaken as part of an Out of Hours Project. We thank Dr C. Salisbury for allowing us to use his adapted questionnaire and scoring system, and Dr R. McKinley for his original questionnaire. We are grateful to the Co-operative and Deputising Service for participation and support provided, and to the patients who responded to the questionnaire. We are also grateful to Lynda Jessopp and members of the Out of Hours team for their support and encouragement, and to the research support staff who worked on the study. The Out of Hours Project is funded by the District Health Authority.
1 Neuberger J. Primary care: core values patients’ priorities. Br Med J 1998; 317: 260–262.
2 Jessopp L, Beck I, Hollins L, Shipman C, Reynolds M, Dale J. Changing the pattern of out of hours care: a survey of GP co-operatives.
Br Med J 1997; 314: 199–200.
3 Payne F, Jessopp L, Dale J. Second national survey of GP
co-operatives: a report. London: Department of General Practice and
Primary Care, King’s College School of Medicine and Dentistry, 1997. 4 Shipman C, Dale J, Payne F, Jessop L. GP’s views about out of hours
working (Letter). Br J Gen Pract 1997(December); 47: 838–839.
5 Salisbury C. Postal survey of patient satisfaction with a general practice out of hours co-operative. Br Med J 1997; 314: 1594–1598. 6 Salisbury C. Observational study of a general practice out of hours
co-operative: measures of activity. Br Med J 1997; 314: 182–186. 7 Hallam L, Henthorne K. Co-operative and primary care emergency
centres: organisation and impact. Manchester: National Primary Care
Research and Development Centre, 1998.
8 McKinley RK, Cragg DK, Hastings AM, et al. Comparison of out of hours care provided by patients’ own GPs and commercial deputising services; a randomised controlled trial. II: The outcome of care. Br Med
J 1997; 314: 190–193.
9 McKinley JK, Manku-Scott T, Hastings AM, French DP, Baker R. Reliability and validity of a new measure of patient satisfaction with out of hours primary medical care in the United Kingdom: development of a patient questionnaire. Br Med J 1997; 314: 193–198.
10 Tukey JW. Exploratory data analysis. Reading, MA: Addison Wesley, 1977.
11 Munro J, Nicholl J, O’Cathain, A, Knowles E. Evaluation of NHS
Direct first wave sites. First interim report to the Department of Health.
Sheffield: Medical Care Research Unit, University of Sheffield, 1998. 12 Florin D, Rosen R. Evaluating NHS Direct. Br Med J 1999; 319: 5–6. 13 Foster J, Jessopp L. Telephone consultation course – an evaluation
report. London: Department of General Practice and Primary Care,
King’s College School Medicine and Dentistry, 1997.
Accepted on 14 September 1999
Appendix 1: Exclusions
Co-operative Deputizing service GP Own on call
Suspected miscarriage 137 16 27
Suspected sexually transmitted disease 4 4
Deliberate self harm 9 3 6
Psychological crisis 176 27 31
Contraception 61 6 2
Palliative care or death 94 31 49
Very elderly or ill 140 33 27
Admitted to hospital 195 16
Dementia 12 8 9
Contact by carer or other 140 29 11
Insufficient details 0 10 20
Very ill, uncertain circumstances 87 15 28
Temporary resident 18 1 1
Domestic violence 8 3 2
Recontact 68 20 11