Risk taking in general practice: GP out-of-hours referrals to hospital

Abstract

Background

Emergency admissions to hospital at night and weekends are distressing for patients and disruptive for hospitals. Many of these admissions result from referrals from GP out-of-hours (OOH) providers.

Aim

To compare rates of referral to hospital for doctors working OOH before and after the new general medical services contract was introduced in Bristol in 2005; to explore the attitudes of GPs to referral to hospital OOH; and to develop an understanding of the factors that influence GPs when they refer patients to hospital.

Design of study

Cross-sectional comparison of admission rates; postal survey.

Setting

Three OOH providers in south-west England.

Method

Referral rates were compared for 234 GPs working OOH, and questionnaires explored their attitudes to risk.

Results

There was no change in referral rates after the change in contract or in the greater than fourfold variation between those with the lowest and highest referral rates found previously. Female GPs made fewer home visits and had a higher referral rate for patients seen at home. One-hundred and fifty GPs responded to the survey. Logistic regression of three combined survey risk items, sex, and place of visit showed that GPs with low ‘tolerance of risk’ scores were more likely to be high referrers to hospital (P<0.001).

Conclusion

GPs' threshold of risk is important for explaining variations in referral to hospital.

INTRODUCTION

Admissions to hospital are an increasing source of pressure on the NHS,¹ and unplanned admissions represented 37% of hospital admissions in the UK in 2005–2006.² Emergency admissions at night and weekends are distressing for patients and disruptive for hospitals, and many of these admissions result from referrals from GP out-of-hours (OOH) providers. Research has shown that there is a greater than fourfold variation between the top and bottom quartiles in OOH admission rates between GPs working for the same OOH service and caring for the same patient population,³ suggesting that doctors' decision making varies and plays an important part in determining admission rates. Qualitative research with the same group of OOH GPs suggests this may be due to lack of confidence, aversion to risk, feelings of isolation, and poor knowledge of alternatives to admission. The results suggest that interventions designed to address these problems may have a considerable impact on emergency admission rates.⁴

In 2005, the new GP contract allowed GPs to opt out of OOH care in the evenings, nights, and at weekends, and also made local primary care trusts (rather than GPs) responsible for the provision of OOH services. GPs are also able to choose their shifts and, consequently, where they work (primary care centre or making home visits).

Following the introduction of the new general medical services (GMS) contract in Bristol in January 2005, the aims of the current study were to compare the referral rates before and after the contract change; to investigate the attitudes of GPs to referral to hospital OOH; and to develop an understanding of the factors that influence GPs when they refer patients to hospital. It also aimed to develop an attitude-assessment tool to identify doctors with low and high referral rates to hospital which could be used in GP training sessions.

METHOD

Using statements derived from major themes from the qualitative study,⁴ a questionnaire was designed, piloted with 10 GPs in a neighbouring primary care trust, and modified based on their results. The final questionnaire comprised 41 Likert statements, 36 of which covered seven attitude dimensions of confidence and experience, tolerance of risk and uncertainty, complaints by patients or relatives, attitudes to hospital admission, patient-related factors, motivation, and organisational factors; an eighth dimension on job satisfaction had five additional questions based on previous research (Appendix 1).⁵ Data were also collected about age, years since qualification, type and size of practice, and what type of referrer GPs thought they were. Open-ended questions asked for GPs' opinions about alternatives to hospital which they felt might change their referral rates to hospital OOH and during their daytime work (Appendix 2).

A list of GP names was obtained from the OOH database (Adastra Software Ltd) for the three OOH providers around Bristol (one of which was the basis of the audit in 2004). Referral rates to hospital (OOH) were available for all GPs; however, only those who saw 20 or more face-to-face consultations in a 20-month time frame (1 January 2005 to 31 August 2006) were included in the survey. Questionnaires were sent to all GPs who had a current address on the OOH services databases. Reminder letters and questionnaires were sent after 3 weeks to the non-responders, and a personal telephone call made 2 weeks later by one of the research GPs for the study. Those who returned the questionnaire were paid £20.

Questionnaire data were entered onto an Excel® spreadsheet and combined with the OOH referral rates. Referral rates were expressed as a face-to-face rate and a total rate. The total rate included those contacts made by phone where they received ‘doctor advice’, whereas the face-to-face rate only included home visits and patient attendance at the primary care centre.

How this fits in

Emergency admissions to hospital are an increasing source of pressure on the NHS and many result from referrals from GP out-of-hours (OOH) providers. There is a greater than fourfold variation in OOH admission rates between GPs working for the same OOH services; research suggests that decision making varies and plays an important part in determining admission rates. GPs with a lower tolerance of risk are more likely to be high referrers to hospital. Providing a GP with special interest in primary care OOH service who could access alternatives to admission might enable GPs to alter their referral practices.

Statistical methods

The analysis was carried out in two parts. First, an analysis of all the GPs on the database modelled whether the associations found were similar to those seen before the contract changed.³ For this the GPs were banded into quartiles, based on their admission rates, and logistic regression was used to determine the odds of being admitted to hospital if seen by a GP in the top quartile, compared with that of being seen by a GP in the bottom quartile. Logistic regression was based on patient contacts, and STATA (version 9) accounted for multiple contacts per GP using clustering by GP. Linear regression explored any associations found, including place of visit and sex differences, with their referral rates to hospital. Comparison of referral rates for individual GPs with data available before and after the contract changed was also made using Spearman's rank correlations and Wilcoxon signed rank tests.

The second analysis involved only the GPs who had returned questionnaires. Comparisons for those who did and did not return questionnaires were carried out using χ² test for sex and Mann–Whitney U tests for the number of contacts and referral rates. The correlation between the type of referrer that the GP perceived themselves to be, and their actual referral pattern was examined using Kendall's tau-b.

Factor analysis was used to investigate the 41 attitude statements, to determine whether there were any distinct subscales similar to those used in the design of the questionnaire on attitudes to referral to hospital from OOH providers. Different rotations were considered, including varimax and direct oblimin, to explore the clinical relevance of the subscales. Cronbach's alpha was used to identify potentially redundant items. The statements were also investigated individually to see if any of them were independently associated with referral rates.

Attitude scores were calculated for the eight questionnaire attitude dimensions, by summation of the items from each dimension. Items that were negatively correlated with the others were reversed and Spearman's rank correlation coefficients calculated to examine the association between referral rates and the eight dimensions. Similar correlations were calculated for individual items and referral rates using a square root transformation of referral rates to produce a normal distribution. Linear regression modelling was then carried out using the statistically significant items combined as a risk score with sex and place of visit.

Where the individual items were used, a statistical significance level of 0.01 was used (rather than 0.05) to reduce the chance of a type I error. A formal Bonferroni correction was not used as this was too stringent given the hypothesis-generating nature of the study. For regression modelling, P<0.05 significance was retained.

RESULTS

Data available from the Adastra OOH database for three OOH providers covering the 20-month period since January 2005 included 77 748 patient contacts with 496 OOH workers. After excluding those with no face-to-face contacts recorded (n = 37), those who had fewer than 20 face-to-face contacts (n = 212), those who were not GPs (n = 6), and a further seven duplicate names, 234 names remained. These GPs had 73 453 total patient contacts (n = 51 688 face-to-face), with 6353 admissions to hospital (n = 4913 from face-to-face contacts).

Admission rates

Admission rates were available for 234 GPs who had from 20 to 2580 face-to-face patient consultations. Face-to-face admission rates varied from 0% for a GP who had 26 face-to-face consultations, to 38.5% for a GP who had 91 face-to-face consultations. Referral rates were compared between the highest and lowest quartiles both before and after the change in contract. Table 1 shows that since the change in contract the odds ratio for being admitted to hospital if seen by a GP in the highest quartile was 4.49 (95% confidence interval [CI] = 3.98 to 5.06) compared with being seen by a GP in the lowest quartile, which was similar to odds ratios for the 3 years prior to the change in contract. The referral rates were also very similar, with the overall rates being 9.54% before the change in contract and 9.50% after the contract changed.

Table 1.

Comparison of being admitted to hospital if seen by a GP in each of the rates of admission quartiles from face-to-face contacts before and after introduction of the new general medical services contract.^a

	Referral rate		OR of being admitted		95% CI
Quartile	Before	After	Before	After	Before	After
Lowest	4.1	3.6	1.00	1.00	Reference	Reference
Second	7.4	7.4	1.80	2.04	1.56 to 2.08	1.81 to 2.29
Third	10.9	10.6	2.75	2.93	2.38 to 3.17	2.63 to 3.26
Highest	17.3	16.3	4.56	4.49	3.86 to 5.38	3.98 to 5.06
Overall rate	9.54	9.50

^an = 149 GPs before and n = 234 GPs after. OR = odds ratio.

Referral rates were also compared for 80 individual GPs with data available for both periods of time. There was a significant correlation in their referral rates (Spearman's r = 0.57, P<0.001) before and after the contract changed. Median referral rates for this group of GPs were not significantly different (Wilcoxon P = 0.17) before (9.2%) and after (9.8%) the contact changed. There were no significant differences in the referral rates of those who continued with OOH work and those who did not (Mann–Whitney P = 0.887). Although fewer female GPs continued with the new contract (44% compared to 51% before), the proportion of males to females was not significantly different.

There was a difference in the number of patients seen at home or the primary care centre for male and female doctors, in that females made fewer home visits (16% of contacts) compared with males (21%) and seemed to choose more shifts based at the primary care centre. The distribution of referral rates for male and female doctors by the site of the consultation (primary care centre or home) was also different, as shown in Table 2, being highest for females when visiting patients at home. Linear regression modelling of referral rates controlling for place of visit showed that these sex differences were statistically significant (P<0.001), indicating that female GPs referred more patients to hospital than male GPs.

Table 2.

Referral rate (%) to hospital out of hours by GP sex and place of consultation.

Sex (n)	Referral rate (face-to-face)	Referral rate PCC only	Referral rate home visit only
Male (143)	8.7	7.8	11.9
Female (91)	11.0	9.8	20.0
Overall (234)	9.5	8.7	15.0

PCC = primary care centre.

Postal survey

Of the 234 names, 17 were excluded as there was no current address, and another 23 were only registered with one OOH provider that did not distribute the questionnaires, leaving an eligible total of 194 GPs. Six of these returned their questionnaires indicating that they did not wish to take part, and 38 did not return a questionnaire, resulting in 150 completed questionnaires (77.3% return rate). There were no differences in sex or referral rates between those who did and did not return their questionnaires.

There was a significant correlation (P = 0.002, Kendall's tau-b = 0.21) between what type of referrer (high, medium, low) GPs thought they were and their referral rates, but interestingly only 5% (n = 7) of them felt that they were high referrers. The correlation was low, indicating that there was a general lack of awareness about their own referral rate compared to those of others.

Factor analysis did not produce any clinically relevant subscales even after trying different rotations. The confirmatory factor analysis of the eight attitude dimensions suggested that there might be further subscales present within some of the dimensions, but due to the low number of items within each dimension it was not possible to identify stable subscales within them. Cronbach's alpha did not indicate that the omission of any items would lead to a substantial improvement.

Spearman's rank correlation showed no statistically significant associations between any of the eight dimensions of the attitude survey and referral patterns, as shown in Table 3. A sample of 150 participants has 80% power to detect correlations of 0.23 or greater as statistically significant at the 0.05 level, and none of these correlations was greater than 0.13.

Table 3.

Correlations between questionnaire attitude dimensions and referral rates (n = 150).

Dimension	Face-to-face referral rate	P-value	Total referral rate	P-value
Confidence and experience	0.03	0.73	0.00	0.97
Tolerance of risk and uncertainty	−0.13	0.11	−0.12	0.13
Complaints by patients or relatives	0.13	0.11	0.11	0.17
Attitudes to hospital admission	−0.05	0.52	−0.04	0.59
Patient-related factors	−0.06	0.48	−0.04	0.61
Motivation	−0.01	0.92	0.02	0.82
Organisational factors	−0.11	0.18	−0.10	0.21
Job satisfaction	0.13	0.12	0.12	0.15

Analysis of the 41 individual items produced three items from the tolerance of risk and uncertainty dimension with statistically significant correlations with referral rates as displayed in Table 4. These indicated that GPs who believe that they are cautious have higher referral rates; those who believe they are carrying out risk assessment and those who believe they are good at living with uncertainty and risk both have lower referral rates.

Table 4.

Item-by-item correlation coefficients and P-values^a for the 41 attitude items with referral rates (both total admitted rates and rates for those admitted following face-to-face consultations).

	% admitted total		% admitted face-to-face
Attitude itemb	Spearman's r	P-value	Spearman's r	P-value	n
conexp1	−0.07	0.424	−0.05	0.544	150
conexp2	−0.04	0.590	0.00	0.962	150
conexp3	0.10	0.203	0.12	0.134	149
tru4	0.25	0.002	0.21	0.011	150
tru5	−0.15	0.066	−0.15	0.060	149
tru6	−0.07	0.419	−0.07	0.366	150
tru7	−0.26	0.002	−0.22	0.008	149
tru8	−0.30	0.000	−0.30	0.000	148
tru9	0.06	0.483	0.00	0.979	148
tru10	0.04	0.657	0.01	0.920	148
comp11	0.00	0.990	−0.04	0.630	150
comp12	−0.09	0.271	−0.09	0.258	149
comp13	0.13	0.124	0.12	0.152	150
att14	0.06	0.485	0.05	0.558	150
att15	0.05	0.509	0.05	0.543	149
att16	0.00	0.985	0.03	0.752	150
att17	−0.10	0.236	−0.08	0.349	150
att18	−0.14	0.098	−0.14	0.084	150
prf19	−0.04	0.656	−0.05	0.565	150
prf20	0.01	0.903	0.06	0.495	150
prf21	0.11	0.165	0.19	0.023	149
prf22	−0.12	0.138	−0.13	0.118	150
prf23	−0.12	0.144	−0.17	0.036	150
prf24	−0.07	0.421	−0.05	0.575	150
prf25	−0.02	0.785	−0.04	0.655	150
motiv26	0.16	0.046	0.14	0.094	150
motiv27	0.04	0.654	0.04	0.609	150
motiv28	0.03	0.724	−0.01	0.860	150
motiv29	−0.13	0.110	−0.10	0.208	150
motiv30	−0.08	0.346	−0.10	0.233	150
motiv31	−0.09	0.297	−0.08	0.309	150
org32	−0.05	0.568	−0.05	0.527	149
org33	0.02	0.795	−0.05	0.534	150
org34	−0.14	0.093	−0.14	0.100	149
org35	0.00	0.983	0.04	0.655	150
org36	−0.09	0.286	−0.08	0.322	150
satis37	−0.16	0.052	−0.13	0.126	150
satis38	−0.04	0.638	−0.04	0.634	150
satis39	0.01	0.934	−0.04	0.663	150
satis40	0.01	0.922	0.05	0.523	150
satis41	−0.06	0.466	−0.09	0.264	150

^aItems in bold are those that were significant at the 0.01 level.

^bRefer to Appendix 1 to view attitude items in full.

A risk score was computed by summing these responses (items 4, 7, and 8) related to tolerance of risk and uncertainty. Linear regression modelling transformed the face-to-face referral rate with sex, controlling for place of visit, showed that female doctors were significantly more likely to refer patients to hospital than male doctors (P = 0.039). However, when the risk score was included in the model, the sex difference was no longer significant and the risk score had a stronger association with referral rate (P<0.001), indicating that it is GPs with a lower tolerance of risk that are more likely to admit patients to hospital OOH.

Free-text comments on the questionnaires suggested that GPs might change their referral behaviour if there were better access to intermediate care, more information about patients, and some follow-up or feedback about patients.

DISCUSSION

Summary of main findings

This study has confirmed the authors' previous finding³ with the current group of 234 GPs working for three OOH organisations, that there remains a greater than fourfold variation in referral rates between the highest and lowest quartiles of GPs since January 2005. Referral rates have also not changed since the new GMS contract started. This is remarkable, since the study was comparing three post-contract OOH organisations with one pre-contract one and a different group of GPs, who had all opted to do OOH work. Previous findings from the authors' qualitative study,⁴ which suggested that GP perception of risk had an influence on decisions to refer, were also confirmed. The survey evidence showed that those who are comfortable living with risk are likely to be low referrers to hospital, and this was also evident in the qualitative study where high-referring GPs were typically cautious, believed it was better to refer if in doubt, and expressed anxiety about the consequences of a decision not to admit, for both the patient and themselves.⁴ The authors had believed that the changes in the new contract would attract different types of GPs to OOH for different reasons and motives, and that those who were not comfortable with it would opt out. However, this was not reflected by GPs' referral decisions.

Strengths and limitations of the study

The strength of the study is the high response rate in the survey from those who received it. Thus the results can be considered as representative of the views of GPs working in OOH services. Limitations include the use of only three GP OOH services and the relatively large numbers of items that were used to try to assess whether attitudes were associated with referral behaviour. While a large majority of these attitudinal items were derived directly from the qualitative study, their validity as a measure had not been evaluated in previous research. Also, the small number of items that were statistically significant did not enable the development of a comprehensive assessment training tool as initially intended. However, the analysis did identify three risk items that might be used as an ‘attitude to risk’ tool to predict referral rates.

Comparison with existing literature

This study showed that female GPs refer more patients to hospital OOH, and they seemed to choose shifts based in the primary care centre rather than doing home visits. When this was allowed for in the analysis, GPs with low scores for tolerance of risk were more likely to be high referrers to hospital. This supports research evidence from daytime referral practices,⁶ which suggests GPs' threshold of risk is important for explaining variations in referral. However, males and females work in different ways, with females being more patient centred,⁷ and perhaps more risk averse. On the other hand, male doctors have a higher output than females, but this is offset by males experiencing more litigation and disciplinary action.⁸

Most GPs seemed to know what type of referrer they were, but a few felt that they were lower referrers than was actually seen in their referral rates. They indicated, as was evident in the previous study,⁴ that they might alter their referral practices if reliable and accessible alternatives to hospital were readily available to keep patients at home, while additional training, feedback, and opportunities for a second opinion without referring might also affect some GPs' referral decisions.

Understanding how different doctors make decisions,⁹ and the difficulties they have estimating and managing risks,¹⁰ is important in planning changes in their performance. Different doctors have different ways of managing uncertainty and may be unaware of how they compare with others; poor performers often have misplaced confidence in their performance.¹¹ Giving doctors educational feedback on their performance compared to the optimal can change the clinical performance of some doctors,¹¹ and may have an impact on referral rates OOH.

Implications for future research or clinical practice

Some GPs have suggested that a separate defined OOH specialty with recognised qualifications and bespoke or mandatory training in primary care OOH services might provide a better service.¹² An ‘attitude to risk’ assessment tool could be used in such training.

Further research could explore in greater depth why female GPs may be more risk averse. Another important area to investigate would be how to facilitate access to intermediate care or other alternatives to emergency admission, and feedback to GPs about patient outcomes, to provide a greater insight into what OOH practitioners would value and whether patients' quality of life could be improved.

Appendix 1. Attitudes to referral to hospital from out-of-hours questionnaire.

The following statements, some of which have been made by GPs, represent a range of views about the influences on decisions to refer to hospital or not from out-of-hours (OOH). Please indicate how far you agree with these statements by selecting the appropriate level of agreement.

Confidence and experience	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
I have a continual feeling that I haven't quite done enough. I feel I don't know whether I'm doing the right thing or not. Experience has taught me that being cautious sometimes isn't necessarily a bad thing.
Tolerance of risk and uncertainty	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
4.When it comes to medicine I'm quite cautious. 5.As a GP you think that you can deal with most things most of the time. 6.I think my risk assessment is reasonably good, and I'm reasonably safe. 7.All GPs take risks; it's risk assessment in general practice all the time. 8.GPs are good at living with uncertainty and risk. 9.I don't worry about my decisions after I've made them. 10.I sometimes go back and check on the patient's outcome after a shift has finished.
Complaints by patients or relatives	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
11.I don't worry about a complaint being made about me. 12.I have had complaints made against me. 13.Fear of litigation influences my practice.
Attitudes to hospital admission	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
14.A benefit of admission is that it reduces the risk of being sued. 15.It's easier to send somebody in and get the hospital to check them out than to leave them at home. 16.Admitting someone to hospital enables me to get a second opinion. 17.Admitting someone to hospital means I can speed up urgent tests. 18.Admitting someone to hospital puts them in danger of being ‘over-tested’.
Patient-related factors	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
19.The most important factor in my decision to admit is the patient's clinical status. 20.I am more likely to admit a person if they want to be admitted. 21.If a family wants me to admit their relative I would be more inclined to admit them. 22.I see myself as a negotiator, ‘selling’ my decision (whatever that is) to family and carers. 23.If members of the family say there's nobody to look after someone, I see that as a problem for the family rather than the doctor. 24.I am more likely to admit someone if they live alone. 25.I am more likely to admit someone if they are poorly housed.
Motivation	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
26.I like the nature of the clinical work in OOH care. 27.I work in OOH because I like the financial rewards. 28.I enjoy working with my OOH organisation. 29.I dislike the work in OOH because it is different from routine GP work. 30.I do not enjoy working with the larger network of GPs, nurses, and other staff OOH. 31.I work in OOH because I feel responsible for the quality of care that our patients receive OOH.
Organisational factors	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
32.I think intermediate care is probably much better for the patient if they can have it. 33.I have experienced difficulty arranging intermediate care for someone. 34.Hospital nurses are less likely than hospital doctors to confront you about your decision to admit. 35.I feel guilty when I admit a patient. 36.We are increasingly under pressure not to admit people.
Job satisfaction (in your work as a doctor)	Agree strongly	Agree a little	Neither agree nor disagree	Disagree a little	Disagree strongly
37.I feel that some parts of my work do not make sense. 38.My work still interests me as much as it ever did. 39.Assuming that pay and conditions were similar, I would just as soon do non-medical work. 40.I find enjoyment in my work. 41.My work involves a great deal of wasted effort on my part.

Appendix 2. Background factors questionnaire.

Please complete the following or circle the appropriate alternative.

^aAbbreviations ‘Brisdoc’, ‘Frendoc’, and ‘Nordoc’ are used by three local out-of-hours services covering three areas of the city of Bristol.

Funding body

Avon Primary Care Research Collaborative small grant scheme (05-IPE4)

Ethical approval

Southmead Research Ethics Committee (ref 05/Q2002/62) August 2005

Competing interests

The authors have stated that there are none

Discuss this article

Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss