Association between primary care organisation population size and quality of commissioning in England: an observational study

Abstract

Background

The ideal population size of healthcare commissioning organisations is not known.

Aim

To investigate whether there is a relationship between the size of commissioning organisations and how well they perform on a range of performance measures.

Design and setting

Cross-sectional, observational study of performance in all 152 primary care trusts (PCTs) in England.

Method

Comparison of PCT size against 36 indicators of commissioning performance, including measures of clinical and preventative effectiveness, patient centredness, access, cost, financial ability, and engagement.

Results

Fourteen of the 36 indicators have an unadjusted relationship (P<0.05) with size of the PCT. With 10 indicators, there was increasing quality with larger size. However, when population factors including deprivation, ethnicity, rurality, and age were included in the analysis, there was no relationship between size and performance for any measure.

Conclusion

There is no evidence to suggest that there is an optimum size for PCT performance. Observed variations in PCT performance with size were explained by the characteristics of the populations they served. These findings suggest that configuration of clinical commissioning groups should be geared towards producing organisations that can function effectively across their key responsibilities, rather than being based on the size of their population alone.

INTRODUCTION

The government’s reforms of the NHS in England have been through a number of iterations.^1-3 The latest proposal, after the government’s ‘pause’ and ‘listening exercise’, looks set to develop clinical commissioning groups (CCGs) with responsibility for commissioning local services and controlling budgets. These will combine local GPs with the input of secondary care clinicians and others.³ The size, structure, and exact function of these groups remain unclear. The question of what size commissioning organisations should be to allow them to function most effectively is therefore highly relevant for local and national decision makers.⁴

Several hypotheses about the relationship between size of commissioning organisation and performance exist.⁵ These include the concern that small commissioning units are more exposed to financial risk, due to their smaller populations, and consequently their smaller risk pool. Similarly, smaller commissioning units may not have a critical mass of expertise or the required ‘market power’ to be able to negotiate effectively with providers to achieve good-value contracts. Alternatively, a smaller commissioning unit size may allow better local engagement and responsiveness for clinicians and patients.

The ongoing lack of consensus about the optimal size of commissioning units, leading to consequent reorganisations, has proved highly disruptive and costly to healthcare systems. The history of the NHS contains several reorganisations of the commissioning function, since the introduction of the purchaser–provider split in the early 1990s. Over this period, organisational units of different size have been tried. In April 1999, there were 481 primary care groups, with an average population of around 100 000.⁶ In 2005, there were 302 primary care trusts (PCTs, average population 170 000).⁷ Since 2010, there are now 152 PCTs (average population 290 000).⁸ The question of ideal organisational size is not limited to the UK health system, as other countries have experimented with different-sized organisational models. In Australia, New South Wales is currently reforming the size of organisational units from area health services (serving populations of around 1 million) to local health networks (serving 500 000),^9,10 having reformed the size of the units only 5 years previously.¹¹ In the US, there is a continuing discussion about the optimal size of physician groups and networks to provide the highest-quality care.¹²

The relationship between size of commissioning units and organisational performance remains under-investigated. A previous analysis of primary care organisations in England in 2003 examined a random sample of 72 commissioning organisations, and their performance against 22 measures.¹³ Using a mixture of telephone interviews and survey questionnaires, it was found that only two performance measures — provision of extended services and provision of intermediate care — were significantly associated with size. However, national analyses are lacking, and, since this study was carried out, performance measurement in the NHS has improved, new data are available at national level, variation between organisations may have reduced, and the optimum size of CCGs as the new commissioning organisations in the UK, remains contested. These factors all suggest it is appropriate to look at the question again.

How this fits in

Commissioning organisations vary in size and performance. New commissioning organisations will be created in the latest NHS reforms. Previous studies have found little evidence of a link between size and performance, but no national analysis has been done before. This study shows a link between commissioning organisation size and some areas of performance, but this is explained by the characteristics of the population. There is still no evidence of an ideal size for clinical commissioning groups in the new NHS reforms.

METHOD

Selecting measures of performance

Measuring the performance of commissioning organisations is difficult, in part because they have many different functions. Several frameworks for measuring quality of health care exist, with differing but often overlapping definitions. In order to ensure that this evaluation of performance considers the various aspects, the Organisation for Economic Cooperation and Development’s (OECD’s) framework for healthcare system performance was used,¹⁴ which focuses on healthcare quality and divides performance into five separate domains: effectiveness, safety, patient centredness, access, and cost. Two additional domains were added: ability to engage with the public and the local health economy and financial ability, as these are also stated aims of PCTs.¹⁵

Thirty-six established and commonly used performance measures across these domains were selected (Table 1). Measures of effectiveness were broken down into clinical and preventative effectiveness, reflecting the role of commissioning organisations in both of these areas. No performance measures were included for safety, as it was not possible to identify any that had sufficient standardisation and availability across all PCTs.

Table 1.

Performance variables, sources, and summary statistics

Variable	Date	Source	Details
Clinical effectiveness
Controlled blood pressure in hypertension	2009–2010	QOF16	% of hypertensive patients with last blood pressure reading of <150/90 mmHg
Controlled blood glucose levels in diabetes	2009–2010	QOF17	% of diabetes mellitus patients with last HbA1C ≤ 8%, age ≥17 years
Emergency admissions: acute conditions	2007–2008	NCHOD18	Emergency admissions: acute conditions usually managed in primary care
			Indirectly age- and sex-standardised rates per 100 000 persons
Emergency admissions: chronic conditions	2007–2008	NCHOD19	Emergency admissions: chronic conditions usually managed in primary care
			Indirectly age- and sex-standardised rates per 100 000 persons
Premature mortality from all circulatory diseases	2006–2008	NCHOD20	Directly age-standardised rates (DSR) per 100 000
			European Standard population, age <75 years
Mortality from causes amenable to health care	2006–2008	NCHOD21	DSR per 100 000 European Standard population
Mortality from all causes	2006–2008	NCHOD22	DSR per 100 000 European Standard population
Non-elective readmission rate	2008–2009	WCC datapack23	Standardised 28-day readmission ratio for non-elective activity
1-year survival index for all cancers	2006	ONS24	1-year survival index (%) for all cancers combined, age 15–99 years
Preventative effectiveness
Breast screening coverage	2009	NCHOD25	% coverage, age 53–64 years
Cervical screening coverage	2009	NCHOD26	% coverage, age 25–64 years
Uptake of influenza vaccinations by over 65s	2008–2009	WCC datapack23	% coverage, age >65 years
Smoking quitters	2008–2009	WCC datapack23	Rate per 100 000, age >16 years
MMR vaccination	2008–2009	NCHOD27	% vaccinated (first and second dose) by 5th birthday
Patient experience
Satisfaction with care received at surgery	2009–2010	GP Patient Survey28	% satisfied
GP recommendation	2009–2010	GP Patient Survey28	% who would recommend their GP surgery to someone who has moved to the local area
Staff noticed views	2009–2010	GP Patient Survey28	Doctor or nurse took notice of views about how to deal with health problem — % yes
Agreed with staff about managing problem	2009–2010	GP Patient Survey28	Did you and the doctor or nurse agree about how best to manage health problem? — % yes
Enough support	2009–2010	GP Patient Survey28	In last 6 months, had enough support from local services or organisations to help manage long-term health condition(s) — % yes
Out-of-hours GP service	2009–2010	GP Patient Survey28	Rating of the care received from the out-of-hours GP service — % good
Cost/efficiency
Tonsillectomy rate	2009–2010	NHS Comparators29	Standardised rate per 100 000 population
DNA rate	2008	WCC datapack23	% not attending for outpatient appointments
Excess bed days per non-elective admission	2008	WCC datapack23	Excess bed-days per non-elective admission (number of days)
Length of stay for fractured neck of femur	2008	WCC datapack23	Inpatient average length of stay for fractured neck of femur (days)
Low-cost statin prescribing	2009–2010	NHS Comparators30	% Prescribing of low-cost statins
Access
See doctor quickly	2009–2010	GP Patient Survey28	Able to see a doctor fairly quickly — % yes
Book appointment ahead	2009–2010	GP Patient Survey28	Able to book ahead for an appointment with a doctor in the past 6 months — % yes
Satisfaction with opening hours	2009–2010	GP Patient Survey28	Satisfaction with opening hours — % yes
2-week cancer wait	2007–2008	WCC datapack23	% of patients first seen by a specialist within 2 weeks when urgently referred
18-week wait	2008	WCC datapack23	% of patients seen within 18 weeks’ referral to treatment for non-admitted pathways
Finance
WCC Financial Governance score	2009–2010	WCC assessment31	Scored out of 10. Score based on subcomponents — 0 for red, 1 for amber, 2 for green
Health Care Commission annual health check	2008–2009	Care Quality Commission32	Scored according to four categories — 0 for poor, 1 for rating of financial performance adequate, 2 for good, 3 for excellent
Efficiency and effectiveness of spend	2009–2010	WCC assessment31	Ensuring efficiency and effectiveness of spend — score out of 12
Engagement
Work collaboratively with community	2009–2010	WCC assessment31	Work collaboratively with community partners to commission services that optimise health gains — score out of 12
Engagement with clinicians	2009–2010	WCC assessment31	Lead continuous and meaningful engagement with clinicians — score out of 12
Work with providers	2009–2010	WCC assessment31	Effectively manage systems and work in partnership with providers — score out of 12

DNA = did not attend. HbA1C = glycosylated haemoglobin. MMR = measles, mumps, and rubella. NCHOD = National Centre for Health Outcomes Development. ONS = Office for National Statistics. QOF = Quality and Outcomes Framework. WCC = World Class Commissioning.

Data collection

Data were obtained on each performance indicator for all 152 PCTS in England. The most recent data available were extracted from a number of different data sources (Table 1). Sources included the National Centre for Health Outcomes Development,³³ the Quality and Outcomes Framework (QOF; a national payment incentive framework for general practice),³⁴ NHS Comparators (a national comparative performance tool run by the NHS),³⁵ World Class Commissioning reports (WCC; a national commissioning performance assessment exercise),²³ and the national General Practice Patient Survey.²⁸ For almost all performance indicators, data were publicly available on the internet, and obtained via this route. Where data were missing on WCC performance,³¹ contact was made with five individual PCTs to provide their WCC assessments, and all provided this information. Data on PCT size were taken from the NHS Information Centre, 2009–2010, and counted the registered population in each area.⁸

Performance may be affected by a number of confounding variables, apart from size, that relate to the populations PCTs serve. For each PCT, information was therefore obtained on age structure (% >65 years),³⁶ ethnicity (% white),³⁷ level of deprivation (Index of Multiple Deprivation [IMD]),³⁸ and rurality (measured as a binary variable: urban, corresponding to predominantly urban in the Office for National Statistics [ONS] rurality classification; and ‘rural’, corresponding to the ‘predominately rural’ and ‘significantly rural categories’).³⁹ PCTs receive additional funding to compensate for these factors as well as for population size.

Data analysis

Population size was compared against each of the performance indicators at the PCT level, using Spearman’s rank correlation as a non-parametric measure of statistical dependence between two variables.

To look for potentially confounding relationships on indicators with a significant correlation coefficient, a univariate linear regression analysis (using ordinary least squares) was performed, comparing each performance indicator with each of: PCT size, level of deprivation, ethnicity, age, and rurality. To investigate the relationship between size, performance, and other potentially confounding variables, a multiple linear regression model was used, which included deprivation, age, and ethnicity as continuous variables and rurality (rural or urban) as a categorical variable, allowing the relative contributions of each variable to be understood. Data analysis was performed with STATA SE (version 11).

RESULTS

Unadjusted findings

Initial analysis of PCT size against the performance indicators shows that 14 of the 36 indicators had a significant correlation (P<0.05) between performance and population size. For 11 indicators, this was at the P<0.01 level (Table 2).

Table 2.

Spearman rank correlation between PCT size and performance

Indicator	Spearman’s rho	P-valuea	What does the relationship mean?
Clinical Effectiveness
Controlled blood pressure in hypertension	0.02	0.78	No relationship
Controlled blood glucose levels in diabetes mellitus	−0.02	0.78	No relationship
Emergency admissions: acute conditions	−0.33	<0.001	Bigger PCT, lower admissions rates
Emergency admissions: chronic conditions	−0.27	<0.001	Bigger PCT, lower admissions rates
Premature circulatory mortality	−0.32	<0.001	Bigger PCT, lower mortality
Mortality amenable to health care	−0.32	<0.001	Bigger PCT, lower mortality
Mortality from all causes	−0.30	<0.001	Bigger PCT, lower mortality
Non-elective readmission rate	0.03	0.72	No relationship
1-year survival index for all cancers	0.28	<0.001	Bigger PCT, better cancer survival
Preventative effectiveness
Breast screening coverage	0.25	0.002	Bigger PCT, better screening uptake
Cervical screening coverage	0.22	0.006	Bigger PCT, better screening uptake
Uptake of influenza vaccinations by over 65s	0.05	0.55	No relationship
Smoking quitters	−0.32	<0.001	Bigger PCT, lower quit rate
MMR vaccination	0.01	0.86	No relationship
Patient experience
Satisfaction with care received at surgery	0.06	0.47	No relationship
GP recommendation	0.15	0.06	No relationship
Staff noticed views	0.04	0.67	No relationship
Agreed with staff about managing problem	0.03	0.74	No relationship
Enough support	−0.27	<0.001	Bigger PCT, worse experience
Out-of-hours GP service	0.00	0.95	No relationship
Cost/efficiency
Tonsillectomy rate	−0.08	0.31	No relationship
DNA rate	−0.21	0.009	Bigger PCT, lower DNA rate
Excess bed days per non-elective admission	0.13	0.11	No relationship
Length of stay for fractured neck of femur	−0.07	0.42	No relationship
Low-cost statin prescribing	–0.16	0.05	Bigger PCT, more expensive prescribing
Access
See doctor quickly	0.19	0.02	Bigger PCT, quicker access
Book appointment ahead	0.11	0.17	No relationship
Satisfaction with opening hours	−0.18	0.03	Bigger PCT, worse opening hours
2-week cancer wait	−0.12	0.13	No relationship
18-week wait	−0.04	0.63	No relationship
Finance
WCC Financial Governance score	−0.12	0.15	No relationship
CQC rating of financial performance	−0.08	0.31	No relationship
Efficiency and effectiveness of spend	−0.04	0.67	No relationship
Engagement
Work collaboratively with community	−0.14	0.09	No relationship
Engagement with clinicians	0.03	0.68	No relationship
Work with providers	0.10	0.21	No relationship

^aUnadjusted P-value. CQC = Care Quality Commission. DNA = did not attend. MMR = measles, mumps, and rubella. QOF = Quality and Outcomes Framework. WCC = World Class Commissioning.

The commonest relationships between size and performance are for clinical effectiveness (six out of nine indicators) and preventative activity (three out of five indicators). There is less relationship between size and measures of access (two out of five), cost (two out of five), and patient experience (one out of six). There is no observed relationship between size and measures of commissioning ability or financial ability.

The general trend is that bigger PCTs provide better services. However, there are anomalies, for example bigger PCTs also have lower rates of smoking quitting, lower rates of generic statin prescribing, and lower satisfaction with opening hours. There is also a relationship between size of PCT and the average level of deprivation within it (Spearman P<0.001, larger PCTs are less deprived) and between size and rurality (Spearman P<0.001, larger PCTs are more rural). There was no observed relationship between size and ethnicity or age structure using Spearman’s rank test.

Adjusted findings

There were a number of relationships observed between the potentially confounding variables and the performance measures. Considering the 14 variables where there is a relationship with PCT size and performance on the Spearman analysis, there is also a relationship between deprivation and performance for 12 of the 14 indicators at the P<0.01 level, age (eight of 14), ethnicity (eight of 14), and rurality (10 of 14).

In the combined regression model that included population characteristics, PCT size is a much weaker predictor of performance than the other variables (Table 3). Size is no longer a significant contributor to the model for any variable (P<0.05). In contrast, deprivation is significant for 12 of 14 indicators (10 of 14 at P<0.001), ethnicity for 10 of 14 (10 of 14 at P<0.001), age in seven of 14 (six of 14 at P<0.001), and rurality in one of 14 (0 of 14 at P<0.001).

Table 3.

Multiple linear regression model of performance including list size, deprivation, age, ethnicity, and rurality

	PCT size		IMD		Age		Ethnicity		Rurality
Variable	t	P-value	t	P-value	t	P-value	t	P-value	t	P-value	R2
Emergency admissions: acute conditions	−1.35	0.18	6.75	<0.001	–0.16	0.88	3.42	0.001	−1.19	0.24	0.36
Emergency admissions: chronic conditions	0.14	0.89	8.21	<0.001	0.26	0.80	0.73	0.47	−1.63	0.11	0.44
Premature mortality from circulatory disease	−0.22	0.82	18.18	<0.001	−5.12	<0.001	4.18	<0.001	−0.62	0.54	0.82
Mortality amenable to health care	0.16	0.87	18.39	<0.001	−2.61	0.01	2.94	0.004	−1.06	0.29	0.79
Mortality from all causes	0.33	0.74	16.91	<0.001	−2.88	0.005	7.23	<0.001	−0.87	0.39	0.74
1-year survival index for all cancers	1.37	0.17	−4.72	<0.001	1.86	0.06	−1.50	0.14	−1.18	0.24	0.22
Breast screening coverage	1.38	0.17	−1.55	0.12	0.66	0.51	7.16	<0.001	0.80	0.43	0.57
Cervical screening coverage	0.96	0.34	−2.41	0.02	3.29	<0.001	3.46	0.001	1.55	0.12	0.58
Smoking quitters	−1.94	0.05	11.23	<0.001	−1.42	0.16	4.28	<0.001	1.74	0.08	0.54
Enough support	−1.72	0.09	6.51	<0.001	1.37	0.17	5.03	<0.001	−1.04	0.30	0.41
DNA rate	−1.06	0.29	3.68	<0.001	−2.32	0.02	−5.32	<0.001	0.62	0.54	0.57
Low-cost statin prescribing	−1.07	0.29	−0.93	0.35	−0.99	0.32	−0.50	0.62	0.46	0.64	0.03
See doctor quickly	−0.15	0.88	−2.02	0.05	4.29	<0.001	1.32	0.19	3.10	0.002	0.55
Satisfaction with opening hours	−0.87	0.38	6.63	<0.001	2.67	0.008	5.75	<0.001	0.31	0.75	0.49

DNA = did not attend. IMD = Index of Multiple Deprivation.

DISCUSSION

Summary

These results suggest that there is a relationship between the size of PCTs and organisational performance in a number of areas of their activity, particularly clinical and preventative effectiveness. However, the relationship is no longer present when population characteristics such as deprivation are taken into account.

Where there is an unadjusted relationship with size, larger PCTs tend to provide higher-quality care (in 10 out of 14 indicators). An explanation might be that larger PCTs do better because they are more likely to serve affluent, rural, less ethnically diverse populations. This may be a result of the recent pattern of reorganisations that has tended to leave smaller PCTs in urban areas (and London in particular), and larger PCTs in rural counties.

Deprivation appears to be the factor that influences performance for the most variables, but ethnicity is also a strong factor in some areas such as screening programme coverage. This is consistent with other work highlighting poor knowledge regarding screening and low uptake of breast and cervical screening programmes in certain ethnic populations.^40,41

The fact that a few indicators seem to demonstrate worse health care with increasing size — for example, larger PCTs have lower smoking quit rates, poorer satisfaction with opening hours, and less-efficient prescribing — is an interesting anomaly. The lower rate of smoking quitters may be because rates of smoking are higher in more deprived, smaller, urban PCTs and the denominator for this indicator is total population size rather than total number of smokers. In these areas, extra resources may be directed towards ‘stop smoking’ campaigns. This is reinforced by the finding that deprivation appears to be the most important explanatory factor in the combined model relating to the smoking quit rate. Lower rates of satisfaction with opening hours may reflect actual differences in activity, or may reflect different expectations of different groups within the populations in PCTs.⁴²

Some domains of quality appear more likely to be related to PCT size than others, in particular measures of effectiveness. There was no relationship, even unadjusted, between size and financial performance or engagement for any of these indicators. This suggests that the hypotheses that smaller PCTs are less able to negotiate contracts effectively, are more exposed to financial risk, and are better able to engage with their community and partners are not supported by the evidence at the current size of PCTs.

Strengths and limitations

There are a number of limitations in the methods used to assess performance of commissioning organisations. Although the study attempted to measure PCT performance across a broad remit, no system for measuring quality will be able to capture all aspects of performance. Moreover, a lack of correlation between various measures of PCT quality has been observed before.⁴³

This paper presents a set of measures, based on an established framework (OECD) that cut across a number of different aspects of performance, and relate to evidence-based guidance for prevention and treatment of common clinical conditions. New national data have been used for the first time, and the study has taken advantage of the more vigorous approach to performance management in recent years to obtain data from many sources. The authors accept that some of the indicators used are not validated. In particular, this includes many of the newer indicators relating to financial and engagement performance, as these have only emerged recently from WCC assessments. Also, these indicators have varying levels of accuracy and completeness of data, and some of the study indicators were based on data collected before 2009/2010, and may not reflect current PCT performance.

Furthermore, some of the outcome variables used may be better than others in terms of reflecting PCT influence. For example, performance on QOF and prescribing indicators may be driven by individual GP performance, and mortality rates by population factors, whereas access to primary care and WCC scores may be more likely to be influenced in the short term by the commissioning organisation. However these results show that relationships between size and performance are observed across several different domains and data sources.

There are also other structural factors that could be viewed as confounders, such as the number of GPs or nurses per 1000 patients. These data were not used, as they were thought to be dependent on PCT commissioning activity. The authors also accept that this analysis makes multiple comparisons. However, significance values have been given at both the 0.05 and 0.01 level, and where relationships are demonstrated, most are highly significant. Due to the nature of the data, causal relationships between PCT size and indicators cannot be concluded.

Comparison with existing literature

These findings are consistent with earlier research on the topic, including a smaller survey of PCT performance by Wilkin et al.¹³ Bojke et al reviewed the UK and international literature in 2001.⁴⁴ They suggested that the size of primary care organisations is only one of the factors that affect performance. They suggested a framework in which primary care organisation performance is affected by a combination of their aims, tasks, functions, organisational features (including both size and governance), and environmental factors, including demographic mix and socioeconomic characteristics. They also suggest that there is no optimum size for a commissioning organisation, because there are different economies of scale for different functions and because of the variety of functions the organisations perform.

A similar study in another field, which examined the performance of local government functions in authorities of different size suggests that organisational size has an impact on some areas of performance, but not others, including positive, negative, and non-linear relationships.⁴⁵ The authors describe the relationship between size and performance as a ‘complex mosaic’. The extent of the effect of deprivation and ethnicity in explaining the link between size and performance has also not been demonstrated so strongly in previous work. The present findings mirror those from practice-level studies that found no strong associations between size of general practices and performance measures, and in which population factors were found to have the greatest impact.^46,47

Implications for practice and research

As a result of the complex, confounded, and multidirectional results observed, this analysis fails to provide any conclusive answers to the question of what size a commissioning organisation needs to be to perform best. This adds to the existing literature that suggests that there is no obvious optimum size of commissioning organisations. Rather than pursuing an optimum size, those designing a new commissioning system could instead look to other characteristics of the organisations that might affect performance, such as the internal structure, the strength of its networks with other organisations, and the composition and skills of its workforce.

The cost of the current NHS reorganisation in England has been estimated at £2–3 billion.⁴⁸ The future configuration of CCGs remains uncertain, but doubts have been expressed about the viability and financial stability of smaller groups. The present analysis suggests that smaller commissioning organisations can function as effectively as larger ones, across a broad range of performance measurement. But, given the enormous pressure to reduce management costs, it is likely that larger CCGs will be the norm. The configuration of CCGs, and similar entities in other health systems, should therefore be geared towards producing organisations that can function effectively across their key responsibilities, rather than just being based on the size of their population alone.

Funding

Felix Greaves is funded by London Deanery and the National Institute for Health Research (NIHR). Utz J Pape is funded by the NW London NIHR Collaboration for Leadership in Applied Health Research and Care. Christopher Millett is funded by the NW London NIHR Collaboration for Leadership in Applied Health Research and Care and the Higher Education Funding Council for England. The Department of Primary Care and Public Health at Imperial College is grateful for support from the NIHR Biomedical Research Centre scheme, the NIHR Collaboration for Leadership in Applied Health Research and Care scheme; and the Imperial Centre for Patient Safety and Service Quality.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article on the Discussion Forum: http://www.rcgp.org.uk/bjgp-discuss