Improvements in quality of clinical care in English general practice 1998-2003: longitudinal observational study

Abstract

Objective To measure changes in quality of care for three major chronic diseases (coronary heart disease, asthma, and type 2 diabetes) between 1998 and 2003.

Design Longitudinal cohort study.

Setting 42 general practices in six geographical areas of England (Avon, Bury/Rochdale, Enfield, Oldham, Somerset, South Essex).

Participants Medical record data for 2300 patients with diabetes, asthma, or coronary heart disease in 1998, and 1495 patients in 2003.

Main outcome measure Quality of care assessed against predefined evidence based review criteria.

Results Between 1998 and 2003, quality of care improved markedly in terms of maximum possible scores on the review criteria, from 60.5% to 78.1% for coronary heart disease (change = 17.6, 95% confidence interval 13.9 to 21.4; P < 0.001), 60.1% to 70.3% for asthma (10.2, 4.6 to 15.8; P = 0.001), and 70.4% to 77.7% for diabetes (7.3, 3.5 to 11.1; P = 0.001). Important changes occurred to several indicators potentially related to improved health outcomes. These included improved control of serum cholesterol (to ≤ 5 mmol/l) from 17.6% to 61.4% in coronary heart disease and from 21.5% to 52% in diabetes and control of blood pressure to ≤ 150/90 in coronary heart disease from 47.3% to 72.2% and to ≤ 145/85 in diabetes from 21.8% to 35.8%. A small, non-significant improvement in glycaemic control occurred among diabetic patients (37.9% to 39.7% with HbA_1c < 7.4%). Significant improvements also occurred in the recording of exercise capacity and diet and weight advice for patients with coronary heart disease; of smoking advice, peak flow, and symptoms for patients with asthma; and of creatinine, weight, and HbA_1c for patients with diabetes. Over the five years, more improvement in coronary heart disease care occurred in large practices and practices in affluent areas.

Conclusions Substantial improvements were seen in quality of care for the three conditions studied between 1998 and 2003, a time of systematic quality improvement initiatives in the NHS. The changes were most marked for coronary heart disease. English general practices could be expected to achieve high clinical quality scores in the initial year of a new contact, which provides financial incentives for high quality care from 2004.

Introduction

Improving quality of care has been a major focus of UK government policy since 1997, including the introduction of clinical governance as part of a 10 year strategy to improve quality of care.¹,² The government introduced several national guidelines (national service frameworks), which set minimum standards for the delivery of health services in England, including care for patients with coronary heart disease in 2000 and diabetes in 2002.³,⁴ The National Institute for Health and Clinical Excellence (NICE) also published guidance on clinical interventions, including some aspects of coronary heart disease in 2001 and type 2 diabetes in 2002 (www.nice.nhs.uk). As part of this thrust to improve quality of care, primary care trusts used a wide range of methods to promote quality improvement, focusing on coronary heart disease and to a lesser extent on diabetes and mental health, conditions that were addressed by national service frameworks. Conditions not covered by frameworks, such as asthma, were targeted less frequently.⁵ The National Primary Care Development Team also did a major study of the management of coronary heart disease in more than 2000 practices (www.npdt.org). Most recently, financial incentives for improved chronic disease management have been introduced as part of a new contract for general practitioners starting in April 2004.⁶

We have previously found significant variation in the quality of clinical care for major chronic diseases in England,⁷ a finding supported by our research on a wide range of conditions in UK general practice⁸ and studies of quality of care elsewhere.⁹,¹⁰ Initial evidence suggested that the government's quality improvement strategy was having broadly positive effects in a range of settings,¹¹ including in primary care,¹² with improvements occurring in targeted areas through the use of new types of contract.¹³,¹⁴

As part of our programme to evaluate quality improvement in the NHS,¹⁵ we now present the results of the first five years of a longitudinal study of quality of care for coronary heart disease, asthma, and diabetes. Two of these conditions (coronary heart disease and diabetes) have been the focus of national service frameworks and NICE guidance.

Methods

Design

Quality of care for coronary heart disease, asthma, and type 2 diabetes was measured in a stratified random sample of 60 general practices in England in 1998 as part of a previous study.⁷ Fifty seven of the practices were still in existence in 2003, and 42 (74%) of these practices are taking part in a longitudinal study of quality of care until 2007.¹⁶ This paper reports changes in quality of clinical care over the first five years of this period (1998-2003). The 42 practices are located in six geographical areas of England: Avon, Bury/Rochdale, Enfield, South Essex, Oldham, and Somerset. Table 1 shows the characteristics of the practice samples in 1998 and 2003 compared with all practices in England.

Table 1.

Practice characteristics in 1998 and 2003, compared with all practices in England. Values are numbers (percentages)

Characteristic	1998*(n=60)	1998†(n=42)	1998‡(England, n=9090)	2003§(n=42)	2003¶(England, n=8832)
Training	17 (28.3)	9 (28.6)	2204 (24.6)	13 (30.9)	2272 (25.7)
≤2 whole time equivalent GPs	29 (48.3)	16 (38.0)	4360 (48.6)	15 (35.7)	4221 (47.8)
Single handed practices	18 (30.0)	9 (21.4)	2779 (30.6)	10 (23.8)	2577 (29.2)
Receiving deprivation payments	34 (56.7)	25 (59.5)	5390 (59.3)	21 (50.0)	4956 (50.8)

GP=general practitioner.

^*Original 1998 study sample.⁷

^†1998 sample restricted to 42 practices taking part in 2003 data collection.

^‡All practices in England in 1998.

^§2003 sample.

^¶All practices in England in 2003 (General Medical Services database, 2003).

Data collection

Trained research staff extracted data to assess the quality of clinical care for coronary heart disease (15 indicators), asthma (13 indicators), and diabetes (22 indicators). Data were collected from medical records on computer and paper, using previously developed evidence based review criteria.¹⁷,¹⁸ Some additional indicators were included that related to guidance in the national service frameworks for coronary heart disease and diabetes. These included levels of blood pressure, cholesterol, and HbA_1c to be achieved. Data on these indicators had been collected in 1998 (though not published in our previous reports on these practices⁷,¹⁸) and were again collected in 2003.

We randomly selected patients with the three conditions from lists of patients receiving relevant drugs on repeat prescription within the previous six months, drawing separate samples of patients in 1998 and 2003. For each condition, we drew up a list of relevant drugs in order to sample patients (list available from authors) and subsequently checked individual records to ensure that the patient did have the relevant condition. This ensured, for example, that patients selected on the basis of a prescription for a β blocker did not solely have hypertension. One inner city London practice in 1998 had no eligible patients with coronary heart disease owing to a young patient population, so results for coronary heart disease relate to 41 practices only.

We collected data for up to 20 patients per condition per practice in 1998 (some small practices did not have 20 patients with each of the conditions in 1998 owing to strict adherence to the inclusion criteria) and for 12 patients per condition in 2003.

Data analysis

We calculated an overall “quality of care” score for each patient with coronary heart disease, asthma, or diabetes in 1998 and in 2003. For each patient, the score was a simple ratio of the number of chronic disease indicators for which care was provided divided by the number of indicators for which care should have been provided (this number differed between patients, as not all indicators applied to all patients). Expressed as a percentage, the score represents the percentage of “necessary care” provided to each patient, within a range from 0% to 100%. We calculated practice level quality scores as the simple average of the scores for the individual patients within each practice. We also analysed individual indicators in the two time periods. We did not analyse data for eight indicators that could be applied to an average of less than one patient per practice. In 1998 we collected data for indicators that needed an activity to be done annually on the basis of data recorded in the previous 14 months, whereas in 2003 we did this on the basis of data from the previous 15 months in line with the quality and outcomes framework of the 2004 contract. We adjusted the resulting difference in scores in 1998 where applicable.

We used Stata 8.1 for statistical analysis. The structure of the data was such that independent groups of patients were observed in 1998 and 2003 in each of the 42 practices. We used the patient as the unit of analysis for comparison of care at the two time points. We analysed patient level results for individual indicators by using logistic regression and patient level quality scores by using ordinary regression, with time point as the independent variable and practice as a cluster variable.

We used regression analysis to investigate several predictors of the change in quality scores from 1998 to 2003. All predictors were at practice level, so the unit of analysis was the practice not the patient (that is, all variables were rolled up to the practice level). The independent variables were the 1998 values for practice size (whole time equivalent general practitioners), whole time equivalent general practitioners per 1000 patients, socioeconomic deprivation score (mean NHS deprivation payment per patient), and routine booking interval for consultations ≥ 10 minutes. We investigated the relations between these variables and the change in quality score by using regression (univariate regressions followed by multivariate regression using variables for which P < 0.1). Each analysis controlled for the quality score in 1998 and applied robust standard errors.

To determine whether practices had converged or diverged since 1998 in terms of the quality of care they provide, we compared the variance in practice level quality scores at the two points in time by using Pitman's t test for correlated variances.¹⁹

Adjustments for loss of practices from the study

The original sample of 60 practices in 1998 was nationally representative of all English general practices.⁷ Eighteen of these did not participate in the current study—three because doctors had retired and 15 because they declined to take part. The 2003 sample may therefore no longer have been nationally representative (table 1). For this reason, we estimated mean quality scores in 2003 for the full sample of 60 practices by using probability weighted regression, with weights derived from forward stepwise logistic regression (with a conservative α = 0.1), to model the probability of remaining in the study on the basis of the following practice characteristics in 1998: coronary heart disease, asthma, and diabetes scores; practice size (whole time equivalent general practitioners and group practices versus single handed practices); training status; and socioeconomic deprivation score. Results are reported with and without this adjustment.

Results

Table 2 shows summary statistics from the regression analyses for the patient level quality scores for coronary heart disease, asthma, and diabetes, together with the comparison between time points. Table 3 gives standard deviations and ranges, together with Pitman's t test for correlated variances. The table on bmj.com shows scores for individual indicators in the two time periods.

Table 2.

Change in clinical quality scores 1998-2003

	1998		2003				Improvement (% maximum possible improvement)
Condition	No of patients	Mean (SD) quality score	No of patients	Mean (SD) quality score	Change (95% CI) in scores, 1998 to 2003	P value*
Coronary heart disease (41 practices)	737	60.5 (23.1)	487	78.1 (18.7)	17.6 (13.9 to 21.4)	<0.001	45
Asthma (42 practices)	785	60.1 (29.2)	504	70.3 (27.5)	10.2 (4.6 to 15.8)	0.001	26
Diabetes (42 practices)	776	70.4 (21.7)	504	77.7 (18.5)	7.3 (3.4 to 11.1)	0.001	25

^*From regression analysis using patient level data, allowing for clustering of patients within practices.

Table 3.

Change in variance in practices' quality scores 1998-2003

			Pitman's t test of correlated variances
Condition	SD; range in 1998	SD; range in 2003	t	df	P value
Coronary heart disease (41 practices)	9.68; 37.36-75.67	11.10; 44.8-94.8	0.84	39	0.41
Asthma (42 practices)	16.47; 20.6-85.5	15.93; 35.8-96.6	−0.23	40	0.82
Diabetes (42 practices)	13.74; 35.0-88.9	11.44; 43.2-93.4	−1.34	40	0.19

Coronary heart disease

Significant improvements in coronary heart disease care occurred between 1998 and 2003 (P < 0.001; 95% confidence interval of change 13.9 to 21.4) (table 2). In 1998 patients received, on average, 60.5% of the care they should have received (that is, 60.5% of the coronary heart disease indicators that applied to the individual patient were met). By 2003 this figure had increased to 78.2%. This represents 45% of the maximum possible improvement on the 1998 figure. Pitman's t test (table 3) indicates that the variance in practice level mean scores for coronary heart disease did not change between 1998 and 2003.

The table on bmj.com shows that the percentage of necessary care provided increased for all the individual coronary heart disease indicators except prescription for aspirin or recorded advice to take it. The increase was statistically significant for 11 of the 15 coronary heart disease indicators—for example, in the measurement of serum cholesterol (from 63% to 89%, P < 0.001), control of serum cholesterol to 5.0 mmol/l or below (from 17.6% to 61.4%, P < 0.001), and control of blood pressure to 150/90 mm Hg or below (from 47.3% to 72.2%, P < 0.001). These are changes in care that are likely to be associated with significant impacts on health.²⁰ Improvements also occurred in the recording of frequency of angina attacks (P < 0.001), exercise capacity (P < 0.001), dietary advice (P < 0.001), advice on weight (P < 0.01), referral to a specialist or for an exercise electrocardiogram (P < 0.001), and smoking advice to smokers (P < 0.05). In 1998 only four out of 15 indicators were achieved for more than two thirds of patients. By 2003 this had increased to 10.

Asthma

Table 2 shows that a significant overall improvement occurred in asthma care (P < 0.001; 95% confidence interval of change 4.6 to 15.8). In 1998 asthma patients received, on average, 60.2% of the care they should have received, which increased to 70.3% in 2003. The increase represents 26% of the maximum possible improvement on the 1998 figure. The variance in practice level mean scores for asthma did not change between 1998 and 2003 (table 3).

The table on bmj.com shows that the percentage of necessary care provided increased for all indicators except peak flow taken during a consultation and recording of speech/pulse/respiratory rate during an exacerbation, which fell. However, the increase was statistically significant for only four out of 13 indicators. These were action taken in relation to exercise induced bronchospasm (P < 0.001) and recording of smoking advice (P < 0.001), peak flow (P < 0.01), and symptoms (P < 0.01). The increase in smoking advice represented 67% of the total possible gain between 1998 and 2003.

Diabetes

A significant overall improvement (P < 0.001; 95% confidence interval of change 3.4 to 11.1) occurred in diabetes care (table 2). In 1998 diabetes patients received, on average, 70.4% of the care they should have received, compared with 77.7% in 2003. The increase represents 25% of the maximum possible improvement on the 1998 figure. The variance in practice level mean scores for diabetes did not change between 1998 and 2003 (table 3).

The percentage of necessary care provided increased for 18 out of 22 indicators and decreased for four indicators, but the improvements were statistically significant for only seven indicators (see table on bmj.com). These included the measurement of serum cholesterol (from 74.9% in 1998 to 97.6% in 2003, P < 0.001), control of serum cholesterol to 5.0 mmol/l or below (from 21.5% to 52%, P < 0.001), and control of blood pressure to 140/85 mm Hg or below (from 21.8% to 35.8%, P < 0.001). Improvements also occurred in recording of creatinine (P < 0.001), weight (P < 0.05), and HbA_1c (P < 0.05). The increase in recording of cholesterol represented 90% of the entire potential for gain, whereas the improvement in the control of cholesterol and blood pressure represented 39% and 18% of the potential for gain. However, the proportion of patients with an HbA_1c < 7.4% increased from only 37.9% to 39.7%, a nonsignificant increase.

Predictors of quality change

We found no significant associations between any independent variable and the change in asthma care or the change in diabetes care. The change in care for patients with coronary heart disease (multivariate analysis, controlling for care in 1998) was positively associated with practice size (P = 0.012) and negatively associated with socioeconomic deprivation score (P = 0.02). Number of whole time equivalent general practitioners explained around 12% and deprivation around 3% of the variation in coronary heart disease care in 2003 unexplained by care in 1998.

Adjustment for loss of practices from the study

Practices that participated in the study in 2003 differed from non-participants (P < 0.1) only in regard to practice size: 22% (9/41) of group practices were lost to the study, compared with 47% (9/19) of single handed practices (P = 0.05). After adjustment for the differential drop-out rate among single handed practices, the estimated 2003 scores for the full nationally representative sample of 60 practices were 77.8 for coronary heart disease, 69.8 for asthma, and 77.3 for diabetes. None of these scores differed from the equivalent unadjusted score (table 2) by more than one point.

Discussion

Quality of care for the three major chronic conditions studied increased substantially, including statistically significant improvements for 22 of the 50 indicators across the three conditions. We saw improvements in various types of care, including recording of symptoms and advice, in-house procedures and test ordering, action taken in response to symptoms, and control of blood pressure and cholesterol. However, across the 42 practices, as much variation existed between the highest and lowest performing practices in 2003 as in 1998. Over the five years, more improvement in coronary heart disease care occurred in large practices and practices in affluent areas, but these factors were not significant predictors of improvement for diabetes or asthma.

Limitations of the study

The power of the analyses of the 1998 and 2003 data relates to the power of the overall longitudinal study, of which the data in this paper form part. Power analysis determined that the 2003 sample was sufficient to meet the requirements of the ongoing longitudinal study. This is designed to have 90% power to detect a six point deviation in the overall quality score in 2005 (the next data collection point) away from the trend before that point. A direct comparison between the first two time points (1998 and 2003) has power to detect only large to moderate effects. We found highly significant changes despite these power limitations.

As a result of practices leaving the study, the original nationally representative sample of 60 practices in 1998 was reduced to 42 practices in 2003. Single handed practices were more likely to leave the study. Overall mean 2003 scores adjusted for this factor were not substantially different from the unadjusted score for any of the three conditions, suggesting that attrition did not bias the results. A limitation is that the analysis assumes that the reasons why practices left the study were not related to their performance. Although we have no information about the performance of these practices in 2003, quality scores in 1998 did not predict non-participation in 2003.

Implications for policy and practice

In a rapidly changing healthcare system, we cannot attribute these changes with certainty to any one intervention. We found improvements in care as great for asthma, where the main national quality improvement thrust was from the British Thoracic Society (www.brit-thoracic.org.uk/sign/index.htm) and its widely publicised guidelines, as in coronary heart disease and diabetes, which were subject to major initiatives by central government and primary care trusts.⁵ These various interventions may have interacted. For example, the thrust to improve quality of care for coronary heart disease may have put practices in a position to improve care for other conditions.

Improvements may also have been associated with general improvements in the standard of data recording in practices. However, these indicators are based on ratings from expert panels when panellists were clear that all items should be both clinically necessary and aspects of care that should be recorded routinely in medical records.¹⁷ Moreover, indicators were included only if more than 80% of nurses and doctors in the original sample of 60 practices reported that they did record them on a routine basis.¹⁸ Evidence also shows that data recording is an important part of quality of care.²¹ The fact that clinical audit, first introduced on a widespread scale in the early 1990s,²² had laid much of the infrastructure for the more focused quality improvement initiatives that developed later in the decade may also be important.

What is already known on this topic

Previous research shows widespread variation in the quality of general practice care

What this study adds

Substantial improvements occurred in clinical quality of care for coronary heart disease, diabetes, and asthma between 1998 and 2003

These changes took place at a time of widespread government and professional initiatives to improve quality of care

The improvements that occurred between 1998 and 2003 suggest that practices are well placed to respond to the quality incentives offered in the 2004 contract

Successful quality improvement is likely to require a focus on close team working within practices,²³ a combination of clinical and organisational approaches, and strategies that widen to include the practice team, the primary care trust, and central government initiatives.²⁴ What our findings clearly show is that, in association with the systems based strategy of clinical governance, the quality of care for all three conditions studied increased substantially over a five year period.

The variation in care between practices did not change between 1998 and 2003 despite overall increases in quality scores for all three conditions. The samples of patients were not large enough to provide reliable estimates of the change within individual practices, so we cannot say whether these results are due to parallel improvements in all practices or if the pattern of change is more complex. If the trend towards improvement continues in the future, however, a reduction in practice level variation is to be expected, simply because some practices will begin to hit the ceiling of quality scores. The results of this study suggest that, for these conditions at least, practices would be expected to achieve high scores in the quality and outcomes framework of the new General Medical Services contract,⁶ especially bearing in mind that the data in this paper make no allowance for patients who fail to attend for review or those in whom additional treatment might not be clinically appropriate.