A comparison of chronic illness care quality in US and UK family medicine practices prior to pay-for-performance initiatives

Abstract

Background. The Quality and Outcomes Framework (QOF) has contributed to modest improvements in chronic illness care in the UK. US policymakers have proposed similar pay-for-performance (P4P) approaches to improve care. Since previous studies have not compared chronic illness care quality in US and UK primary care practices prior to the QOF, the relative preparedness of practices to respond to P4P incentives is unknown.

Objective. To compare US and UK practices on P4P measures prior to program implementation.

Methods. We analysed medical record data collected before QOF implementation from randomly selected patients with diabetes or coronary artery disease (CAD) in 42 UK and 55 US family medicine practices. We compared care processes and intermediate outcomes using hierarchical logistic regression.

Results. While we found gaps in chronic illness care quality across both samples, variation was lower in UK practices. UK patients were more likely to receive recommended care processes for diabetes [odds ratio (OR), 8.94; 95% confidence interval (CI), 4.26–18.74] and CAD (OR, 9.18; 95% CI, 5.22–16.17) but less likely to achieve intermediate diabetes outcome targets (OR, 0.50; 95% CI, 0.39–0.64).

Conclusions. Following National Health Service (NHS) investment in primary care preparedness, but prior to the QOF, UK practices provided more standardized care but did not achieve better intermediate outcomes than a sample of typical US practices. US policymakers should focus on reducing variation in care documentation to ensure the effectiveness of P4P efforts while the NHS should focus on moving from process documentation to better patient outcomes.

Introduction

Recent studies have documented significant gaps between the care that US patients with chronic illnesses receive and that recommended for achievement of optimal health outcomes.1 The fee-for-service (FFS) structure of the US health care payment system, which rewards physicians for providing a high volume of services but offers few incentives for high-quality care, has been identified as one of the causes of the chasm between the quality that could be achieved and the care that is actually delivered in the USA.2 To bridge this chasm, a number of US health plans and the US Medicare program have implemented pay-for-performance (P4P) programs designed to provide payment incentives aligned with quality-of-care targets.3,4 For example, the Medicare program, administered by the US Centers for Medicare and Medicaid Services, offers a 1.5% payment bonus for reporting achievement on 74 quality targets across a variety of areas. The effects of the CMS pilot on patient health are not yet known and the relatively limited evidence from other P4P programs in the USA is mixed.5–12 However, results from the UK Quality and Outcomes Framework (QOF) P4P initiative suggest that such programs can have a modest positive effect on the quality of care delivered in primary care settings.13–16

The improvements achieved under the QOF were preceded by previous investments in primary care and quality improvement capacities, including the computerization of medical records systems, which may have prepared UK primary care practices to respond to quality incentives.13,17–19 Evidence from the first year of the QOF suggest that such investments did prepare UK primary care practices with expectations of 75% achievement rates for clinical indicators exceeded by the median reported achievement rate of 83.4%. For diabetes and coronary artery disease (CAD), average quality measurement achievement rates were 80.1% and 85.7%, respectively.14 Many of these measures reward the documentation and organization of care and these are the areas that could be expected to have improved in response to previous quality improvement efforts in the UK. Similar investments in primary care capacity were not made in the USA during this same period. Understanding differences in practice-level quality between the two countries prior to major P4P initiatives is of crucial importance for understanding the effects of such initiatives across the 2 health care systems. However, no studies have compared practice-level quality of care and care documentation in the USA and UK in the period just before implementation of the QOF.

We compare samples of US and UK primary care practices on both process of care and intermediate outcomes typically targeted in P4P programs such as the QOF during a 1-year period prior to the advent of major P4P initiatives in either country. This comparison provides an opportune baseline for comparing the relative preparedness for reporting and improving P4P quality measures of primary care practices in these settings.

Methods

US data

US medical record review data were collected in 55 family medicine practices in New Jersey and Pennsylvania from April 2003 through December 2004 as part of the baseline data collection for a quality improvement intervention study, Using Learning Teams for Reflective Adaptation.20 Clinical data were collected from both computerized and handwritten medical records within each of the practices. Prior to any intervention efforts associated with the study, records were reviewed by chart auditors with nursing backgrounds using criteria adapted from existing clinical practice guidelines for diabetes and CAD.

Each practice generated lists of patients seen in their office during the previous 12 months based on billing codes for CAD [International Statistical Classification of Diseases and Related Health Problems (ICD-9) codes 410–414.9] or diabetes (ICD-9 codes 250–250.9). Within each practice, 20 patients were randomly selected from each list of patients. In cases where there were fewer than 20 patients per diagnosis code, all patients were used. In four practices, patients with CAD could not be identified and these practices are excluded from the CAD analyses.

UK data

UK medical record review data were collected as part of a longitudinal study of family medicine practices in the UK prior to the implementation of the UK P4P program. In 1998, a stratified nationally representative random sample of 60 family medicine practices in the UK was assessed to determine the quality of care for their patients with CAD or diabetes.21 From February 2003 through August 2003, trained research staff collected medical record review data in 42 of these practices.17 Clinical data were collected from both computerized and handwritten medical records, using evidence-based review criteria for CAD and diabetes developed using the RAND Corporation/University of California at Los Angeles consensus panel method.22,23 For the analyses reported here, we examined patient records from 2003.

Samples of up to 12 patients were selected at random from lists of patients registered with a particular practice for at least 15 months, who were receiving regular medication (list available from authors) for either CAD or diabetes. The records of these selected patients were then reviewed to confirm a diagnosis of either targeted chronic illness. In practices where there were 12 or fewer patients for a condition, all patients were used.

Measures

For the comparative analyses reported here, we selected process of care and intermediate outcome measures that were recorded in both data sets. For diabetes processes of care, we selected the following quality indicators: haemoglobin A1c (HbA1c) value recorded within the last 12 months, blood pressure (BP) recording with the past 12 months, cholesterol recording within the past 5 years, weight and height recorded and smoking status ever recorded. For intermediate outcomes of diabetes care, we selected HbA1c <7.4, systolic BP <140 and diastolic BP <85 and cholesterol <200 mg/dl. For CAD, we used the same BP, cholesterol and smoking status recording indicators as for diabetes. Intermediate outcome measures for CAD were limited to BP and cholesterol using values as those for diabetes.

When quality standards differed between countries, the more easily achieved quality standard was used as the review criterion for assessing care quality. For example, US guidelines for diabetes recommend that HbA1c testing be conducted every 6 months, whereas UK review criteria are based on HbA1c testing every 12 months.24 Therefore, in comparing the two countries, we used the review criterion that HbA1c be tested within the last 12 months.

A separate quality of care score was created both for processes of care and achievement of intermediate outcomes for each individual with diabetes or with CAD. This score was a count of either the services received (processes) or of targets achieved (intermediate outcomes). These counts have the same upper bound for each individual with a particular condition. For example, all individuals with diabetes are eligible for five processes of care and those with CAD for three. If information about the outcome had not been collected within the recommended time frame, analyses concerning achievement of intermediate outcomes were conducted in two ways: (i) the indicator for whether the target had been achieved was set to missing and (ii) it was assumed that the individual had not achieved the target. Since the findings did not differ using these two methods, we report only the second, more stringent, approach.

Statistical analysis

We examined individual components of care for patients with diabetes or CAD using hierarchical logistic regression to model the binary responses for processes or achievement of intermediate targets (yes/no). These models included fixed effects for country (USA or UK), patient age and gender as well as random effects to take into account the fact that patients are nested within practices. Since electronic medical record (EMR) use is part of the UK system, but not the US system, of health care delivery, any effect of EMR usage on quality would be confounded with system-level effects in these samples. Estimates and confidence intervals (CIs) for the age- and gender-adjusted rates, based on the regression coefficients for the fixed effect of country, describe the mean rate of achievement of review criteria standards for a practice. We created these estimates by calculating the least square means for the log-odds of adherence for country and transforming these to obtain estimates for proportions or rates.

Although differences in intermediate outcome achievement between the two samples may be affected by the relative health of the two patient populations, differences in chart abstraction variables made it impossible to adjust our findings for co-morbidities. We also were unable to obtain information on patients’ socioeconomic status, race or ethnicity and thus are unable to identify disparities in quality of care related to these domains. However, these demographic variables do not affect recommendations for appropriate care for particular patients and thus would not change our scoring of quality for particular patients even if available. Information on practice size in the two samples was not directly comparable and thus was not included in our analyses.

Counts of processes of care and intermediate outcomes for each condition as well as averages of practice rates for each individual component of these measures were compared using univariate statistics. To compare counts of processes and intermediate outcomes achieved across the two samples, we used a hierarchical proportional odds model with the counts as the dependent variable. Estimation was conducted using pseudolikelihood with Taylor series expansions around the random practice effect, creating practice-level inferences for estimates related to practices’ location.25

Results

All UK practices used EMRs while only nine US practices did so with five of those using some combination of EMR and paper charts. All physicians in both samples were generalists (called family physicians in the USA and GPs in the UK). Patients in US practices included those with insurance coverage from the US Medicare program, state Medicaid insurance, private insurers and a small number of patients who paid directly for medical care. In the UK, all patients had insurance coverage from the National Health Service (NHS). Practices in the US sample had somewhat higher proportions of women and younger patients but these differences were not statistically significant (see Table 1). For all processes of care and intermediate outcomes assessed, the variability of practice rates was lower in the UK sample. There were significant differences between the two samples in rates of adherence for individual processes of care for both diabetes and CAD (see Tables 2 and 3). For diabetes indicators, practices in the UK had a significantly higher average percentage of patients meeting three of the five processes of care review criteria while practices in the USA had a higher average percentage of patients achieving HbA1c and BP targets (see Table 2). For CAD indicators, UK practices had a significantly higher average percentage of patients meeting two of the three processes of care review criteria while differences between the two samples in attainment of intermediate outcome targets were not significant (see Table 3).

TABLE 1.

Patient and practice characteristics

	USA (2003–2004)		UK (2003)
	Diabetes	CAD	Diabetes	CAD
Patients
Mean age, years (SD)	60.1 (14.3)	67.6 (13.1)	62.6 (10.8)	68.4 (9.8)
Number of male patients	434	416	262	301
Number of female patients	449	298	242	186
Total number	883	714	504	487
Practices
Total number	55	51	42	42
Average number of charts audited per practice	16.1	14.0	12	11.6

TABLE 2.

Mean practice rates (SDs) of processes of care or achievement of intermediate outcomes for patients with diabetes by country

Average rates	USA		UK		P value
Mean, % (SD)	95% CI	Mean, % (SD)	95% CI
Processes of care within appropriate time frame
HbA1c recording (last 12 months)	87.5 (15.9)	85.4–92.0	92.7 (8.7)	89.9–95.1	0.0769
BP recording (last 12 months)	94.7 (14.1)	94.2–97.8	94.6 (11.6)	93.1–97.8	0.8014
Cholesterol recording (5 years)	94.1 (7.8)	93.1–96.3	97.6 (5.0)	95.8–98.8	0.0232
Weight and height recorded	55.0 (43.3)	37.7–80.3	77.0 (26.0)	74.1–89.6	0.0404
Smoking status (ever recorded)	49.1 (37.0)	37.3–68.8	88.5 (18.3)	87.8–96.0	<0.0001
Attainment of intermediate outcomes
HbA1c < 7.4	54.4 (17.4)	50.4–59.4	36.7 (14.7)	32.3–41.3	<0.0001
BP, systolic <140 and diastolic <85	50.6 (18.8)	45.5–54.7	33.9 (17.4)	28.5–39.2	<0.0001
Cholesterol <200 mg/dl	57.6 (16.2)	54.6–62.8	57.9 (16.7)	52.8–63.1	0.8233

95% CIs obtained from hierarchical logistic regression model.

TABLE 3.

Mean practice rates (SDs) of processes of care or achievement of intermediate outcomes for patients with CAD by country

Average rates	USA		UK		P value
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI
Assessment within appropriate time frame
BP recording (last 12 months)	69.1 (20.9)	64.2–75.0	90.2 (13.1)	88.0–94.7	<0.0001
Cholesterol recording (5 years)	86.6 (17.4)	85.2–91.7	88.4 (15.9)	86.2–93.8	0.4758
Smoking status (ever recorded)	49.1 (39.3)	35.9–68.4	87.3 (17.3)	85.1–94.0	<0.0001
Attainment of intermediate outcomes
BP, systolic <140 and diastolic <85	36.9 (23.5)	29.8–42.1	43.0 (17.4)	38.5–48.6	0.0605
Cholesterol <200 mg/dl	59.8 (24.4)	55.0–67.9	61.3 (23.7)	55.3–70.1	0.7488

Using a hierarchical proportional odds model, we found significant differences between the two samples. After adjusting for potential patient-level confounders (age and gender), practice-level means of counts of processes of care measures achieved were significantly higher in UK practices for patients with either diabetes or CAD. Thus, the odds of a UK patient with diabetes receiving a higher number of recommended services were 8.94 times that of a patient in a US practice. For CAD, the odds of a UK patient receiving a higher number of recommended services was 9.18 times that of a patient in a US practice. Differences in the recording of height–weight and smoking status between the two samples are responsible for these relatively high odds ratio estimates (see Tables 2 and 3). In fact, if we consider just smoking status alone, the odds of recording the smoking status of a patient in a UK practice is 11.41 (95% CI: 4.76–27.36) and 8.62 (95% CI: 3.75–19.86) times the odds of a patient in a US practice, for diabetes and CAD, respectively. Practice-level mean scores for attainment of intermediate outcomes for patients with diabetes were significantly higher in US practices (see Table 4).

TABLE 4.

Practice-level means (SDs) of services provided or outcomes attained for diabetes and CAD in USA and UK

	USA	UK	UK versus USA
	Mean (SD)	Mean (SD)	Proportional odds ratio	95% CI	P value
Diabetes
Processes (maximum = 5)	3.80 (0.67)	4.50 (0.52)	8.94	4.26–18.74	<0.0001
Intermediate outcomes (maximum = 3)	1.63 (0.34)	1.29 (0.27)	0.50	0.39–0.64	<0.0001
CAD
Processes (maximum = 3)	2.05 (0.44)	2.66 (0.32)	9.18	5.22–16.17	<0.0001
Intermediate outcomes (maximum = 2)	0.97 (0.35)	1.04 (0.32)	1.25	0.88–1.78	0.2214

Proportional odds of an increased number of processes delivered or intermediate outcomes attained in UK practices as compared to US practices.

Discussion

Better recording of processes of care in the UK compared with the US sample suggests that both the more coordinated health system in the UK and investments in primary care prior to the QOF may have encouraged a more systematic approach to documenting chronic illness care than in comparable US practices. These differences between the two samples may be attributable to the fact that, in the UK but not in the USA, GPs have responsibility for a defined patient population and serve as gatekeepers to secondary care. The fragmentation of care in the USA that results from this distinction may mean that some of the differences in processes of care observed here are due to a fragmented documentation process. For example, patients may have their smoking status recorded while visiting a subspecialist such as an endocrinologist and these results may not be communicated to their primary care physician. While this could lead to overstatement of the differences in this measure between the two samples, it is, nonetheless, an indicator of the potential health costs of a fragmented approach to care since primary care physicians could, for example, provide additional support for smoking cessation efforts only if they are aware of the smoking status of their patient. However, the fact that patients in US practices were more likely to meet intermediate outcome guidelines for diabetes care suggests that better documentation is not sufficient.

The persistent gaps in chronic illness care quality identified here point to areas of potential improvement in both health systems. In the USA, gaps in documentation of processes of care, especially related to recording of smoking status among patients at elevated cardiovascular disease risk, represent significant missed public health improvement opportunities. In both samples, documentation of patient weight and height was relatively poor. With the increasing incidence of obesity in both countries and the significant association between overweight or obesity and clinical outcomes for both diabetes and CAD, this documentation will be increasingly important to appropriate clinical management. Although the gaps identified here may not reflect actual practice (e.g. physicians or nursing staff may check patient height and weight but fail to record them), documentation failures place patients at risk for not receiving appropriate health services. Further, such documentation gaps may contribute to clinical inertia or the failure to appropriately intensify treatment for patients with chronic illness since patterns of poor control may be missed if screening and test results are not properly recorded and monitored over time. Moreover, documentation gaps represent potential increased health system costs if, for example, laboratory testing must be repeated due to inadequate reporting or recording of results. In the UK sample, we found that prior to implementation of the QOF but following extensive investment in quality improvement capacities, better processes of care and lower rates of quality variation but not better intermediate outcomes were achieved in comparison to US practices assessed here. In both of our samples, achievement of outcome targets showed considerable room for improvement suggesting that, while the approach of the UK practices was more systematic as would be expected given prior investments, future NHS efforts should focus on improving patient outcomes.

The main limitations of this study derive from the fact that while UK practices were chosen to be a representative sample US practices could not be similarly selected for evaluation. Nonetheless, the independent primary care practices in the US sample are typical of the settings where the majority of such care is provided in the USA and our US findings are similar to those derived from representative patient-level samples.1 Constructing a representative sample of US primary care practices would require a nationwide database of such practices and while such a database is available for the UK no comparable US database exists. Development of such a monitoring capacity will be essential for monitoring the effect of P4P or other health system reforms in the USA. Since the US sample is made up of private primary care physician practices, which typically do not serve uninsured patients, the quality of chronic illness care among members of the large uninsured population in the USA is likely underrepresented. Since uninsured patients and those without a usual source of care in the USA have poorer health outcomes than insured patients,26,27 the patients represented in the US sample are likely healthier than the general population of US patients.28 Although a few members of disadvantaged groups in the UK may not register for care despite having full access to health care under the NHS, UK patient panels are likely more representative of the general population of potential patients than the US panels examined here. Patients in the UK sample may also have been somewhat sicker than those in the US sample since UK, but not US, patients were selected only if they were being treated with medication rather than only lifestyle modification. The cross-sectional nature of these data makes it impossible to discern trends in either sample. Finally, comparing US to UK primary care physicians presents challenges since, given system-level differences in coordination of care, their role in primary patient care is somewhat different. Nonetheless, comparisons of primary care quality in the two systems can inform debates about which approach is better for patient health.

In the years prior to the collection of the data examined here, the US and UK primary care environments were quite different. In the UK, there had been significant and sustained investment in quality improvement initiatives focused on primary care, with notable efforts to improve quality and access to care. These included limited financial incentives to improve preventive health care (immunization and cervical cytology) since 1990 and some limited payments to support the establishment of practice-based clinics for chronic disease management since the mid-1990s. In addition, investments were made in teams of care providers, support for linkages between clinical practices and community resources, and clinical infrastructure including major financial support for the provision of EMRs (50% of the costs reimbursed by the government before 2004 and 100% after 2004).29 Much of the UK evaluation efforts coupled with these investments were focused on documenting and improving processes of care. Primary care practices in the USA have not generally enjoyed similar investments except in some integrated delivery networks. FFS, the predominant US payment mechanism in primary care, rewards physicians for the number of visits rather than for care coordination, team-based care, community health or investment in health information technologies.30 In the USA, FFS payment mechanisms have been found to be associated with poorer adherence to recommended processes of diabetes care and may also lead to relatively poor documentation of unreimbursed services.31 Unlike in the UK, the US health system is neither based on primary care nor does it protect its financing from hospital and subspecialty services. These system-level differences, coupled with the UK investments in primary care, are likely among the determinants of the differences in quality observed here.

Investments in quality improvement over the decade prior to the QOF helped to establish the capacities of UK primary care practices to document care and, with appropriate incentive, to make improvements in quality.17,32 One area of investment illustrates this well. A key element of the prior UK investments was a focus on building the capacity of primary care practices to effectively use EMRs designed for improving the quality of care. In the USA, the effective translation of EMR use into clinical practice for quality improvement has not occurred, perhaps due to the existence of numerous competing platforms, implementation variation and an overriding focus on enhancing billing.33–35 Addressing barriers to quality improvement such as this one is likely to require substantial investment. Given that the USA spends ∼2.4 times as much per person on health care as does the UK, such investments in US primary care are theoretically possible but making them could require shifting resources from other sectors of the US health care system and are likely to be resisted by those benefiting from the current payment environment.36 However, the recently passed American Recovery and Reinvestment Act of 2009 includes incentives to primary care physicians for the ‘meaningful use’ of ‘certified’ EMRs indicating that policymakers see a need for investment in developing the infrastructure of US primary care. Encouraging the type of coordination present in the UK system while improving quality may require a move away from FFS reimbursement in the USA to one, such as P4P, that aligns physician and practice incentives with quality outcomes. However, pushing P4P into practice without adequate preparation could lead to premature rejection of this policy tool and another missed opportunity to reorient the US health care system towards more effective care coordination and preventive service delivery.

In order to cross the quality chasm in the USA, policymakers should P4P by investing in or providing incentives for the development of key quality improvement and documentation capacities of primary care practices. Without such a commitment to realigning the priorities of the US health care delivery system, P4P is likely to lead to results that are not significantly different from the relatively poor ones currently achieved.

Declaration

Funding: National Heart, Lung and Blood Institute, National Institutes of Health, Bethesda, MD (NHLBI R01 HL 70800-01 to B.F.C.).

Ethical approval: Institutional Review Board of UMDNJ-Robert Wood Johnson Medical School.

Conflicts of interest: None.