Use of risk adjustment in setting budgets and measuring performance in primary care II: advantages, disadvantages, and practicalities

Risk adjustment could help to improve decisions about budgets as well as help measure the performance of doctors. In the first of these two articles we discussed why risk adjustment could become more important in the United Kingdom and how it works in the United States.¹ In this article we consider the benefits and problems of risk adjustment and assess how one US system would perform in the United Kingdom.

Summary points

• Use of risk adjustment in the United Kingdom could help ensure that general practices and primary care trusts are not penalised for taking on patients with complex health needs

• Risk adjustment methods may also help ensure that computerised clinical records in primary care are complete and accurate

• Risk adjustment could add to the administrative complexity of healthcare systems

• It may draw attention away from the overall level of healthcare spending

• There is no gold standard method of risk adjustment

• Use of risk adjustment in setting budgets and monitoring performance should be explored further

What are the uses of risk adjustment?

In the United States, risk adjustment is starting to be used to adjust capitation or other types of payments to healthcare providers such as family practices, multispecialty medical groups, or consortiums of physicians and hospitals (integrated delivery systems).² For large populations (such as that of a broad geographical area), age, sex, and ecological measures may be adequate for this purpose. But for smaller populations, such as those managed by one family practice or a small consortium of physicians, risk adjustment helps ensure that providers who manage patients with more complex medical problems have their budgets adjusted to take this into account (box). The use of risk adjustment systems has also given doctors and health maintenance organisations a powerful incentive to provide more accurate and complete diagnostic data.¹

Potential advantages of risk adjustment

• Reduces the risk of adverse selection

• Provides information useful for planning and monitoring health services

• May lead to fairer methods of resource allocation for health services

• May help reward physicians and healthcare providers who treat more complex cases

• Reduces the pressure on physicians to limit the health services provided to patients on financial grounds

• Provides case mix adjusted measures of performance of physicians and healthcare providers

• Provides measures of the health status of primary care populations

• Encourages physicians and healthcare providers to ensure that clinical records are complete and accurate

• Should help improve the process of clinical governance

• Can add to the value of NHS investment in information and communication technology

The second important use of risk adjustment is to adjust for case mix when comparing practice patterns across providers.³ For example, the NHS plan states that it “promises better performance and accountability systems to reduce variations in service across England.”⁴ The performance indicators published by the Department of Health show wide variation in performance among doctors in both primary and secondary care.⁵ However, as they do not take into account differences in case mix, we do not know how valid such indicators are as measures of clinical efficiency and efficacy. Risk adjustment can help correct such variations for underlying differences in population case mix and thus could lead to fairer and more accurate performance measures for providers.

Another use of risk adjustment is to measure the health of a population.⁶ The traditional way of doing this has been to use death rates or self reported measures of chronic illness derived from censuses or surveys of the population. Because many important chronic illnesses do not result in death, mortality is not always a good measure of the health of a population. Similarly, a generic self reported measure of chronic illness may be an unreliable measure of health status. Risk adjustment models based on all conditions treated by primary and secondary care physicians can take into account the full spectrum of illnesses in a population. Hence, they include chronic illnesses such as arthritis or epilepsy, which cause considerable population morbidity but are often not recorded on death certificates, as well as illnesses such as ischaemic heart disease.

What are the limitations of risk adjustment?

Risk adjustment methods are relatively new and have only recently started to be used to adjust capitation payments in the United States. Hence, we have limited knowledge about whether risk adjustment improves the equity of resource allocation or the efficiency of health services (box).⁷ When applied retrospectively, as they would be when used to compare primary care physicians, they explain 30-50% of the variation in physician performance. However, when the models are applied prospectively, as they would for capitated payments, they explain less than 10% of the variation in future healthcare costs after adjustment for age and sex.

Potential disadvantages of risk adjustment

• Adds to the administrative complexity of healthcare systems and may increase spending on administration and management

• Leaves a large proportion of differences in healthcare spending unexplained

• Not adequate in explaining use of health services by patients with high cost disorders

• Redistribution of resources between healthcare providers could lead to winners and losers

• Focuses attention on developing fairer methods of redistributing resources and may decrease attention paid to whether overall healthcare spending is sufficient

• May encourage the further fragmentation of healthcare systems and a loss of the population focus when planning health services

• Current approaches do not use information on patients' social and cultural circumstances, which could be important predictors of use of healthservices

• Different methods of risk adjustment may give very different results

Risk adjustment models are based on data derived from computerised patient records or from the billing records of insurance companies. They therefore suffer from problems such as incomplete or inaccurate coding of diagnostic data. The models, which have a bias to create estimates clustered around the mean, are also poor at dealing with “outlier” cases (patients with very high healthcare costs or with unexpected healthcare needs) such as patients who need organ transplantation, HIV positive patients who are prescribed expensive antiretroviral drugs, and patients who suffer serious injuries. Patients with such disorders require special funding arrangements and are usually excluded from budgets or performance reports when risk adjustment methods are applied to a general population.⁸,⁹

Another potential problem associated with risk adjustment is that of “upcoding.” This arises when providers use diagnoses that result in their patients appearing to have more complicated illnesses than is really the case in order to benefit from additional resources or improve their ratings on case mix adjusted measures of performance. Furthermore, because patients in the United States frequently move between health maintenance organisations and health plans, the risk adjustment models are often based on data on use of health care for a limited period (one year in many cases).

Another caveat about risk adjustment is that it developed partly as a response to greater fragmentation and increased competition in the US healthcare industry.¹⁰,¹¹ Although risk adjustment may have a role in the United Kingdom, it is important not to repeat the mistakes made in the United States, such as the substantially greater spending on administration of health services. Hence, any introduction of risk adjustment in the United Kingdom should be driven by the need to improve the fairness of current methods of allocating resources and of improving equity within the NHS.

Despite these limitations, risk adjustment seems to do as well as other measures of health status in explaining variations in performance that may arise from differences in case mix. For example, in a study by Fowles and colleagues, one case mix adjustment method, aggregated diagnostic groups, predicted spending on health care better than information derived from the short form health status survey (SF-36), a commonly used and well validated instrument for measuring health status in individuals.¹²

Could risk adjustment models be used in the United Kingdom?

The NHS in England is shifting from a health funding system based on allocating budgets to large (typically around 500 000 patients) and geographically well defined populations of health authorities to funding based on the much smaller and less well geographically defined populations of primary care trusts and general practices. Differences in the health characteristics of patients among these groups are likely to be greater than that seen among health authorities. Hence, methods of funding that take into account differences in case mix will increasingly be needed. Methods are also needed that are capable of producing measures of performance adjusted for case mix. This is particularly the case for general practices, where age, sex, and ecological measures are not sufficient to measure the clinical characteristics of their populations. This leads to, for example, large variations in prescribing rates and costs, and disputes between general practitioners and prescribing advisers about the fairness of prescribing budgets. The use of risk adjustment could help reduce such problems by providing better measures of the health status and clinical characteristics of practice populations.

US methods of population based diagnostic risk adjustment have never been applied in the United Kingdom. Hence, some research is needed to assess how well these methods can be adapted for use in the United Kingdom and how effective they are at explaining differences in use of health care or performance of providers.

One difficulty in applying risk adjustment methods in the United Kingdom is that we do not have computerised diagnostic information on the care provided to patients in all healthcare settings. Although data are available for hospital admissions, there is no information on outpatient consultations or from general practices that are not fully computerised. However, the NHS is committed to developing fully computerised medical records, integrated between primary, community, and secondary care, over the next few years.¹³

Finally, general practices in the United Kingdom generally use Read codes to classify clinical data whereas all the US risk adjustment programmes use International Classification of Diseases (ICD) codes. Hence, either the US risk adjustment programs would have to be adapted to use Read codes or the diagnostic codes in clinical systems in primary care will have to be converted to ICD codes.

Application of risk adjustment to UK population

We investigated the feasibility of applying the Johns Hopkins adjusted clinical group case mix system in the United Kingdom using data from the fourth national morbidity survey.¹⁴ This was a prospective one year study of all consultations with general practitioners in 60 general practices in England and Wales. Diagnostic information on all consultations was entered directly on to computer by the participating general practitioners. The study population comprised a 1% sample of the population of England and Wales and was representative of the population for age, sex, social class, and housing tenure. A validation survey carried out at the end of the study confirmed the accuracy and completeness of the data collected. All diagnoses made during the survey were converted from Read to ICD-9 codes.

Of the patients in the survey, 72% (365 209) consulted at least once. In total, there were 1 273 051 separate ICD-9 diagnostic codes, of which just under 99% could be assigned an aggregated diagnostic group; 14 667 (1.15%) ICD-9 codes could not be assigned an aggregated diagnostic group, a similar percentage to that seen in North American studies. The mean number of aggregated diagnostic groups per patient was 2.0, and the mean number of major aggregated diagnostic groups per person was 0.2. In total, 17% of patients in the survey had at least one major aggregated diagnostic group. Patients with several aggregated diagnostic groups or one or more major diagnostic groups are likely to have an important influence on practice workload and use of health services.

The table shows the breakdown of the distribution of the total number of aggregated diagnostic groups and number of major aggregated diagnostic groups per person in the morbidity survey. Also shown are comparative data from enrolees of two large health insurance plans in the United States. One set of comparative data is from the billing database of a fee for service managed care population in a midwestern state. The second is from a commercial point of service health maintenance organisation in a northeastern state.

The distributions of aggregated diagnostic groups are similar for all three populations, although the US populations have a higher percentage of patients with ⩾5 aggregated diagnostic groups and ⩾3 major groups. This may reflect differences in medical practice between the two countries, information lost in the translation of Read to ICD-9 codes, or more complete recording of diagnostic data in the United States. The preliminary findings of our empirical analysis suggest that US derived risk adjustment methods may work reasonably well in the UK and that further research is warranted.

Conclusions

The ultimate objective of any healthcare system is to improve the health of the public. An important task for the NHS is therefore to ensure that patients, particularly those with complex medical problems that are expensive to manage, gain access to the health services they require to help them lead longer and healthier lives. Risk adjustment models like those now being used in the United States offer the possibility of fairer methods of funding primary care trusts and general practices and fairer methods of comparing the performance of healthcare providers.

One advantage that the United Kingdom has over the United States in developing risk adjustment methods is that the UK healthcare system can provide information on patients' use of both primary and secondary care, and for a longer period, than most American health maintenance organisations or health plans can do. This may allow the United Kingdom to develop more valid methods of risk adjustment and to benefit from the substantial investment being made by the NHS in information and communication technology.

Table.

Distribution of aggregated diagnostic groups in populations drawn from England and Wales and United States

No of aggregated diagnostic groups/ person	England and Wales	Midwest USA	Northeast USA
All groups:
0	27.3	21.5	30.0
1	21.0	16.8	16.8
2	18.0	16.2	15.2
3	13.3	14.2	12.0
4	8.7	10.7	8.7
⩾5	11.4	20.7	17.3
Major groups:
1	14.3	12.0	14.3
2	2.4	2.2	3.0
⩾3	0.5	0.7	1.0