Incorporating considerations of resources use into grading recommendations

Abstract

Guideline panellists have differing opinions on whether resource use should influence decisions on individual patients. As medical care costs rise, resource use considerations become more compelling, but panellists may find dealing with such considerations challenging

In this last part of a series describing the GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach to making recommendations we will look at how guideline panellists and clinicians can incorporate matters related to the use of resources into recommendations and practice. Clinical recommendations inevitably involve judgments about the allocation of resources, judgments commonly referred to as costs. We will deal with some of the challenges of considering costs, explain reasons for focusing on resource use rather than costs, and discuss how to incorporate considerations of resource use into recommendations.

Cost as an outcome presents special challenges

In one sense, cost is just another potentially important outcome—like mortality, morbidity, and quality of life—associated with alternative ways of managing patients. In addition to these clinical outcomes, an intervention may increase costs or decrease costs. However, costs differ from other outcomes in several ways (box).¹

In what way do costs differ from other health outcomes?

• Patients receive health benefits and bear the burden of adverse health outcomes, but healthcare costs are typically shared by society as a whole (as represented by the government), employers, and patients

• Attitudes differ as to whether costs should influence a doctor’s decision about treating individual patients

• Healthcare costs may vary widely among and even within jurisdictions and quickly change over time

• What societies can purchase if they forego use of healthcare resources (opportunity cost) varies widely between countries. A year’s supply of an expensive drug may pay a nurse’s salary in the US and 30 nurses’ salaries in China

• When healthcare expenditures demand foregoing expenditures elsewhere, attitudes differ as to whether the health system, public expenditures, or society as a whole should bear the burden

• Matters relating to resource use are highly political and may result in conflict of interest for a guideline panel (for example, panellists may have an association with industry or government)

Firstly, for most outcomes other than costs, it is clear who gains the advantages and has to live with the disadvantages—the patient and, secondarily, his or her family. This is not true of all outcomes. Consider vaccinations, in which the entire community benefits from the herd effect, or widespread use of antibiotics that may have downstream adverse consequences of drug resistance. It is costs, however, where the matter of who pays and who gains is most prominent. Healthcare costs are often borne by society as a whole, or in a complex arrangement that involves the patient, an employer, and an insurer. Even within a society, who bears the cost may differ depending on the patient’s age (for example, under or over 65) or situation (for example, whether the patient is receiving social assistance).

Summary points

• Costs differ from other healthcare outcomes—costs are shared by patients, employers, and society, and opinions differ as to who should bear the burden; some people think costs should not influence doctors’ decisions; costs differ across and within jurisdictions

• A balance sheet should inform judgments about whether the net benefits are worth the incremental costs

• Evidence profiles should always present resource use, not just monetary values

• A guideline panel may legitimately choose to omit costs as a consideration

• Formal economic modelling may—or may not—be helpful

Secondly, depending on who bears the cost, opinions on the extent that costs should influence the decision differ. If costs are borne by a third party payer (the government or another third party payer) some people would argue that the doctor’s responsibility to the patient means that costs should not influence the decision. On the other hand, a clinicians’ responsibility when caring for a patient is discharged in a broader context—resources that are used for an intervention cannot be used for something else and can prevent the health system from best meeting the needs of those it serves.

Thirdly, costs tend to vary widely across jurisdictions. Costs of drugs are largely unrelated to the costs of producing them but more to marketing decisions and national policies. Most medicines under patent, for example, cost around twice as much in the United States as in Canada. Furthermore, costs may vary widely even within jurisdictions. Hospitals or health maintenance organisations may, for instance, negotiate special arrangements with drug companies for prices substantially lower than are available to patients or other providers. In addition, costs can vary widely over time (for example, when a drug comes off patent).

Even when resource use remains the same, the resource implications may vary widely across jurisdictions. A year’s supply of a very expensive drug may pay a nurse’s salary in the US, six nurses’ salaries in Poland, and 30 nurses’ salaries in China. Thus, what can be bought with the resources saved if a certain drug is not purchased (the “opportunity cost”)—and the health benefits achieved with those funds—will differ greatly.²

Lastly, people may have different ideas about which “envelope” opportunity costs fall into. A hospital pharmacy with a fixed budget considering purchase of an expensive new drug will have a clear idea of what that purchase will mean in terms of other drugs the pharmacy cannot afford. In broader settings the opportunity costs will be less clear. For instance, funding a new drug or programme will constrain resources for other public health expenditures, but will refraining from that purchase really mean the equivalent resources are available for our healthcare system? People often assume the envelope from which health care costs must come is public health spending, but might refraining from tax reductions or reducing military expenditure be legitimate alternatives for making an effective but expensive drug widely available?

All of the points mentioned up to now make the consideration of resource allocation a far more political issue than consideration of other outcomes. Whether the panel does or does not explicitly consider matters related to resource allocation, those politics may bear on a guideline panel’s function through conflict of interest. Conflicts may arise through the association of panel members with commercial entities, the government, or other third party payers.

A balance sheet should be used to assess benefits versus costs

Despite the differences, approaches to resource use are similar to other outcomes because guideline panels need consider only important resource implications; decision makers require an estimate of the difference between treatment and control; and panels must make explicit judgments about the quality of the evidence regarding incremental resource use. Special considerations of using resource use as an outcome include which perspective to take, which resource implications to include, how to make judgments about the quality of the evidence, how to present these implications, and the potential usefulness—or lack of usefulness—of a formal economic model in arriving at a wise recommendation.

A balance sheet is a simple but powerful way to present the advantages and disadvantages of the management options under consideration.³ We illustrated this in a previous article in this series where we used an evidence profile, a type of balance sheet. We did not consider resource implications in that example. Tables 1 and 2 show an evidence profile informed by a large international clinical trial⁴ and an associated economic analysis⁵ to assess the advantages and disadvantages of magnesium sulphate in women with pre-eclampsia.

Table 1 .

Summary of findings on whether clinicians should use magnesium sulphate to prevent eclampsia: clinical outcomes

Outcome	Severity of pre-eclampsia	Typical control group risk	Typical absolute effect (95% CI)	Relative risk (95% CI)	No of participants	Quality of evidence
Eclampsia	Severe*	27/1000	16 fewer/1000 (11 to 19)	0.41 (0.29 to 0.58)	11 444	High†
	Not severe	15/1000	9 fewer/1000 (6 to 11)
Maternal death	Severe	6/1000	3 fewer/1000 (0.6 more to 4 fewer)	0.54 (0.26 to 1.10)	10 795	Moderate‡
	Not severe	3/1000	1 fewer/1000 (0.3 more to 2 fewer)
Side effects§	Severe and not severe	46/1000	196 more/1000 (165 to 231)	5.26 (4.59 to 6.03)	9992	High†

*Severe eclampsia was defined as (diastolic blood pressure >110 mm Hg on two occasions, or systolic blood pressure >170 mm Hg on two occasions and proteinuria >3+) or (diastolic blood pressure >100 mm Hg on two occasions, or systolic blood pressure >150 mm Hg on two occasions and proteinuria >2+ and at least two signs or symptoms of imminent eclampsia) or for women who had an antihypertensive in the 48 h before randomisation: (in 48 h before trial entry, highest diastolic blood pressure >110 mm Hg, or highest systolic blood pressure >170 mm Hg and proteinuria >3+ at trial entry) or (in 48 h before trial entry, highest diastolic blood pressure >100 mm Hg, or highest systolic blood pressure >150 mm Hg and proteinuria >2+ and at least two signs or symptoms of imminent eclampsia).

†Evidence comes from randomised trials and there was no reason to grade down for study limitations, imprecision, inconsistency, indirectness, or publication bias.

‡The confidence interval was wide so the evidence was graded down for imprecision.

§Mostly flushing. Other side effects include nausea, vomiting, slurred speech, muscle weakness, dizziness, drowsiness, confusion, and headache.

Table 2 .

Summary of findings on whether clinicians should use magnesium sulphate to prevent eclampsia: resource use viewed from the perspective of the health system

Resource	Cost*	Typical absolute effect (95% CI)	No of participants (studies)	Quality of evidence	Comment
Magnesium sulphate ampoules (6×10 ml ampoules/patient)
Setting:
High income countries	$20 more/patient		9996	High†
Middle income countries	$3 more/patient
Low income countries	$5 more/patient
Administration of magnesium sulphate (1 ampoule/patient)
Setting:
High income countries	$66/patient		9996	High†	Resources for giving magnesium sulphate included midwives’ time (main cost), intravenous cannula or needles, syringes, intravenous fluids, and the drug
Middle income countries	$14/patient
Low income countries	$8/patient
Other hospital resources (varied widely)
Setting:
High income countries	$12 839	$20 less/ patient ($0 to $60)	9.996	Moderate‡	Use of other hospital resources varied greatly in both intervention and control groups. Other hospital costs have been adjusted for on the basis of the influence of eclampsia to control for the many other factors that influenced these costs
Middle income countries	$1 416	$4 less/ patient ($0 to $10)
Low income countries	$157	$2 less/ patient ($1 to $3)

*$1=£0.5=€0.7.

†Evidence comes from randomised trials and there was no reason to grade down for study limitations, imprecision, inconsistency, indirectness, or publication bias.

‡The confidence interval was wide so the evidence was graded down for imprecision.

Evidence profiles should present resource use, not just monetary values

We suggest that guideline developers should document best estimates of resource use, not best estimates of costs. Costs are a function of resources expended and the cost per unit of resource. Given the wide variability in unit costs, reporting only total costs across broad categories of resource expenditure does not provide users with the information needed to judge whether estimates of unit costs apply to their setting.

Users of guidelines will be best informed if the guideline developers specify resources consumed by alternative management strategies. Firstly, they can judge whether the resource use reflects practice patterns in their setting. Secondly, they can focus on the items of most relevance to them (such as the drug costs for a pharmacy or the hospital costs for a hospital administrator). Thirdly, they can, if monetary values are subsequently assigned to the resources used, ascertain whether the unit costs apply in their setting. If they don’t, they can substitute unit costs that do.

The magnesium sulphate example (tables 1 and 2) shows the importance of documenting resource use and specifying the setting. The variation in costs associated with magnesium sulphate, its administration, and the associated hospital costs across countries with high, medium, and low gross national income is great. Table 2 documents these differences, but many economic analyses will not. Unless resource use is specified, users in settings other than that on which the analysts focus cannot estimate the associated incremental costs of the intervention.

The specific context is crucial for considering resource use

The extreme variability in costs over time and jurisdictions has several implications. Firstly, we have argued in previous articles in this series that clinical practice guidelines should be as specific as possible about the patient population, the nature of the intervention, the comparator, and the healthcare setting; this specificity is even more crucial when bringing cost into the equation. The choice of comparator can be a particular problem in economic analyses. If the choice of comparator is inappropriate (such as no treatment rather than a less effective treatment) conclusions may be misleading.⁶

Secondly, a guideline panel may legitimately not consider resource use and make recommendations solely on the basis of other advantages and disadvantages of the alternatives being considered. Resource allocation must then be considered by the ultimate decision maker—the patient and healthcare professional, an organisation such as a hospital pharmacy or a health maintenance organisation, a third party payer, or a government. Guideline panels should be explicit about this decision.

Thirdly, if panellists do consider resource use they should—before bringing cost into the equation—decide on the quality of evidence regarding other outcomes and weigh up the advantages and disadvantages. Decisions regarding the importance of resource use will flow from this first step. For example, resource implications may be irrelevant if evidence of net health benefits is lacking. If advantages of an intervention far outweigh disadvantages, resource use is less likely to be important. Resource use usually becomes important when advantages and disadvantages are closely balanced.

Therefore, the GRADE Working Group considered the possibility of panels issuing two recommendations, one not taking resource use into account and a second doing so. This would have the advantage of explicitness, on which GRADE places great value. We were concerned, however, that those with commercial interests in dissemination of an intervention (such as a drug company) would use the recommendation ignoring resource implications as a weapon in their battle for funds—particularly public funds. To prevent this undesirable outcome, GRADE decided that panels considering resource use should offer just one recommendation.

A broad perspective is desirable

Conceivably, a recommendation could be intended for a very narrow audience, such as a single hospital pharmacy, an individual hospital, or a health maintenance organisation. Alternatively, it could be intended for a health region, a country, or an international audience.

Regardless of how narrow or broad the intended audience, guideline groups that choose to incorporate resource implications must be explicit about the perspective they are taking. For example, a guideline panel that took the individual patient’s perspective would consider only resource implications that directly affect individual patients (such as out of pocket costs) and would exclude resource implications for others (such as costs borne by the government). In European healthcare systems in which governments mostly bear the cost of health care, expenses borne directly by patients are usually minimal. Alternatively, a guideline may choose to take a societal perspective, and include all important resource implications, regardless of who bears the costs.

In most circumstances, few guideline audiences would be happy with a perspective that is narrower than that of the entire health system. For instance, in a publicly funded health system the patient perspective would ignore most of the costs generated. A pharmacy perspective would ignore downstream cost savings resulting from adverse events (for example, stroke or myocardial infarction) prevented by a drug, and a hospital perspective would ignore outpatient costs, either incurred or prevented.⁷ In the private sector, where disenrollment and loss of insurance can shift the burden of costs from one system to another, estimates of resource use should include the downstream costs of all treated patients, not just those who remain in a particular health plan.

An even broader viewpoint—that of society—is the most comprehensive perspective because it includes all costs, regardless of who bears them. This perspective is often preferable, particularly if the health intervention has a broad effect (for example, an intervention for heart failure that improves patients’ functioning and reduces the time and cost of family caregivers). Whether cost effectiveness analyses should include the implications of health effects—such as changes in earnings—is more controversial. Economic guidelines advise that such implications are presented separately, not as part of the formal cost effectiveness analysis.

Clinicians seeing patients who are not covered by either public or private insurance may need to help them make decisions that take into account their out of pocket costs. This is particularly true when clinical advantages and disadvantages are closely balanced and out of pocket costs are substantial. In these circumstances, if a guideline panel has used the GRADE approach, clinicians can review evidence summaries such as those in tables 1 and 2. It is then the clinicians’ responsibility—either through communicating the information directly to the patient or by finding out about the patients’ situation, values, and preference—to ensure that the patient’s decision about drug consumption is consistent with his or her values and preferences.

Judging quality of evidence for resource use

As with evidence of rare but serious adverse effects, evidence of resource use may come from sources other than evidence of health benefits. This may be because trials of interventions do not fully report resource use; because the trial situation may not fully reflect the circumstances—and thus the resource use—that we would expect in clinical practice; because the relevant resource use may extend beyond the duration of the trial; or because resource use may vary greatly across settings.

For resource use reported in the context of trials, criteria for quality assessment are identical to those for other outcomes, as described in the second article in this series.⁸ As for other outcomes of a trial, the quality of evidence may differ across different resources. For example, drug use may be relatively easy to estimate, whereas use of health professionals’ time may be more difficult, so the estimate of drug use may therefore be of higher quality. For magnesium sulphate in pre-eclampsia, we are more confident of resource use associated with the drug itself and administration of the drug than we are of use of hospital resources (table 2).

Formal economic modelling may—or may not—be helpful

Formal economic modelling results in cost per unit benefit achieved: cost per natural unit, such as cost per stroke prevented (cost effectiveness analysis); cost per quality adjusted life year gained (cost-utility analysis); or both cost and benefits valued in dollars (cost-benefit analysis). These summaries can help inform judgments. Unfortunately, published cost effectiveness analyses, particularly of drugs, have a high probability of being flawed or biased,⁹ and they are specific to the particular setting.

Guideline groups may therefore consider developing their own formal economic model. Before guideline groups can consider this option, however, they must have the necessary expertise and resources. The larger the difference in resources consumed by the alternative management strategies, the greater the uncertainty about whether the net benefits of an intervention are worth the incremental costs, and the higher the quality of evidence regarding resource consumption, the more likely it is that a full economic model would help inform a decision. Creating an economic model may also be advisable if implementing an intervention requires large capital investments, such as building new facilities or buying expensive equipment.

Modelling is necessary for taking into account complexities and uncertainties in calculating cost per unit benefit, but it does reduce transparency. In addition, any model is only as good as the data on which it is based. When estimates of benefits, harms, or resources used come from low quality evidence, results of any modelling exercise will be highly speculative.

Criteria are available to assess the credence to give to results from statistical models of cost effectiveness or cost-utility.^{10 11 12 13} However, these models generally include a large number of assumptions and evidence of varying quality for the different estimates that are included in the model. For these reasons, we do not recommend including cost effectiveness or cost-utility models in evidence profiles. These models may, however, inform a guideline panel’s judgments, or those of governments or third part payers considering whether to include an intervention among their programmes’ benefits.

Tables 1 and 2 provide crucial information for making estimates of the incremental cost per episode of pre-eclampsia prevented for severe and non-severe pre-eclampsia in high, middle, and low income countries (table 3). Even when—as in this case—cost effective estimates are credible, they provide no clear answers about appropriate action. Most people, however, would consider the cost per episode of eclampsia prevented to be worth the money in the case of severe pre-eclampsia. For non-severe pre-eclampsia—particularly in low income countries—the decision is more difficult. Ultimately, decision makers must weigh the relative value of preventing eclampsia against the benefits that the health system, or society, would forego in allocating resources to magnesium sulphate administration.

Table 3 .

Incremental cost for each episode of eclampsia prevented by magnesium sulphate

National income	Severity of eclampsia
	Severe	Non-severe
High	$4125	$7333
Medium	$813	$1444
Low	$688	$1222

*$1=£0.5=€0.7.

Concluding remarks

Clinical decision making is complex. Guidelines have the potential to help clinicians and patients with complex choices, to improve the quality of care, and to help ensure the best use of limited resources. To ensure that guidelines inform rather than misinform, they should build on the best available evidence, and guideline panels should use systematic and transparent processes to make judgments about the quality of the evidence,⁹ moving from the evidence to a recommendation,¹⁴ and incorporating considerations of how resources are used.

Clinicians and their patients will be best served by guidelines that use an approach, such as the one we have described in this series, to explicitly grade the quality of evidence and the strength of recommendations. Front line clinicians or those constructing local guidelines need not replicate the work done by well resourced guideline developers. However, to make the best use of guidelines, they need an understanding of the evidence and judgments underlying a guideline. They should have access to concise summaries of recommendations, including ratings of the quality of the underlying evidence and the strength of the recommendation, and they should understand the meaning of these grades and their implications for clinical decision making.