Healthcare rationing—are additional criteria needed for assessing evidence based clinical practice guidelines?

In 1995 the case of “Child B” reached the headlines of British newspapers and stirred public debate about the decision to withhold a second bone marrow transplant from a child with acute myeloid leukaemia.¹,² The decision was based on the weakness of scientific and clinical evidence of the treatment's efficacy. It was also argued that the decision was in accordance with guidelines for patient selection that were already in place for such specialised treatment. A central issue is why judgments such as this are not perceived as legitimate, even when they are based on clinical guidelines. The explanation may lie partly in the fact that the guidelines used have not been developed through a process considered as legitimate. Why should the patient, her parents, or the public accept some little known guidelines developed within the closed communities of medical experts? This issue is valid for all types of clinical practice guidelines. In this article, I examine guidelines as a mechanism for rationing and argue that this mechanism can be improved by involving the patient and the public.

Summary points

• Clinical practice guidelines can be mechanisms for rationing and tools for improving the quality of rationing decisions

• However, additional criteria for assessing the acceptability of evidence based clinical practice guidelines are needed

• Rationing decisions based on guidelines could be acceptable if guidelines are developed through open and fair procedures

• Guidelines used for rationing should be accessible to the public and explicit reasons for recommendations should be provided

Rationing and evidence based medicine

Rationing can be defined as the withholding of potentially beneficial health care through financial or organisational features of the healthcare system in question. The definition is broad enough to encompass the view that withholding of treatment perceived to be beneficial should be seen as a question of rationing.

One of the basic assumptions in published reports on priority setting in health care is that services with no documented effect can be withheld legitimately. The methods of evidence based medicine therefore seem to be natural building blocks in any system of setting fair priorities. The aim is to make the best choice for the patient on the basis of the evidence available.³

Guidelines as rationing tools

Practice policies or clinical guidelines can be thought of as “generic decisions—recommendations intended for a collection of patients rather than for a single patient.”⁴ Evidence based clinical practice guidelines can be viewed as a way of extending the approach of evidence based medicine for the single patient to improving clinical practice for a group of patients. Sackett et al define clinical practice guidelines as “user-friendly statements” for a collection of patients, based on the best external evidence.³

Stakeholders

However, practice guidelines have “users” other than the medical practitioner. Eddy considered that the purpose of a clinical practice policy was “to modify the behaviour of practitioners to steer their decisions toward actions that the policy-makers consider desirable.”⁵ This definition introduces other, legitimate stakeholders into the arena of guideline development. It goes without saying that there are conflicting views about the actions that are considered desirable in relation to these different stakeholders.

Direct role in rationing

Grimshaw and Hutchinson say that guidelines should play a direct part in the rationing process: “Since the rationing of scarce resources requires a targeting of those resources to obtain best value for money, it is important to have mechanisms for assuring effective health care. Clinical practice guidelines offer an opportunity for introducing evidence-based health care into local practice and for influencing the commissioning of effective health care.”⁶ These authors argue that the goal of effective service provision can be achieved by using evidence based clinical practice guidelines as tools for rationing. However, the problem is that guidelines could end up as instruments for unjustified and covert rationing disguised as expert recommendations.

Grey areas

Even when the methods of evidence based medicine are applied, there are abundant grey areas and uncertain indications for treatment remain.⁷ Setting limits within grey areas should not be separated from the issue of rationing. It is in these grey zones of decision making that clinical guidelines have the potential to change the pattern of practice—sometimes with rationing as a by-product.

LIANE PAYNE

Accountability to the public

If clinical guidelines are to have an impact on rationing practices, they must be perceived as legitimate—that is, they must command the respect of patients and society. Public accountability can be achieved by direct and indirect representation of affected parties.² The public cannot, for obvious reasons, participate directly at all levels of decision making. However, few workers have explained how the requirements of accountability apply at the level of decision making discussed here. Theories of deliberative democracy—the idea that legitimate democracy issues from the public deliberation of citizens—offer a basis for developing a set of minimal requirements that the process of rationing should satisfy.^8–11

Democratic deliberation

To simplify the debate, we could say that rationing decisions satisfy the requirements of public accountability if all relevant reasons for a decision are given by those responsible for it to those affected by it. The definition emphasises two key principles—reasons for decisions should be public, and they should be explicit. One relevant formulation of the principles states that the reasons given by officials and citizens to justify political actions, and the information necessary to assess those reasons, should be public.⁹ Explicitness ensures that conflicts between different values or preferences can be explored.⁸ Explicitness or transparency—the disclosure of the rationale and values on which decisions are based—is a precondition for democratic deliberation.

Guidelines and acceptability

In discussing evidence based practice guidelines and acceptability, I start from the following premise. Exclusion and inclusion of patients according to the recommendations of clinical practice guidelines are acceptable and legitimate if—and only if—the method of guideline development and the product itself satisfy some minimal requirements of deliberative democracy.

Although guidelines could easily function as rationing tools, published reports on evidence based medicine do not consider them as such.⁶ The Evidence-Based Medicine Working Group classifies the criteria for evaluating evidence based clinical guidelines into three groups—validity, importance, and applicability.³,¹²,¹³ Although important, criteria for determining applicability need not concern us here.¹⁴ Validity criteria must be satisfied to accomplish the requirements of professional accountability. The criteria for evaluating importance are related to the requirements of economic and political accountability, but have not been fully explored and developed in respect of rationing. This fact is evident from the criteria of importance suggested by the Evidence-Based Medicine Working Group (box).

“The problem is that guidelines might end up as instruments for unjustified and covert rationing disguised as expert recommendations”

Criteria of importance for assessing guidelines³,¹²,¹³

Key question—does this guideline offer an opportunity for appreciable improvement in the quality of healthcare practice?

• Is there a large variation in current practice?

• Does the guideline contain new evidence (or old evidence not yet acted upon) that could have an important impact on management?

• Would the guideline affect the management of so many people, or concerned individuals at such a high risk, or involve such high costs that even small changes in practice could have a major impact on health outcomes or resources (including opportunity costs)?

Criteria of importance

The test of importance (box) is related to the potential impact of the guideline—that is, whether it can reduce local variations in practice, have an impact on management, or have an impact on major outcomes or costs. However, it is worth observing that the impact on the individual is not clearly specified. The implicit perspective is a collective one, as seen by the inclusion of opportunity costs in the third question. The tacit assumption here is that if a guideline is able to have a major impact on the population and shift the pattern of healthcare consumption, it satisfies the criterion of importance. Little is said, or asked, about the distribution of this impact—the distribution of burdens and gains. Nor is anything said about what kind of impact is acceptable for individual patients. This indicates that the criteria of evidence based medicine are not concerned with acceptability.

It might be possible for a given guideline to satisfy the criteria of importance even though the recommendations were entirely unacceptable to the patients affected. Consider a guideline recommending stricter inclusion criteria for coronary surgery. This would have a major impact on reducing treatment costs, but that does not mean that the guideline is acceptable. Another set of criteria is needed. If a guideline does, in fact, recommend rationing, why should the “users” comply with it? If guidelines are to be used as rationing tools, the criteria for systematic evidence based evaluation must be modified to include any legitimate concerns the public has about their use.

Acceptability of guideline procedure

Even if a guideline is valid, important, and applicable, it cannot be considered legitimate if it has not passed the test of acceptability. A guideline should not be implemented if the impact on key stakeholders is not acceptable to those affected by it. Acceptability concerns both the procedure of guideline development and its product. Five criteria for assessing whether the procedure of guideline development is acceptable are given in the box.

Criteria for judging the acceptability of a guideline development procedure

Key question—can the procedure for developing this guideline be considered acceptable?

• Does the methods section of the guideline include information on development?

• Did the guidelines development group include representatives of all key disciplines?

• Was input from patient representatives obtained or were other efforts made to include perspectives of patients and public?

• Has the guideline been the subject of consultation among key stakeholders?

• Are the recommendations influenced directly by economic or political decisions, and are these connections recognised and discussed?

Information and legitimacy

The first question in the box identifies some minimum information necessary to assess acceptability. The second question asks whether other clinical disciplines are represented, and is derived from one of the best appraisal instruments currently in use.¹⁵ Indirect representation of other patients with competing interests and other perspectives might be secured by involving clinicians from different disciplines, and transparency within the clinical community will also be improved by this. The third question introduces the issue of representation of competing interests and perspectives. Procedures that do not include patient and citizen perspectives, either directly or indirectly, are considered to be seriously flawed. Decisions based on guidelines that reflect only the values of doctors or fundholders cannot be regarded as legitimate.

Public and stakeholder participation

The two main arguments for public participation in the process are that it enhances public accountability and that it secures a wider representation of interests so that conflicts between different values or preferences can be explored and considered.¹⁶ There is, however, little experience or evidence showing that public participation can improve the process in this way. Although there are some notable exceptions, public participation in priority setting might be desirable in theory, but is difficult to implement in ways that achieve its goals.^17–22 The criterion suggested in the box is therefore a weak one. It asks whether efforts were made to include patients' and citizens' perspectives, either through direct or indirect representation. Participation of the public and of patients can better be incorporated into the process through wide consultation.The fourth criterion is therefore stronger—it requires that guidelines should be subjected to a wide process of consultation among key stakeholders. This criterion is justified by the principle of publicity.

“Rationing decisions satisfy the requirements of public accountability if all relevant reasons for a decision are given by those responsible for it to those affected by it”

Transparency

The last criterion for guideline development focuses on the need for explicitness. If recommendations are influenced by economic or political decisions, these constraints should be recognised and discussed. This includes considerations of cost effectiveness. A decision to withhold services that might benefit patients marginally but at high costs might be perfectly acceptable when resources are scarce. The point is that these reasons should be owned and not disguised as “clinical” decisions. Political accountability at this level is a complex issue, but if the criteria for inclusion and exclusion reflect the resource constraints of the service in question, this should be made clear.

Appraising the consequences

Apart from the procedure of guideline development, tests of acceptability should also contain an assessment of the information necessary to appraise the consequences of applying the guideline (box). The key issue is whether the information necessary for appraisal is included in the final document.

Transparency

The first question requires that the inclusion and exclusion criteria are transparent and that the rationale behind decisions is stated explicitly. Since guidelines are not normally considered as rationing tools, the criteria for exclusion are sometimes stated vaguely and the true rationale is often omitted. These practices reduce the likelihood and possibility of public assessment, and therefore of legitimacy.

Criteria for acceptability of the information provided

Key question—is the information needed for guideline appraisal provided?

• Are the inclusion and exclusion criteria transparent, and is their rationale stated explicitly?

• Is this information accessible to all key stakeholders in written and understandable form?

• Are the inclusion or exclusion criteria discussed and justified with reference to:Medical criteria?Costs and opportunity costs?Non-medical criteria such as age, productivity, social status, gender?

• Are the reasons for exclusion and inclusion stated in a form that can be recognised as valid and relevant?

Accessibility

Correspondingly, the second question requires that criteria for inclusion and exclusion are accessible to all key stakeholders in a written and understandable form. It is not enough for them to be available only to doctors. This is because accessible and clearly stated indications (compared with informal rules) secure equal opportunities for taking part in any debate about the guideline's importance and acceptability.

Justification

The third question concerns the basis of the rationale given. For patients, and others, it is important to know whether exclusion criteria are justified with reference to medical considerations, economic considerations, or non-medical characteristics of patients such as age, productivity, social status, or gender.

Universal validity

The final criterion recognises the value of impartiality. It asks whether the reasons for exclusion are stated in a form that can be recognised by all as valid and relevant. This fundamental test is based on the close relation between impartiality and publicity.²³,²⁴ The requirements of publicity impose a special form on arguments. For example, arguments that are strictly self serving will not pass the test of publicity. Other reasons for exclusion, such as those based on race, religion, or sexual orientation, cannot be accepted as valid and relevant.

“Economic or political decisions should not be disguised as clinical decisions”

Conclusion

I have discussed clinical practice guidelines as a mechanism for rationing (withholding of potentially beneficial treatment) and as a potential tool for improving the quality of decisions about rationing. If guidelines are developed through a fair process—and the public views this process as legitimate—the decisions based on guidelines are likely to be acceptable. However, the criteria for developing evidence based guidelines do not recognise explicitly the fact that guidelines might become powerful rationing tools, and additional criteria that translate deliberative democratic theory into medical practice are needed. Clinical decisions should be based on the best available evidence within the twin constraints of resource scarcity and public scrutiny.