HEALTH TECHNOLOGY REASSESSMENT: THE ART OF THE POSSIBLE

Abstract

Background: Health technology reassessment (HTR) is “a structured, evidence-based assessment of the clinical, social, ethical, and economic effects of a technology currently used in the healthcare system, to inform optimal use of that technology in comparison to its alternatives.” The purpose of this study is to describe the key themes in the context of current HTR activities and propose a way forward for this newly emerging field.

Methods: Data were gathered from a workshop held as part of the 2012 Canadian Agency for Drugs and Technology in Health (CADTH) symposium. The workshop consisted of two panel presentations followed by discussion; data gathered, including presentations and rich audience discussion transcripts, were analyzed for key themes emerging in the field of HTR using constant comparative analysis.

Results: The language chosen to describe HTR will set the tone for engagement. The identification of champions at multiple levels and political will are essential. Key lessons from international experience are: disinvestment is difficult, focus on clinical areas not specific technologies, identify clear goals of the HTR agenda. Six key themes were identified to move the HTR agenda forward: emphasize integration over segregation, focus on development of HTR methods and processes, processes are context-specific but lessons must be shared, build capacity in synergistic interdisciplinary fields, develop meaningful stakeholder engagement, strengthen postimplementation monitoring and evaluation.

Conclusions: To move this field forward, we must continue to build on international experiences with a focus on developing novel methodological approaches to generating, incorporating, and implementing evidence into policy and practice.

Technologies, broadly defined as drugs, diagnostic tests, including indicators and reagents, devices, equipment and supplies, medical and surgical procedures, support systems, and organizational and managerial systems used across the spectrum of health care, are a major source of expenditure within the healthcare system (1). For decades, the healthcare system has focused on managing the entry of technologies, including the requirements for licensure mandated by Health Canada for drugs and devices. For specific technologies, including new drugs which must be assessed in Canada by the Common Drug Review, a demonstration of clinical value and cost-effectiveness is also required (2). However, once a technology enters the system, there is no standardized process for monitoring its use, nor managing its exit if it is superseded by advances in knowledge (1;3).

The proposed working definition for health technology reassessment (HTR) is: “a structured, evidence-based assessment of the clinical, social, ethical, and economic effects of a technology currently used in the healthcare system, to inform optimal use of that technology in comparison to its alternatives.” (4) (Box 1). The language chosen to describe the assessment of technologies currently in use in the healthcare system can be influential in setting the tone for engagement. For example, language such as “disinvestment” and “reinvestment” focus attention on the cost component of technology, and the potential inference of a foregone conclusion to the process, rather than the actual goal of HTR; optimizing the use of technologies in health care and achieving the greatest clinical benefit with our social dollars. The term HTR is proposed as it is value neutral and does not pre-suppose the outcome of the process. Although HTR has perhaps traditionally been thought of as managing the exit of particular technologies from the system, a complete removal of a technology is unlikely to be warranted in most circumstances; experience to date suggests that a change in scope of use is far more common (5;6).

While intimately linked to both health technology assessment (HTA) and disinvestment, HTR is a distinct concept. While HTA application has been mainly limited to managing the entrance of “new things” into the healthcare system, HTR is a process to support on-going evidence informed policy regarding the optimal use of technology throughout the lifecycle of technology. Technologies undergoing HTR may or may not have undergone HTA as part of their adoption decision. However, all technologies selected for HTR are actively being used within the healthcare system with a requirement to manage their scope of use. Disinvestment is one potential outcome of HTR; other outcomes include no change in use, narrowing the scope of use (including the potential to increase use of a comparator technology in place), increasing scope of use and in rare circumstances, stopping use all together. In the past, HTR has been an implicit part of HTA as often usage of a competing technology will be affected by the adoption of a new technology. However, there is now a desire to see HTR become standard practice with a goal of optimizing the use of technologies in health care and achieving the greatest clinical benefit with our social dollars (1;3;7–9).

A recent systematic review of the literature on HTR identified eight countries with some evidence of past or current work related to HTR (10). Many organizations realize the potential benefits of HTR and are interested in integrating it into their healthcare system. However, a lack of top-down support, push back from clinicians, financial and human resource limitations, and a lack of expertise in health technology reassessment are inhibiting program development (8;11–14). Several critical factors for success were also identified, including the importance of early and ongoing stakeholder engagement, the identification of champions at multiple levels, embedding HTR into existing structures if and where appropriate, and developing HTR models in context that are adaptable (8;12;14;15).

Given the mounting interest in continued assessment beyond the adoption decision, the 2012 Canadian Agency for Drugs and Technology in Health (CADTH) symposium held in Ottawa, Canada in April 2012 featured a preconference workshop entitled Health Technology Reassessment (HTR): Promoting Value and Evidence-Based Practice. The purpose of this study is to describe the key themes in the context of current HTR activities and propose a way forward for this newly emerging field.

METHODS

The workshop was hosted by the Health Technology Assessment Unit at the University Calgary, who are actively engaged with the provincial Ministry of Health to develop a HTR model for Alberta, Canada. The workshop was open to all attendees of the CADTH symposium. Attendees came from a variety of contexts including representation from provincial, national, and international HTA agencies, ministries of health, not-for-profit health agencies, academia, and industry. The workshop began with a brief presentation to establish a common understanding of HTR and its development as a field. Two panel presentations followed. The first panel focused on methodological challenges and consisted of a health economist, an expert in implementation science and an international expert in HTR. The second panel focused on current practices with representation from three jurisdictions (Alberta, Australia, and United Kingdom) each at different stages with their HTR programs. During the panel sessions, each panelist was given 5 minutes to provide an introduction followed by audience discussion. The workshop was recorded and transcribed. All data gathered, including presentations and rich discussion transcripts, were analyzed for key themes emerging in the field of HTR using constant comparative analysis.

Table 1.

Definitions

Disinvestment: the process of withdrawing (partially or completely) health resources from those existing healthcare practices, procedures, technologies, and pharmaceuticals that are deemed to deliver no or low health gain and are thus not efficient health resource allocations (8).

Health Technology: Any intervention that may be used to promote health, to prevent, diagnose or treat disease or for rehabilitation or long-term care. This includes the pharmaceuticals, devices, procedures and organizational systems used in health care (24).

Health Technology Assessment: a multi-disciplinary field of policy analysis that examines the medical, economic, social and ethical implications of the incremental value, diffusion and use of a medical technology in health care (24).

Health Technology Reassessment: a structured, evidence-based assessment of the clinical, social, ethical and economic effects of a technology currently used in the healthcare system, to inform optimal use of that technology in comparison to its alternatives (4).

Table 2.

Summary of Themes

Disinvestment is difficult: Limited success has been achieved internationally.

Focus on clinical areas in addition to specific technologies: changes in utilization of one technology may impact utilization of a competing technology. Efficiencies in process may be achieved by reviewing clinical areas as opposed to single technologies.

Clear goals of the HTR agenda are required to prioritize candidate technologies: A common understanding of the goal supports transparent prioritization of HTR candidates.

Emphasize integration over segregation: Embed HTR into existing processes.

Focus on development of HTR methods and processes: methodology to measure costs, benefits, and values is necessary for non-drug technologies

Processes are often context-specific: Learnings from other jurisdictions must be tempered by local contextual and system realities.

Build capacity in synergistic interdisciplinary fields: Innovative, interdisciplinary teams of highly skilled people from disciplines such as economics, implementation science, KT, health services research, policy analysis, change management, evaluation, and stakeholder engagement are required.

Meaningful stakeholder engagement: An effective strategy should “grab both the hearts and the minds of stakeholders.” Thoughtful consideration of patient and public engagement should be undertaken.

Strengthen post-implementation monitoring and evaluation: Post-implementation monitoring and evaluation must be undertaken with an emphasis on active monitoring for unintended consequences.

RESULTS

Themes from International Experiences

The United Kingdom and Australia report the most experience in HTR. At the national (Medicare) level in Australia, HTR has been integrated under the responsibility of the Department of Health and Ageing, including the Medical Services Advisory Committee (MSAC), which makes recommendations to the Health Minister about what medical services offer sufficient safety and (cost)effectiveness to warrant public subsidy. Since 2009, a Quality Framework for Australia's Medicare Benefits Schedule was developed under the auspices of the MSAC (16). As part of this initiative, the Health Minister commissioned a review of HTA in Australia, with one outcome being the development of a postmarket surveillance system that included a parallel approach to HTR. This parallel approach involved a review of individual items (17), which dovetailed with existing HTA processes, as well as a whole-of-specialty review (17). Both proved to be novel, effective approaches to engaging clinicians from the beginning of the HTR process.

For the review of individual items, a proposed scanning process was adapted to identify appropriate candidates for HTR (15). Using this process, 156 possible candidates were identified a nd provided to government for consideration, from which they identified fifteen for initial rapid review and potential full HTR (16). Outcomes from review include: an amendment to the item description such that it better captures the patient group/s most likely to benefit from any procedure, for example; an increase, decrease or maintenance of the fee; or a complete stop to public funding of the item. One example of an individual technology that underwent a complete review was vertebroplasty for osteoporotic vertebral fractures. The recommendation from MSAC was that Government stop public funding of this procedure because of a lack of evidence of effectiveness. Some clinicians noted their dissent to MSAC's decision yet concomitantly, multiple reports from around the world have been developed supporting MSAC's position (18).

In ophthalmology, a whole-of-specialty review was conducted. The initial impetus for the review was a government decision to cut the fee for cataract surgery, which resulted in dissent from ophthalmologists. The government's observation was that advances in cataract surgery technology delivered on its promise: procedures could be done far more quickly and safely then when it was first introduced (and priced), so a decrease in fee seemed the appropriate market response. The ensuing whole-of-specialty (ophthalmology) review of the Medical Benefits Schedule was approached collaboratively with ophthalmology stakeholders, asking the field what in the schedule they thought required some modification. Of the sixty-one item descriptors on the schedule only twenty remained unchanged through the process. Items were either clarified, modified (e.g., refinement of indications and/or eligible patient groups), split, merged, or entirely removed (17). In all respects, this was considered a success with good outcomes for all including patients. The whole-of-specialty approach is now rolling out more broadly.

In the United Kingdom, the National Health Service (NHS), through the National Institute for Health and Clinical Excellence (NICE), has launched a HTR program. NICE has developed two streams; the “do-not-do” recommendations and the Cochrane Quality and Productivity Topics. Both are passive HTR activities resulting in guidance that clinicians may choose to follow. The “do-not-do” recommendations is a searchable database of recommendations made since 2007. The recommendations are identified during the process of guidance development where NICE's independent advisory bodies often identify NHS clinical practices that they recommend should be discontinued completely or should not be used routinely. Each record contains the “do not do” recommendation and includes additional information including the intervention, health topic, the guidance it comes from (with a link to the relevant paragraph in the guidance), and the other “do not do” recommendations from the same guidance. Each recommendation also includes the healthcare setting that describes the main clinical environments in which the intervention or investigation may be initiated.

In addition to the “do-not-do” recommendations, NICE has also developed the Cochrane Quality and Productivity Topics. These are based on Cochrane reviews with the goal of helping the NHS identify practices that could be reduced or stopped freeing up cash and resources that could reinvested in more effective practices. NICE's experience over the past 12 years, however, is that although reassessment and potential disinvestment has the potential to increase efficiency and quality, actual cost savings, and therefore the potential for reinvestment has been less than anticipated. In the year 2011–2012, half of the efficiency savings came from the freezing of health professionals’ salaries, not disinvestment in technologies (19).

An analysis of the shared international experiences resulted in three common themes emerging:

Disinvestment is difficult

Even in the presence of very strong evidence that a particular technology is harmful and/or (cost)ineffective, withdrawing the technology from the healthcare system is difficult. Players, both within and outside the healthcare system, must be prepared for political and professional fall-out. Strong, unwavering political will is critical.

Focus on clinical areas in addition to specific technologies

It is difficult to tease out the appropriate scope of use of a particular technology from the broader pathway of care. Very little done in health care is done in isolation, but rather procedures and devices are used in combination with other treatments within a pathway of care. A change in scope of use of a technology is likely to have ripple effects, resulting in unintended consequences, such as an increase in use of an alternative treatment. A clear understanding of the care pathway within which the technology is embedded, along with social and ethical implications, is critical. The unintended consequences may be mitigated by focusing on clinical areas where multiple comparators can be assessed side-by-side in any care treatment algorithm. In addition, this approach provides a natural way for engaging clinicians as one could begin by approaching a particular department or specialty. Clinical specialty support of any disinvestment list was described as critical to moving forward.

Clear goals of the HTR agenda are required to prioritize candidate technologies

If the focus is actual cost savings, then it is likely that much could be achieved by focusing on redesigning service delivery mechanisms, and optimizing the human resources within the healthcare system. However, if the goal is to optimize the use of technology, which may include cost-savings, focus must move to technologies which are misused in the system.

Moving Forward with Health Technology Reassessment: A Call to Action

Analyzing the audience discussion in the context of current literature and expert presentations, several key themes emerged to move the field of HTR forward.

Emphasize integration over segregation

Integrating HTR with on-going work such as the development of clinical practice guidelines and care pathways, as well as quality improvement initiatives will likely result in the greatest uptake of HTR. Advocating for the integration of economic considerations into guideline and care pathway development will support value for money as one criterion to inform decisions about what should and should not be done.

Focus on development of HTR methods and processes

The need for methodology to measure costs, benefits, and values is dire. These methods are particularly necessary for non-drug technologies. Methodologies must maximize usefulness for decision makers, and incorporate outcomes of importance to both clinicians and patients. In addition, they must represent feasible tools to use within a policy/decision-making context; they must be relevant, timely, and interpretable.

Processes are often context-specific

Although there is much to be learned from processes developed in various jurisdictions, processes ought to be context-specific. Learnings from other jurisdictions must be tempered by contextual and system realities, and processes are likely to require local adaptation. This might include retro-fitting new HTR processes to existing policy structures. Key stakeholders must be engaged in this work. This will ensure that processes developed are supported by, and meet the needs of, the system using the outcomes to inform their decisions.

Build capacity in synergistic interdisciplinary fields

Highly skilled people are required in a variety of disciplines including: economics, implementation science, KT, health services research, policy analysis, change management, evaluation, and stakeholder engagement. It will take innovative teams that are interdisciplinary and cross institutional in nature to take on the HTR challenge.

Meaningful stakeholder engagement

Meaningful stakeholder engagement is crucial. Essential elements to engage stakeholders are: transparent processes for identifying and prioritizing potential HTR candidates, involvement throughout the process of evidence generation and review, and authentic meaningful engagement (11;13;14;20). Stakeholders should be defined broadly to include any group impacted by the resulting decision. As clinicians are likely to be an important stakeholder group in any HTR, an effective strategy should “grab both the hearts and the minds of clinicians.” Thoughtful consideration of patient and public engagement should be considered (21).

Strengthen postimplementation monitoring and evaluation

HTR must include a developed postimplementation monitoring and evaluation process with an emphasis on active monitoring for unintended consequences.

“There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know. (22)”

Active monitoring for these “unknown unknowns” is critical to ensure that health outcomes, access, and equity are not adversely impacted. For example, within a surgical program, the operation suite will never be un-used. Thus, if one procedure is stopped, the operational suite time could be filled with another procedure perhaps equally as, or more, harmful and wasteful. Active monitoring for these situations will ensure that the vacuum created by removing one procedure is back-filled by a more effective procedure resulting in better value for money. This point emphasizes the need for buy-in by clinicians to mitigate unintended consequences. Monitoring for these consequences will require a variety of evaluation techniques drawing on both quantitative and qualitative research methodologies.

CONCLUSION

HTR is not merely an academic exercise. If HTR is to be successful it must generate useful knowledge to inform real world decisions around the optimal use of technologies. The term “inform” is deliberately used here, as it is recognized that research evidence is only one of many kinds of knowledge that decision makers use when making these decisions, and that these decisions are always informed by values. In addition, decision making is neither rational nor linear, and as Jonathon Lomas has noted “is not an event; rather it is a generally diffuse, haphazard & somewhat volatile process that is ethereal in nature” (23).

As one panelist noted, evidence-based guidance can be viewed as a practical manifestation of social contracts in deliberative democracies to ensure the most efficient and ethical allocation of finite healthcare resources. To achieve this goal, social values as well as technical issues need to be considered and guidance developed from HTR should reflect the social/political milieu.

Healthcare budgets will not continue to increase and we need to shift our thinking about how we can use these dollars differently with the goal of optimizing “value for money.” Managing technologies throughout their lifespan is required to ensure that technologies continue to achieve optimal value for money. HTR is required to support evidence-informed continued management. To move this field forward, we must continue to build on international experiences with a focus on developing novel methodological approaches to generating, incorporating, and implementing evidence into policy and practice.

CONTACT INFORMATION

Gail MacKean, PhD, Research Consultant, Health Technology Assessment Unit, Department of Community Health Sciences, Institute of Public Health, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada

Tom Noseworthy, MD, MPH, Professor, Health Technology Assessment Unit, Department of Community Health Sciences, Institute of Public Health, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada

Adam G. Elshaug, PhD, Associate Professor, Menzies Centre for Health Policy, School of Public Health, The University of Sydney, NSW, Australia

Laura Leggett, BSc., Research Assistant, Health Technology Assessment Unit, Department of Community Health Sciences, Institute of Public Health, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada

Peter Littlejohns, PhD, Professor, Division of Health and Social Care Research, School of Medicine, King's College London, Capital House, London

Joan Berezanski, Assistant deputy minister, Clinical Advisory and Research Branch, Alberta Health, Edmonton, Alberta, Canada

Fiona Clement, (fclement@ucalgary.ca), PhD, Assistant Professor, Health Technology Assessment Unit, Department of Community Health Sciences, Institute of Public Health, Faculty of Medicine, University of Calgary, Calgary, Alberta, Canada

CONFLICTS OF INTEREST

All authors report they have no potential conflicts of interest.