Achieving optimal technology use: A proposed model for health technology reassessment

Abstract

Background:

Healthcare providers, managers and policy-makers in many jurisdictions are focused on a common goal: optimizing value and quality of care provided to their citizens within a resource envelope. Health technology reassessment is a structured, evidence-based assessment of the clinical, social, ethical and economic effects of a technology currently used in the healthcare system to inform optimal use of that technology in comparison with its alternatives. There are, however, few practical experiences with health technology reassessment and, as such, a nascent theoretical and methodological base. Health technology reassessment is a key strategy to achieve optimal healthcare resource utilization, and establishing a model for health technology reassessment is a required methodological step.

Methods and results:

The purpose of this article is to answer three formative questions: (1) What is health technology reassessment? (2) When should a health technology reassessment be implemented? (3) What is the role of health technology reassessment in evidence-informed health policy? Finally, we propose a conceptual framework for health technology reassessment, which others can modify, adapt, or adopt in their own context. The model consists of three broad phases and six iterative stages: (1) identification, (2) prioritization, (3) evidence synthesis, (4) determine policy/practice recommendation, (5) policy/practice implementation and (6) monitoring and evaluation. Two foundational components (meaningful stakeholder engagement and ongoing knowledge exchange and utilization) are represented across all stages.

Conclusion:

This description of health technology reassessment and the proposed model can be used by healthcare policy-makers and researchers to advance the field of technology management, with the goal of achieving optimal use throughout a technology’s lifecycle.

Background

Efforts to control expenditures on health technologies have focused on managing the entry or adoption of new technologies into the healthcare system, such as through the process of health technology assessment (HTA).¹ However, for existing technologies, already in use within the healthcare system, there is currently no standardized process to continue monitoring their use or manage their exit if superseded by advancements in knowledge.^2,3 This gap in monitoring can lead to sub-optimal use of technologies, including the overuse, underuse or misuse.⁴ Overuse or misuse of ineffective or harmful treatments and practices, recognized as low-value care, has been documented to compromise patient safety and quality of care and jeopardize valuable healthcare resources.⁵

It is well-acknowledged that areas of low-value care exist, and a number of efforts to identify, prioritize, and reduce or remove such sub-optimal use have been described.⁶ A recent scoping review conducted by Niven et al.⁶ identified 43 different terms used to refer to processes examining low-value care, including the terms “disinvest*” (cited 39% of the time), “reassess*” (cited 14% of the time) and “de-adopt*” (cited 3% of the time).⁶ Interestingly, this work underscored the paucity of conceptual frameworks and models, particularly relating to the design and implementation of interventions to achieve the desired change in technology use. A model is, therefore, required to facilitate practice and advance the theoretical base.

Here, we propose a three-phase conceptual model for addressing sub-optimal technology use. This model is grounded in the policy process known as health technology reassessment (HTR), which is the active management of health technologies throughout their lifecycle within the healthcare system to achieve optimal value for money.⁴ This article will begin by answering three formative questions: (1) What is HTR? (2) When should an HTR be implemented? and (3) What is the role of HTR in evidence-informed health policy? A description of each phase of the HTR model is provided, followed by considerations of the potential limitations and expected challenges for the field as a whole. This information is intended to empower healthcare professionals, decision-makers and policy-makers to modify, adapt or adopt the HTR model for their own healthcare contexts.

What is HTR?

The process of HTR involves a structured, evidence-based assessment of the clinical, social, ethical and economic effects of a technology currently used in the healthcare system to inform optimal use of an existing technology in comparison with its alternatives.^4,7 To operationalize this, we conducted three foundational pieces of knowledge synthesis: a systematic review of the literature, an international environmental scan, and an invitational workshop of experts and key stakeholders. The details of each of these works have been described elsewhere.^7–9 Collectively, these knowledge synthesis pieces have directly informed the establishment of 11 guiding principles (Box 1) and were used to set the scope, reinforce the definitions and outline the boundaries for an evidence-informed model of HTR. Briefly, the guiding principles focus on integration of HTR within existing concepts and processes, engagement of stakeholders early and repeatedly in the process, flexibility and robustness.

Box 1.

Guiding principles for HTR.

1. HTR should be conceptualized as a mode 2 knowledge-generation activity 2. HTR is best integrated with other evidence-informed decision-making processes, such as the development of clinical practice guidelines and/or high-value care pathways, and overall quality improvement initiatives 3. HTR should not be viewed as a separate initiative but rather as a broadening of the scope of traditional HTA 4. HTR requires high-level political support 5. The language used to describe HTR is critically important and must be defined 6. An HTR model must be context-specific and flexible, with an expectation that it will evolve over time 7. Stakeholders must be meaningfully engaged and ideally embedded within any HTR process 8. Feasibility assessment, done collaboratively with stakeholders, must be done early in the HTR process 9. Cost accounting of real savings must be robust and accurate 10. Monitoring and evaluation are essential and need to be integrated into the HTR process 11. Monitoring and evaluation process must be flexible and robust enough to capture unintended consequences

HTR: health technology reassessment; HTA: health technology assessment.

What HTR is not

Confusion over HTR is rooted in terminology; it is important to identify what HTR is not and what it is not intended to do. HTR is distinct from concepts such as reduced use, removal, decommissioning, and disinvestment. It has been documented that these terms may portend controversy and disentitlement.⁴ The pejorative nature of these terms elicits fears of rationing and/or drastic budgetary cuts among stakeholders.¹⁰ Other have noted that clinicians themselves may interpret disinvestment as a threat to their autonomy and capacity for medical decision-making.¹¹

In stark contrast, HTR is contingent on collaboration with diverse stakeholders, such as clinicians, to optimize the value of care and increase appropriateness. This may be achieved by decreasing, increasing, or maintaining current levels of use. In rare cases, one might even completely withdraw the technology from the system (obsolesce).^4,7

Resource reallocation is an enticing outcome of an HTR. However, the freeing of resources would not come from rationing or cutting of existing budgets, but rather the cessation of inefficient or harmful treatments and practices. The resources freed up from decreasing or stopping inefficient practices can then be reallocated—a necessary condition to fund new things within a fixed budget.

What is the “value-add” of HTR

Establishing the HTR message also requires articulation of how HTR (and the model itself) adds conceptually and methodologically to the existing body of knowledge in the field. Through our synthesis exercises, we identified a number of established techniques that have provided an important foundation for HTR. For example, pair-wise and network meta-analyses and comparative effectiveness studies help to establish the relative clinical effectiveness of comparator technologies that are new or already in use.¹² The primary intent for these approaches is to identify the most effective technology for a given indication. Technologies that are less effective, and potential candidates for HTR, are identified only indirectly, and there is no understanding of how to ensure their optimal use.

Conversely, with HTR, there is an explicit search for and acknowledgement of what is considered low-value or sub-optimal use of a technology, as well as guidance on how to achieve its optimal use. The HTR process goes beyond the identification and prioritization of candidate technologies, which has also been well established through disinvestment activities,⁶ by incorporating the policy implementation phase directly as part of the process. This is a novel and critical factor that differentiates HTR from existing efforts to optimize existing technology use and has often been noted as a clear paucity in the field.⁶

Furthermore, HTR is not simply a means of ensuring the removal of obsolete technologies from the healthcare system, but rather the continued management of technologies at any point in their lifecycle. Certainly, the obsolescence of a given technology serves as a candidate technology whose exit from the system can be managed through an HTR process. However, as earlier defined, an HTR can also result in the increased use or decreased use of an existing technology. In such cases, a technology would not be in the obsolescence phase of its lifecycle. Thus, ensuring optimal use of existing technologies and efficiency of the healthcare system through HTR activities demands that efforts be taken to continually monitor use, beyond their adoption, to determine whether they continue to provide value for money.

When should an HTR be implemented?

As alluded to above, HTR should be considered as an ongoing policy process to inform technology use throughout its lifecycle. Just as HTA is used to inform the adoption of a technology into the healthcare system, HTR could be used as part of an active technology management program across the lifecycle of the technology (Figure 1).

Figure 1.

Lifecycle of a health technology and the potential zone of reassessment reproduced and adapted with permission from Joshi et al.²

The feasibility of achieving change is a crucial consideration as a first success may determine the trajectory of an ongoing HTR program. Determining the feasibility of an HTR requires a multi-pronged approach. Within a given healthcare system, assessing readiness for change, the capacity and availability of resources and the willingness of appropriate stakeholders, including opinion leaders, to engage in the process are required initial considerations before launching an HTR. An initial scoping review may be useful to frame the feasibility discussion with the key stakeholders. In addition, qualitative research approaches (e.g. interviews and/or focus groups with key stakeholders) may be required to understand the context of the technology or practice area being considered for reassessment. A list of initial feasibility questions for consideration is outlined in Box 2.

Box 2.

Questions to assess feasibility of HTR.

• What is the healthcare system’s or organization’s readiness for change? • Are there champions ready to be engaged? • Is there known evidence of harm, a lack of effect or a more cost-effective alternative? • Is the “authority” promoting the policy/practice change also the one implementing it? • Is there high issue polarization (i.e. disagreement about which policy/practice to focus on)? • How many different actors are involved in implementing the recommended policy/practice change? • What are the anticipated undesirable consequences (i.e. trade-offs)? • Are current incentives in the system misaligned with adoption of the new policy/practice? Can incentives be aligned with the desired practice be identified and implemented? • What incentives/tools are available to implement the policy? • Are the personnel, skills and material and technological resources that are required to implement the policy/practice change available?

HTR: health technology reassessment.

What is the role of HTR in evidence-informed health policy?

If healthcare providers, managers and/or policy-makers are seeking to maximize the value and quality of care provided to citizens within a particular resource envelope, HTR is a tool to achieve this goal. HTR provides a transparent, inclusive and methodologically rigorous process that can empower stakeholders to make evidence-informed health policy decisions. Reassessment provides an opportunity to ensure that such technologies are used to their full potential, optimizing value for money.

HTR can also be applied to identify “innovation headroom” and act in concert with or as a corollary process to HTA; if we want to continue introducing new effective and cost-effective technologies without increasing the healthcare budget, an active approach to managing technologies already in use is required.⁴ Ensuring optimal use of technologies throughout their lifecycle ultimately contributes to a sustained culture of evidence-informed decision-making.

The conceptual HTR model

The HTR model is depicted in Figure 2. Broadly, the model has three sequential phases. Phase 1 (Technology) involves selection of the technology to be reassessed and includes identification and prioritization. In Phase 2 (Decision), evidence needed to inform the development of a policy recommendation is collected and synthesized. In Phase 3 (Execution), the policy is implemented and the initiative is monitored and evaluated. There are also two foundational components (meaningful stakeholder engagement and ongoing knowledge exchange and utilization) that must be engaged throughout the entire process. A detailed discussion of each component follows.

Figure 2.

A proposed HTR model.

Foundational components

Meaningful stakeholder engagement

A “stakeholder” is broadly defined to include any person or group the decision will impact. Relevant stakeholders may include physicians, nurses, other clinicians, hospital managers, decision-makers, government, patients, families, caregivers and citizens. Stakeholder engagement is most effective and meaningful when stakeholder engagement is continuous throughout the HTR process, there is transparency in both methods and processes, and the engagement is authentic. Meaningful engagement will legitimatize the HTR process and increase the likelihood of success.

Ongoing knowledge exchange and utilization

Deliberate, continuous knowledge exchange will similarly increase the likelihood of success. Research to date shows that if decision-makers are involved in the knowledge-generation process, they are more likely to use the knowledge to change their practices and/or policies, and therefore, successful implementation is more likely.^13–15

Stakeholders who will be using the knowledge generated through the research or evidence synthesis process to inform their policy and/or practice must be involved from the beginning. This includes embedding stakeholders in determining the research questions throughout the knowledge-generation process, the interpretation of findings and the development of recommendations to inform practice and policy (mode2 knowledge).¹³

Phase 1: technology selection

Step 1: identification of technologies

An HTR process is likely to be more effective if clinicians adopt leadership roles in identifying potential technologies or practice areas for reassessment. Based on reported experiences, it is the clinicians working in the healthcare system who are best positioned to identify misused, overused or underused technologies. Approaches such as horizon scanning of the literature and developing de novo lists of potential candidate technologies for HTR are also methods of identification, but often require substantial infrastructure and resources to maintain.

Step 2: prioritization of technologies

When a number of possible HTR candidates are identified, prioritization is necessary. Individual healthcare contexts will need to adopt or tailor their own prioritization processes and may benefit from leveraging those that are already established (e.g. processes for HTA). If a system-wide approach is being adopted, the prioritization criteria should be the same for all identified technologies, irrespective of where the technology is in its lifecycle. Best practices include transparency, clarity and availability to all stakeholder groups. Feasibility of achieving change is also a crucial consideration for prioritizing a technology. Key questions are outlined in Box 2 and discussed in depth in the previous section.

Phase 2: decision

Step 3: evidence synthesis

A broad evidence synthesis including clinical, economic, social, ethical and legal issues is required. Best practices in each of these areas should be employed and may include systematic reviews of the published literature, an environmental scan of jurisdictional practices, analysis of clinical or administrative data on utilization of the technology and primary data collection (quantitative and qualitative). Much of the methodology (e.g. systematic review, meta-analysis, economic evaluation) applied in the HTA field can be adopted in this context.

Stakeholders must be engaged throughout the evidence synthesis process: in the development and refinement of the initial evidence synthesis questions to ensure that the questions are well defined, appropriate, and inclusive and during critical revisions and development of the final evidence product.

Step 4: policy development

A policy or practice recommendation will be informed by the knowledge generated through the evidence synthesis. The process for policy development will vary based on context and could range from targeted clinician or policy-maker focus groups to broad stakeholder deliberative processes. Irrespective, to maximum implementation success, the recommendation should be developed collaboratively with the stakeholders responsible for implementation.

Decision-making criteria worth considering in the policy development process include equity of access to the technology, capacity in the system to adjust the scope of use of the technology and how adjusting the scope of this technology may affect the use of other technologies in the clinical area.

Phase 3: execution

Step 5: policy implementation

Implementation of the recommended policy will likely be the most difficult step in the HTR process. Importantly, implementation must be feasible, factoring in the health system’s readiness for change. A key consideration will be the policy levers (e.g. financial incentives, changes to the schedule of medical benefits) or tools (e.g. utilization and costing data) for change available to those implementing the policy. Responsibility for policy and/or practice implementation will vary; the policy may focus on the micro, meso or macro level.¹⁶ Good policy practices should be employed, including clear responsibility, accountability and courses of consequence.

Step 6: monitoring and evaluation of policy/practice implementation

Monitoring and evaluation should start immediately after implementation to ensure that lessons learned can be captured and used to refine the implementation process. Conceptualizing monitoring and evaluation as ongoing routine processes, as opposed to one-time events, is critical in the development and sustainability of complex initiatives. Evaluation should incorporate formative, process and outcome evaluation components as well as monitoring for unanticipated consequences (i.e. outcomes that we did not expect) using both quantitative (e.g. data analysis of utilization and costs) and qualitative (e.g. observation, interviews, focus groups) methods.

The HTR model is flexible and should be adapted depending on the technology being reassessed and user objectives and needs. Finally, as new knowledge is generated more broadly in the field of reassessment, key learnings from these works should be incorporated into the HTR model.

Limitations of the conceptual HTR model

One important limitation of the conceptual HTR model is that it has not been applied in practice. It is unclear how long the HTR process may take and of the entirety of resources that may be required to achieve success. Therefore, the model requires pilot testing within a real-world healthcare context in order to determine its feasibility and utility. Pilot testing will also enable the identification of areas for adaptation.

We acknowledge that there is likely no one model to drive all HTR initiatives; individual healthcare environments are widely varied and context-specific. Our intent was not to be overly prescriptive but rather contribute to the much-needed knowledge base, as well as to stimulate discussion among researchers, policy-makers and health service decision-makers considering implementation of HTR initiatives.

Expected challenges for HTR

Due to the entrenched nature of existing technologies, it is likely that the strength, type and nature of the evidence required for HTR will be different than for other evidence-informed policy contexts. Experience with HTR, to date, has also shown that a higher quality of evidence is expected to inform the reduction in scope of use of a technology.^6,8,9 In addition, HTR initiatives are likely to face the challenge of there being little or no high-quality research evidence on the effectiveness of a particular technology.⁷ Thus, further understanding of the thresholds of evidence to inform reassessment decisions will need to be determined.

Establishing meaningful stakeholder engagement may also prove challenging. Receptivity of stakeholders to participate in HTR may be dampened due to perceived unpopularity of previous associated initiatives, such as disinvestment. Stakeholders may also have vested interests in not having a given technology reassessed, particular if they are healthcare providers actively using the technologies that are under scrutiny. However, it is important to recognize that there is an ethical imperative to ensuring optimal use of technologies and the highest quality of care for all patients in a given healthcare context. As the de facto stewards of healthcare resources, healthcare providers must not only acknowledge this imperative, but we argue that they must also actively engage in HTR processes to ensure it.

Finally, determining what technologies provide good value for money, and ultimately what defines optimal use for a given technology, may prove challenging across various healthcare contexts. As with the implementation of HTR process itself, criteria for decision-making will likely need to be context-specific. Building off existing local criteria for the adoption of new technologies, for example, those employed for HTA, is a recommended starting point. Beyond this, future empirical work in the area of HTR would also benefit from addressing the operationalization of a reinvestment program downstream of freed resources and, relatedly, uncovering optimal incentives to facilitate HTR.

Conclusion

HTR is a tool to optimize use of technology throughout its lifecycle; it recognizes that optimal use not only involves the decreased use or exit of low-value care from the healthcare system but also encompasses the increased use and promotion of high-value practices. Practically and theoretically, the field of HTR is gathering momentum. Much focus to date has been placed on the identification and prioritization of low-value care or opportunities for HTR,⁶ yet studies addressing and implementing methods to facilitate or address low-value care are still in nascent stages. To move this field forward, we must continue to build on international experiences—including learning from identified barriers for disinvestment programs and complex, large-scale change management initiatives—by focusing on developing novel methodological approaches to generating, incorporating and implementing evidence into policy and practice. The conceptual HTR model presented here is a first step in this ongoing process. We acknowledge that there is no one model that will drive all HTR initiatives. Therefore, evidence, stakeholders and resources required to achieve a successful HTR initiative must ultimately be tailored to those individual contexts.