Decision-Making on New Non-Drug Health Technologies by Hospitals and Health Authorities in Canada

Abstract

Introduction:

Unlike those for publicly funded drugs in Canada, coverage decision-making processes for non-drug health technologies (NDTs) are not well understood.

Objectives:

This paper aims to describe existing NDT decision-making processes in different healthcare organizations across Canada.

Methods:

A self-administered survey was used to determine demographic and financial characteristics of organizations, followed by in-depth interviews with senior leadership of consenting organizations to understand the processes for making funding decisions on NDTs.

Results:

Seventy-three and 48 organizations completed self-administered surveys and telephone interviews, respectively (with 45 participating in both ways). Fifty-five different processes were identified, the majority of which addressed capital equipment. Most involved multidisciplinary committees (with medical and non-medical representation), but the types of information used to inform deliberations varied. Across all processes, decision-making criteria included local considerations such as alignment with organizational priorities.

Conclusions:

NDT decision-making processes vary in complexity, depending on characteristics of the healthcare organization and context.

Health Technology Assessment and Management in Canada

Canada has had a long tradition of health technology assessment (HTA) and was one of the first countries to institutionalize HTA processes (Battista 1992; Battista et al. 2009; Feeny and Stoddart 1994; Menon and Stafinski 2009). However, HTA processes for drugs and non-drug health technologies (NDTs) (such as medical devices, diagnostic tests and surgical procedures) have evolved along different trajectories. Thirty years ago, individual jurisdictions in Canada had separate HTA processes for making decisions on which drugs to cover through public plans. Those processes typically included an expert committee who reviewed applications from manufacturers and formulated recommendations (Menon 2014). Pan-Canadian, centralized approaches (the Common Drug Review and the pan-Canadian Oncology Drug Review) that make recommendations to provincial federal and terrirtorial public drug plans excluding Quebec now exist (Canadian Agency for Drugs and Technologies in Health [CADTH] 2019a, 2019b). However, there is no similar pan-Canadian process for non-drug technologies (NDTs). While Alberta, British Columbia, Ontario and Quebec have established formal provincial mechanisms for assessing NDTs to inform decision-making, other jurisdictions have primarily relied on rapid response services offered by CADTH.

It has been reported that between 1996 and 2008, expenditures on NDTs in Canada grew by $23 billion, compared to $5 billion for drugs (Grootendorst et al. 2011), leading to questions about the added value of these technologies. (A residual approach, which included both the cost of purchasing a health technology and the costs associated with their use, was used to generate such estimates, and therefore, $23 billion is likely an overestimate.) In response, a Federal/Provincial/Territorial Policy Forum discussed the possibility of establishing a centralized NDT review process, and more recently, the Conference of Deputy Ministers of Health identified health technology management (HTM) as a priority for Canada. Specifically, it tasked CADTH with the development of a pan-Canadian HTM strategy to “improve” how NDTs are adopted and diffused into institutions across Canada (CADTH 2016).

However, implementation of an effective HTM strategy first requires an understanding of how NDTs currently “enter” healthcare organizations in Canada. The last major study of decision-making processes for NDTs in Canadian hospitals was published 25 years ago (Deber et al. 1994). Since then, substantial changes in technology and in the organization and funding of health systems across Canada have taken place, creating a need to revisit this topic.

Objective

This project aimed to understand how decisions around the introduction of NDTs are made in different healthcare organizations across the country and what types of information are used to inform them.

Methods

The project, which included two parts, was overseen by a pan-Canadian advisory committee (PAC) comprising seven healthcare system executives, four senior-level individuals from HTA organizations and four academic researchers.

Part 1

A self-administered survey (Appendix 1, available online at longwoods.com/content/25936) was sent to the heads of healthcare organizations who were identified through the 2012 Guide to Canadian Healthcare Facilities and the PAC. It contained questions on the demographic and financial characteristics of the organization, their approaches to funding NDTs and the extent to which NDT decision-making was seen as a challenge. It also invited organizations to participate in Part 2 of the project, which involved in-depth telephone interviews. The survey was pilot-tested with PAC members prior to its administration.

To optimize response rates, the Dillman Tailored Design Survey Method (Dillman et al. 2014) was used. Questionnaires included cover letters co-signed by the lead researcher, a member of the PAC and the President and CEO of HealthCareCAN (an organization of 45 health institutions across Canada). Two rounds of reminder letters and surveys were sent (both by e-mail and regular mail) to non-responding organizations. In addition, PAC members personally contacted organization leaders by telephone and/or e-mail.

Responses were analyzed quantitatively using basic descriptive statistics.

Part 2

Telephone interviews were designed to elicit in-depth information on the scope of NDTs considered, decision-making structures (e.g., committee membership, mandate) and processes (e.g., initiators of NDT requests, information used to support/inform deliberations, factors involved in decisions/recommendations). Each interview involved a minimum of two researchers. Two were female (PhD, MPH) and three were male (MD, MHA, MA). Three were academic researchers and two were consultants. All of them had previous experience conducting interviews. None of the interviewers had a prior relationship with the participants, who were also unaware of the characteristics of the interviewers. There were no other participants in the interviews. In compliance with ethics approval for the project granted by the University of Alberta Health Research Ethics Board 2, interviews were not audiotaped, but detailed notes were taken. Also, member checking, in which interviewees receive the opportunity to review notes for accuracy, was performed.

Responses to categorical questions were analyzed quantitatively using basic descriptive statistics. Responses to open-ended questions were analyzed qualitatively using thematic analysis and constant comparative methods. Specifically, open coding was first used to analyze responses line by line in the notes and identify as many concepts (codes) as possible. New codes were continually compared to those already assigned to chunks of text in the notes to reveal any consistencies and differences. Patterns between codes were examined to develop potential categories. Then, axial coding was used to make connections between categories and determine those which represented the central focus (axial categories). The resulting codes were subsequently converted into themes. When themes comprised a step in a decision-making process, they were organized sequentially to create a visual display of their interconnectedness. To minimize interpretation bias, all of the responses were coded independently by two researchers.

Results

Seventy-three organizations completed Survey 1 (response rate: 20%) (Figure 1). Almost half of the organizations reported that decision-making on NDTs was extremely or very challenging (83% of these were Regional Health Authorities [RHAs], Centres Intégrés de Santé et de Services Sociaux [CISSSs] or academic hospitals with budgets ranging from $35 million to $13.6 billion). Forty-three per cent found it moderately or slightly challenging (51% RHAs/CISSSs/academic centres, with budgets ranging from $42 million to $3.8 billion). The remaining 8% indicated that it was not challenging at all. These organizations provided largely non-technology-intensive services (e.g., palliative care, rehabilitation and mental health) or comprised non-university-affiliated community care centres.

Figure 1.

Number of organizations participating in each step of the project

Forty-seven of 75 organizations from eight jurisdictions (two procurement organizations were also invited at the suggestion of interviewees) participated in key informant interviews (Survey 2) within the available time for data collection. A comparative analysis of characteristics between participating and non-participating organizations revealed no statistically significant differences (e.g., in budget or hospital size). The 48 participating organizations consisted of 26 hospitals (11 academic hospitals), 10 health authorities, eight CISSSs/Centres Integrés Universitaire de Santé et de Services Sociaux (CIUSSs), one local health integration network (LHIN) and two procurement organizations. CISSSs/CIUSSSs combine various institutional health and social service providers (hospitals, nursing homes, Centres Locaux de Services Communautaires (CLSCs), youth detention centres, etc.) into a single organization. LHINs are responsible for planning, integrating and distributing funding from the government for all public healthcare services at a regional level. Up to three interviews were conducted with each of the participating organizations. Interviewees varied in their roles within organizations and included CEOs, other senior executives (vice-presidents, chief financial officers, chief operating officers, chief information officers), executive directors, directors, managers, department chairs, clinical program leads and administrators. On average, interviews lasted between one and two hours.

The 47 organizations yielded information on 55 separate processes. Their characteristics are summarized in Table 1.

Table 1.

Characteristics of organizations (N = 47) participating in Part 2 (telephone interview)

Characteristic	Number (%) of organizations
Type of organization
Academic centre(s) only	16 (34)
Non-academic centre(s) only	20 (43)
Both academic and non-academic centre(s)	8 (17)
Not applicable	3 (6)
Size of organization (based on number of beds)
Small (<100 beds)	7 (15)
Medium (100-1,000 beds)	28 (60)
Large (>1,000 beds)	10 (21)
Not applicable	2 (4)
NDT funding source(s)*
Hospital foundations	41 (87)
Specific/targeted government grant	34 (72)
Global budget government funding	32 (68)
Research funding	16 (34)
Manufacturers	11 (23)
Other	3 (6)
Not applicable	2 (4)

^*more than one category may apply

Organizational Budgets

Annual operating budgets were between $1 million and $13.6 billion (mean: $879.9 million).

Scope of NDTs Considered

Centralized organizational processes were identified for new capital or new non-capital technology, NDTs for off-label use or those not licensed for sale in Canada and existing NDTs. Consumables and supplies were excluded because their processes varied widely, even within a single organization, and were less complex in terms of the factors considered and implications for the organization. Technologies within a previously approved budget were also excluded, since the extent of decision-making required was largely limited to a purchasing authorization, requiring little additional review.

Capital/Non-capital

Fifty-five centralized decision-making processes were identified: 28 (51%) for capital equipment only, three (6%) for non-capital equipment only and 24 (43%) for both. Ten were applicable to NDTs within a specific program area only (e.g., diagnostic imaging, interventional radiology or ambulatory care technologies).

Off-label Use

Only four organizations had established processes for identifying and addressing off-label use (three academic teaching centres and one RHA that included an academic teaching centre).

Unlicensed Technologies

Health Canada's Special Access Program (SAP) allows healthcare providers access to technologies that have yet to receive regulatory approval. Seventeen (36%) organizations had used SAP to introduce new NDTs; 14 were academic centres, which viewed the SAP as an important tool to support innovation in their efforts to be recognized as “cutting-edge” institutions.

Existing Technologies

In 10 organizations (21%), decision-making processes included some mechanism for reviewing existing technologies. However, these were usually informal and applied on an ad-hoc basis. In most cases, such technologies were identified as a result of utilization monitoring. None of the organizations had implemented separate, explicit processes for managing the exit of NDTs or for making disinvestment decisions in a systematic way.

Structure

Formal/Informal

Almost all of the processes identified by organizations were specified by the respondents as “formal,” and about two-thirds were tied to annual budget planning.

Timing of Decisions

Across organizations, requests for NDTs had been made on an ad-hoc basis or at regular scheduled periods. Often, ad-hoc requests arose through donors and were for consumables or low-cost or replacement technologies. Timed acquisitions were typically for capital equipment or new initiatives (10 of 55 processes [18%]). The majority of processes (41) were used for both ad-hoc and planned decision-making and linked to the organization's annual budget planning cycle. Typically, NDTs with a larger impact (not explicitly defined) or above a certain cost were a part of the annual process.

Process

NDT decision-making processes typically had four sequential components: (1) initiation of requests, (2) information requirements, (3) development of recommendations and (4) formulation of decisions.

Request Initiation

In all cases, physicians or clinical program leaders could formally propose a new NDT. In almost half of the processes, formal requests (usually requiring the completion of a standard template or preparation of a business case) from other healthcare professionals were also accepted. Requests were forwarded to a committee (42 cases), an individual (two cases) or both (13 cases), depending on whether separate processes exist for capital equipment or multiple organizational levels are involved and the potential budget impact (cost threshold). Committee included (at a minimum) physicians and clinical managers/department directors.

Information Requirements

All processes considered information on safety and budget impact, and most relied on evidence of effectiveness and cost-effectiveness (Table 2 available at longwoods.com/content/25936). Patient preference information was considered in nearly half of the processes when available, but none routinely collected it in a systematic way. The most common sources of information were expert clinical opinion, peer-reviewed literature and regulatory documents. Whereas two-thirds of the processes used HTA reports from Canadian HTA bodies (i.e., documents containing findings from technology assessments conducted using well-established HTA methods), an almost equal proportion considered promotional material from manufacturers. Sources of information on public/patient preferences were organizational councils or groups with patient/public members. Only three processes appeared to have a minimum evidence threshold requirement, although the threshold was not specified by respondents.

Recommendations

Recommendations were formulated by multidisciplinary committees with representation from medical and non-medical departments/programs (Table 3 available at longwoods.com/content/25936). However, the breadth of representation from non-medical departments/programs appeared to be greater for capital planning/equipment committees. All processes required evidence of clinical benefit and alignment of the NDT with organizational priorities. The vast majority also took into account cost/affordability/sustainability, regulatory status and speed of uptake of the technology. The last, which relates to the “learning curve,” is particularly relevant to NDTs, as their effectiveness relies on characteristics of the user (the healthcare provider) and the system within which the technology is used. In multi-site organizations, equity across institutions was an important consideration. Political factors (desire to satisfy stakeholders, consumer demand and prestige of requestor/technology) also played a role, and according to respondents, it could be a significant one when decisions were not “clear-cut.” About half of the processes, predominantly in academic centres, considered innovativeness/economic development. Respondents expressed a desire to be at the “leading edge” of research and innovation.

Decisions

Final decisions were made by either individuals, such as a senior executive, or groups, such as the senior executive/management/leadership team, the board of the organization or government; this depended on whether the decision pertained to capital equipment and on the type of organization. Innovative funding arrangements for new NDTs (excluding philanthropic funding), such as risk-sharing, were not commonly reported. The exceptions were four organizations in which a manufacturer had provided the equipment and a higher price was paid for the consumables/supplies and six organizations in which lease arrangement contracts were established with the manufacturer.

Evaluation of Decisions

The implementation of specific new NDTs was reported as being evaluated (usually on an ad-hoc basis) in two-thirds of the organizations. A lack of capacity was cited by the remaining organizations as the reason for the lack of post-implementation evaluation.

Other Information from the Surveys

Organizational Considerations

Many respondents reported that their organizations were “in a state of evolution.” At the time of the study, two were in the midst of major structural changes. About one-quarter indicated that they are moving toward more standardized processes for NDTs. An equal number mentioned that the study had prompted them to consider developing not only formal post-implementation evaluation processes but also the capacity needed to achieve successful change management related to NDTs.

Centralized/Provincial Processes

Organizations that used HTA reports as a source of information mentioned limitations of such reports. Although they provided clear and concise evidence of clinical effectiveness, they lacked the context-specific information required for decision-making. This suggests that a single HTA capable of addressing the needs of many health organizations may not be adequate.

Other Challenges

Regardless of policies and processes that may exist, there always appeared to be ways of getting around them, particularly in the case of smaller, less-expensive items (e.g., surgical tools). To quote, “smart advocates can get most things done through this approach.” An additional concern was funding for new NDTs. Many organizations relied on philanthropic foundations for the financial resources needed to acquire one-off innovative and expensive technologies.

Limitations

The response rate was 20%, lower than recently published expected values of approximately 30% for “top management” (Anseel et al. 2010). This may be explained by the fact that the study was commissioned during a time when several healthcare systems were undergoing significant changes, particularly in Quebec and Nova Scotia. Organizational structures and roles were being modified, making it difficult to identify an individual with the appropriate level of accountability who could meaningfully participate in the survey. Also, based on the differing positions held by interview participants, senior executives (to whom the initial survey was sent) may not always have been involved enough in NDT decision-making to feel comfortable responding to questions about their processes. The low response rate may well limit the generalizability of the findings of this study.

The results of this study were based on information from members of organizations who agreed to participate. Whether other individuals within the same organization would have provided the same responses is unknown.

Discussion

This study was undertaken to address a gap in our knowledge of how individual health organizations manage decisions on new NDTs across Canada, including current “decision points” in the organization and the types of data and information required. Such information is needed to determine whether a centralized HTA process for NDTs in Canada is feasible.

The last Canadian study on technology decision-making in health institutions was published 25 years ago (Deber et al. 1994). The authors concluded that most of the decisions on new technologies were made by administrative, medical, board or mixed committees. In addition, they found that technical experts, such as biomedical engineers, played a minimal role. Our study also demonstrated that committees play a major role in NDT decision-making. Regarding the involvement of technical experts, little has changed in the past 25 years. Their involvement remains minimal. In contrast, the use of information on clinical effectiveness and cost-effectiveness appears to have increased significantly. This is likely attributable to the advent of “evidence-based medicine” in the mid-1990s, investments in the development of HTA capacity and improved access to HTA information over this period.

Studies on factors addressed in HTAs used to support NDT decision-making elsewhere in the world have produced results consistent with those of this study. In a review of official documents from member organizations of the International Network of Agencies in HTA from nine countries, seven attributes of HTA were identified for organizational decision-making: clinical (safety and efficacy), economic (comparing costs and benefits), social and ethical, organizational (such as clinical expertise, training, environment, culture), innovation and “admissibility” (defined as regulatory factors) (Usaquén-Perilla et al. 2017). In a university medical centre in the Netherlands, HTA was expected to address the consequences of adopting a new NDT on the hospital as an organization (van der Wilt et al. 2016). Therefore, questions such as “What are the implications on operating room flow?”, “Is training required?”, “Are there training and research opportunities?”, “Is this in line with the strategic direction of the institution?” and “What might it replace locally?” needed to be addressed. Finally, a survey of more than 100 hospital managers in Europe concluded that institutional NDT decision-making requires information on the organizational, strategic and political implications of adoption, and that the focus of any economic evaluation should take the perspective of the institution, not just a societal perspective, which is typically what provincial/state HTA bodies adopt (Kidholm 2016).

Issues pertaining to the introduction and use of NDTs (e.g., prestige of a healthcare organization and ties to physician reimbursement) are, in general, more complex than those surrounding drug therapies. Organizations have created multiple levels of scrutiny for managing NDT requests, which involve different steps and types of information. As a result, different NDT issues challenge organizations at any single time point. Whereas the findings on clinical safety and effectiveness of a new NDT may be portable across organizations, the need for local contextualization (institutional priorities, local budgetary circumstances, the role in research/innovation, etc.) would make the HTA requirements of an organization somewhat unique. Consequently, a centralized pan-Canadian review of NDTs, unless it only addresses clinical effectiveness of a technology, may offer limited value.

Conclusions

The results of this study provide important “baseline” information on the current state of NDT decision-making in healthcare organizations across Canada. Such information is critical to the development of relevant, feasible strategies for managing NDTs in Canada.