The Missing Stakeholder Group: Why Patients Should be Involved in Health Economic Modelling

Abstract

Evaluations of healthcare interventions, e.g. new drugs or other new treatment strategies, commonly include a cost-effectiveness analysis (CEA) that is based on the application of health economic (HE) models. As end users, patients are important stakeholders regarding the outcomes of CEAs, yet their knowledge of HE model development and application, or their involvement therein, is absent. This paper considers possible benefits and risks of patient involvement in HE model development and application for modellers and patients. An exploratory review of the literature has been performed on stakeholder-involved modelling in various disciplines. In addition, Dutch patient experts have been interviewed about their experience in, and opinion about, the application of HE models. Patients have little to no knowledge of HE models and are seldom involved in HE model development and application. Benefits of becoming involved would include a greater understanding and possible acceptance by patients of HE model application, improved model validation, and a more direct infusion of patient expertise. Risks would include patient bias and increased costs of modelling. Patient involvement in HE modelling seems to carry several benefits as well as risks. We claim that the benefits may outweigh the risks and that patients should become involved.

Key Points for Decision Makers

Health economic models play an important role in assessments of healthcare interventions.

Patients are important stakeholders who can add to health economic modelling.

Ways of involving patients in health economic modelling should be explored further.

Introduction

Health technology assessment (HTA) has been used as a way to limit the trend of increasing healthcare expenditures in many European countries since the 1990s [1, 2]. Many countries use cost effectiveness as an important formal (e.g. Austria, Belgium, the Netherlands, and Sweden) or informal (e.g. France) criterion for HTA information in their evaluation process [3]. The clearest example is provided by the UK, where official thresholds for cost effectiveness play a structured role [4].

Many cost-effectiveness analyses (CEAs) use health economic (HE) models to obtain estimates of costs and benefits of new interventions. Validation (the evaluation of ‘whether [a] model is a proper and sufficient representation of the system it is intended to represent’ [5]) is therefore emphasized by guidelines on good HE modelling practice [6–11]. These guidelines mostly cover the scientific credibility of the models. Experience from other fields like climate research, nature policy assessments, and water management shows that validation also entails other aspects related to fitness for purpose. These are stakeholder perception of model salience (model applicability within the context) and legitimacy (the inclusion of stakeholder concerns, values and views in a proper way) [12]. Hence, stakeholder involvement is considered to be a critical aspect of good modelling practice [12–21].

Patients are one group of stakeholders that is obviously important in terms of HTA decisions. But while the validation of HE models has become a point of progression in HE literature, the involvement of patients in modelling appears to remain underappreciated. HE models play an implicit role by providing relevant information for the reimbursement decision process. However, they often are a ‘black box’ to patients, and one can argue there may be a moral obligation to further open this ‘black box’ and involve patients in the development and application of HE models. Moreover, many patients have become experts who can provide valuable information for HE modellers. Their involvement could improve the credibility, salience and legitimacy of HE models. The relevance of this issue is even higher as HE models are used in the early stages of HTA, affecting the downstream drug-evaluation processes.

In this opinion paper, we discuss benefits and risks of patient involvement in HE model development and application as well as possible ways of involving patients. We conducted an exploratory literature search to gain insight into the issues at stake and arguments applied. An extensive literature exists about stakeholder involvement in modelling, but only few studies report on the current area of interest (HE decision modelling). In addition, discussions were held with patient experts (see the Sect. 6 for more details). In this paper, double quotes (“…”) indicate arguments made by these patient experts (where it has been attempted to retain as much as possible of the original meaning in the translation from Dutch to English).

Why Involve Patients in Health Economic (HE) Modelling?

We distinguish between arguments in favour of patient involvement based on improving model credibility, salience and legitimacy [12].

Model credibility can be improved by communication about uncertainties in the model design and the inclusion or exclusion of factors (such as side effects, time horizon, or capturing of cost, to name a few). The few patients who are aware of HE modelling feel like “health care calculations are too narrow, excluding various second-order costs and benefits from treatments,” as “there are many patients that suffer from several chronic diseases simultaneously, and for whom CEAs could be positive that are otherwise negative.” Patient involvement can improve mutual understanding and model designs. Patient involvement “may make modellers aware of things they have not considered” while patients “ask the naive questions” and “contribute knowledge about co-occurring diseases and side-effects of drugs.” At the same time “patients see the calculations and become more aware of medical costs” and learn about the consequences of model-based decisions [14].

Model salience can be improved by jointly selecting model performance criteria [15, 16] and appropriate indicators for decision making [15]. For instance, patients indicate that “the standardization of patients is a risky affair, and HE model output should cover patient heterogeneity.” Confidence limits of model output contain essential information for patients and modellers alike to judge the usefulness of the model output for decision making [13]. The inclusion of ‘policy levers’ that represent typical options for decision makers [15] and the development of computer interfaces for model usability [17] also improve salience.

Model legitimacy can also improve with patient involvement. Only few patients are currently aware of the use or existence of HE models. Patient experts indicate that they expect that “HE modellers will be fair and trust-worthy because of their medical background,” but “there may be a sort of attic room reality created by modellers.” Whether or not this is a valid perception, it indicates that the legitimacy of HE models can be much improved. Experience shows that the inclusion of stakeholders can create a feeling of ‘ownership’ regarding the findings, while a lack of stakeholder involvement creates a feeling of ‘not invented here’ and a dismissal of (model-based) outcomes [22], or, in the words of patients, “We would like to have a say about what goes into our bodies.”

How Can We Involve Patients in HE Modelling?

Involving patients in HE modelling should maximize the benefits and minimize the risks. Research on this is needed. Literature on trans-disciplinary science and stakeholder involvement in modelling provides guidelines on how to involve stakeholders [13–15]. These guidelines may or may not equally apply to patient involvement. The patients we interviewed also provided suggestions on what would be more or less likely to work.

A vital factor for the success of modelling projects is a continuing inclusion of patients, i.e. from start to finish [13, 14], and with regular interactions between modellers and patients in which results can be presented, questions can be asked, and answers and feedback can be given. Patients can likely be involved in the following modelling aspects [15]:

1. The problem formulation, including the establishment of a shared terminology and an agreement on the goal of the model;

2. Setting model performance requirements, based on an inventory of what is expected of the model in terms of scientific credibility, its application, and the views, values and concerns of patients and other stakeholders;

3. The design of the conceptual model, for instance as was done by the School of Health and Related Research (ScHARR), UK, through a number of sessions in which patients participated in the design phase of the conceptual model of two diabetes-prevention models [23];

4. Setting model applicability limitations, i.e. it is understood by patients what the model cannot do;

5. A selection of appropriate indicators for decision making that are understood by patients; and

6. A discussion about uncertainties in the model, their effect on model output, and the ramifications of these uncertainties for decision making.

HE models are most likely to benefit from patients “who are capable of abstract thinking and can rise above their own disease.” Patients “who followed courses on HE modelling can translate model outcomes for other patients.” At the same time, it is also “a responsibility of the modellers to explain their work to patients.” Clear and concise documentation on model assumptions, limitations, and alternative model designs should be available for communication between modellers and patients [14, 15]. The discussion between modellers and patients can also benefit from user interfaces to ‘play’ with the model [17, 18].

Risks, Drawbacks, and Limitations

Patient involvement in the HE modelling process may also be subject to risks, drawbacks, and limitations. Many of these were indicated by the patients we interviewed.

One obvious limitation is that “most patients are not even aware of the use of HE models.” They “recognize the evaluation process, but not what is before that,” i.e. the use of HE models for HTA. Patients will thus have to be educated in this regard. A current limitation for education is mentioned, “at the moment there is very good documentation available at the website of Zorginstituut Nederland [the Dutch National Health Care Institute that advises the Dutch government with regard to reimbursement decisions], but one cannot access material about HE models used in CEAs.”

A second important limitation is that patient involvement will require “a serious investment in terms of time and money for patients to get to a level at which they can really contribute.” The same also applies for modellers. Such an investment of resources is not free of risks and, furthermore, it is unclear who should supply these resources in what way. One risk is that “many patients will not be suited for [an involvement in HE modelling], as they will not be able to rise above their disease.” It is therefore indicated that “we need patients who are able and willing to dive into this material out of interest, and who can have an objective view.” A proper selection procedure for such patients will have to be developed.

Despite the additional costs, it is expected that “the cost-effectiveness of patient involvement in HE modelling may be positive in the long run” as the benefits eventually outweigh the initial investment costs. In particular, “the involvement of patients may reduce the time before availability through faster acceptance by the intended users and an improved knowledge transfer,” i.e. the additional costs of patient involvement may (partly) be recovered from savings in time or costs further downstream of the decision process, since the model is more acceptable, credible and legitimate. In addition, “there is a lot of knowledge to share that isn’t made available at the moment.” As patients get educated and develop experience through being involved in modelling, they “could play an important role in the spread of experience and knowledge that is relevant also for CEAs of different diseases.”

The inclusion of patients in HE models is a potential source of bias towards patient views. Such bias could for instance be that certain side effects are over-emphasized by involved patient(s). Patients may also be viewed upon as shilling for certain interventions. Such bias can be avoided by including patients who are able to take a neutral view, and by the inclusion of “at least five patients who differ significantly in their background.”

Concluding Opinion

Patients are key stakeholders in HE modelling, yet their involvement is sparse at best. A positive exception is the ScHARR project [23]. One can argue there is a moral obligation to involve patients in HE model development and application because of the hidden impact that HE model-based decisions can have on their lives. Also, there may be significant benefits if patients were to be commonly involved in modelling. These benefits include an expected increase in the validation status of HE models through improved model credibility, salience, and legitimacy, through a greater understanding and possible acceptance by patients of HE model applications and a more direct infusion of patient expertise. There are also risks associated with patient involvement, most notably a possible patient bias and an increased cost of modelling, at least in the short run. We believe that the benefits and obligations outweigh the risks and that the need for patient involvement in HE modelling is obvious.

What remains is the matter of how to involve patients in HE modelling in a good way. The involvement of stakeholders in modelling has rapidly become common in other scientific disciplines in the past decade (as suggested by the vast literature on stakeholder involvement in modelling [12–21]) and, although many issues are mentioned, overall the effect of stakeholder involvement in modelling seems to be positive. Yet, the HE modelling field seems to be lagging behind in this respect when compared with, for instance, the fields of commerce and defence [24] or environmental sciences. It seems wasteful to re-invent the wheel; there are sufficient guidelines on stakeholder-involved modelling in other scientific disciplines that may be applicable for patient involvement in HE modelling.

Both additional research and investments of resources are required to establish effective ways of patient involvement in the field of HE modelling. At the very least, it requires the training of both modellers and patient experts to communicate effectively with each other. We argue that effort should be spent towards these goals.

Appendix

This opinion paper presents an exploratory spin-off of a larger project on HE model validation [5]. The references from outside the HE modelling discipline are based on the experience and work of the first author in ecological assessments. This work included a search in Google Scholar using a combination of the term ‘stakeholder’ with either ‘modelling’ or ‘assessment’. The 70,000+ hits has been reduced through the use of more advanced terms, such as ‘participatory modelling’, ‘credibility’, ‘salience’, ‘legitimacy’, and ‘trans-disciplinary science’. References [12–21] present a very compacted overview of this search and present important lessons to be learned on stakeholder involvement that are likely equally relevant for HE modelling.

In addition, one or more interviews have been conducted with seven Dutch patient experts (all with a chronic illness) in different formats (in person, in a focus group, by phone, and via e-mail) about their experiences with CEAs and their perceptions on collaboration between patients and HE modellers. These patients were contacted in a limited timeframe through advertisements in patient expert groups active in different areas, and ‘snow-balling’, i.e. patients who knew patients who might be interested. All interviewed patients indicated they had rather in-depth knowledge of the Dutch drug reimbursement process. They mostly knew that part of the information they received in preparing the reimbursement advice was based on HE models, but indicated that they knew little to nothing about the particular HE models used. At least one patient had taken courses on the topic to gain in-depth knowledge of HE modelling but indicated there is no transparency with regard to models. The interviewed patients thus represented a small but appropriate subset of patients who are (potentially) interested in learning about HE model development and application, who in turn are a subset of patients with knowledge about the drug reimbursement process in the Netherlands. Because of the limited, exploratory character of the research, no systematic efforts were undertaken to avoid bias of any type (e.g. age, sex, type of illness, etc.).

Compliance with Ethical Standards

The research was funded by the Netherlands Organization for Health Research and Development (ZonMW), project no. 152002050. Maiwenn Al and Talitha Feenstra declare they are members of the scientific advisory committee (WAR) of the National Healthcare Institute (Zorginstituut Nederland). George van Voorn, Pepijn Vemer, Dominique Hamerlijnck, Isaac Corro Ramos, Geertruida J. Teunissen, Maiwenn Al, and Talitha Feenstra declare there are no further conflicts of interest. This paper did not require institutional ethics committee approval. As far as they did apply to the current research, the study has been performed in accordance with the ethical standards of the Declaration of Helsinki. Informed consent regarding correctness of quotes and permission to use these in the manuscript was obtained from all patient experts included in the study.

Author contributions

TF, GvV, and MA initiated the overall project and obtained funding. GvV and TF developed the original idea for the study. GvV, PV, DH, GJT, and TF were involved in (organizing) the meetings with patients. DH and GJT contributed their expertise as patient experts. GvV wrote the initial draft, while all co-authors contributed to the writing and discussed interview results.