Skip to main content
BMJ Open logoLink to BMJ Open
. 2019 Jan 30;9(1):e021832. doi: 10.1136/bmjopen-2018-021832

Economic impact of potentially inappropriate prescribing and related adverse events in older people: a cost-utility analysis using Markov models

Frank Moriarty 1, Caitriona Cahir 2, Kathleen Bennett 2, Tom Fahey 1
PMCID: PMC6359741  PMID: 30705233

Abstract

Objectives

To determine the economic impact of three drugs commonly involved in potentially inappropriate prescribing (PIP) in adults aged ≥65 years, including their adverse effects (AEs): long-term use of non-steroidal anti-inflammatory drugs (NSAIDs), benzodiazepines and proton pump inhibitors (PPIs) at maximal dose; to assess cost-effectiveness of potential interventions to reduce PIP of each drug.

Design

Cost-utility analysis. We developed Markov models incorporating the AEs of each PIP, populated with published estimates of probabilities, health system costs (in 2014 euro) and utilities.

Participants

A hypothetical cohort of 65 year olds analysed over 35 1-year cycles with discounting at 5% per year.

Outcome measures

Incremental cost, quality-adjusted life years (QALYs) and incremental cost-effectiveness ratios with 95% credible intervals (CIs, generated in probabilistic sensitivity analysis) between each PIP and an appropriate alternative strategy. Models were then used to evaluate the cost-effectiveness of potential interventions to reduce PIP for each of the three drug classes.

Results

All three PIP drugs and their AEs are associated with greater cost and fewer QALYs compared with alternatives. The largest reduction in QALYs and incremental cost was for benzodiazepines compared with no sedative medication (€3470, 95% CI €2434 to €5001; −0.07 QALYs, 95% CI −0.089 to –0.047), followed by NSAIDs relative to paracetamol (€806, 95% CI €415 and €1346; −0.07 QALYs, 95% CI −0.131 to –0.026), and maximal dose PPIs compared with maintenance dose PPIs (€989, 95% CI -€69 and €2127; −0.01 QALYs, 95% CI −0.029 to 0.003). For interventions to reduce PIP, at a willingness-to-pay of €45 000 per QALY, targeting NSAIDs would be cost-effective up to the highest intervention cost per person of €1971. For benzodiazepine and PPI interventions, the equivalent cost was €1480 and €831, respectively.

Conclusions

Long-term benzodiazepine and NSAID prescribing are associated with significantly increased costs and reduced QALYs. Targeting inappropriate NSAID prescribing appears to be the most cost-effective PIP intervention.

Keywords: elderly, medications, appropriateness, economic evaluation, stopp


Strengths and limitations of this study.

  • This study represents a novel application of economic modelling methods to assess three common types of potentially inappropriate prescribing.

  • Analysis included the principal adverse effects of each potentially inappropriate medication.

  • Uncertainty of estimates was quantified using probabilistic sensitivity analysis.

  • The study did not consider differences in adverse event risk among individual drugs within each class or heterogeneity in economic impact among patient subgroups.

Introduction

Potentially inappropriate prescribing (PIP), the use of medicines where the risks outweigh the benefits, is prevalent among adults aged ≥65 years, particularly in individuals taking multiple medicines or with multiple chronic conditions.1 2 Several explicit measures of PIP have been developed, including Beers criteria and the Screening Tool for Older Person’s Prescriptions (STOPP), and while their relationship with some patient outcomes has been evaluated, the effect on the wider health system is also important to consider, in particular on healthcare costs.3 The use of potentially inappropriate medicines can have an impact on healthcare costs due to pharmaceutical expenditure relating to the prescriptions themselves and due to managing the adverse events which may result. In two systematic reviews, one of studies assessing the STOPP criteria and another on the economic impact of inappropriate drug prescribing more generally, only direct medication costs of PIP drugs were assessed.3 4 Increased life expectancy has called into question the use of 65 years and above as a threshold for old age; however, the literature on PIP (including STOPP) still focuses on this population due to physiological changes in ageing and the prevalence of multiple comorbidities which can predispose to medication harm.3

Furthermore, in only assessing the direct cost of inappropriate drugs, the economic consequences of appropriate prescriptions used as an alternative to PIP medicines are not accounted for.4 5 The costs of managing any resulting adverse events have yet to be quantified for PIP as a whole and have only been assessed for individual medication classes to date, such as benzodiazepines and non-steroidal anti-inflammatory drugs (NSAIDs).6–8 The economic impact of PIP is important when considering whether interventions to reduce PIP are an efficient use of resources and health professionals’ time relative to other competing priorities. Few economic evaluations of trials to optimise prescribing for older people have been published,3 9 10 which may limit implementation of such interventions by decision-makers, given scarce healthcare resources.

A recent analysis of PIP among older adults in Ireland found that the most common indicators related to long-term use of NSAIDs, benzodiazepines and maximal dose proton pump inhibitors (PPIs).2 NSAIDs are indicated for treating pain in arthritis and low back pain for example; however, due to their gastrointestinal and cardiovascular risks, they are not recommended for long-term use. Benzodiazepines are sedative agents used to treat insomnia, but carry risks of day-time drowsiness as well as tolerance and dependence following long-term use. PPIs are used for gastrointestinal conditions such as peptic ulcer disease and gastro-oesophageal reflux disease. While maximal doses are indicated for up to 8 weeks in the majority of cases, following this a maintenance dose has comparable efficacy if continued treatment is necessary. Despite strong evidence that the balance of benefits and harms for such prescriptions is unfavourable, the prevalence of these indicators ranged from 4% to 24% in a primary care population analysis (where most prescribing of these agents occurs).2

The aim of this study is to estimate and compare the economic impact of these three common indicators of PIP: long-term use of NSAIDs, benzodiazepines and maximal dose PPIs. Specifically, we compare each of the three PIP drugs to a more appropriate treatment using Markov models to assess differences in quality and quantity of life and cost to the health system. We then apply the models to explore the cost-effectiveness of potential interventions based on recently published trials targeting these PIP drugs.

Methods

Markov models

The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement was used in the design and reporting of this research (included as online supplementary appendix 1).11 A Markov model was developed for each of the included PIP drugs using TreeAge Pro 2015 (TreeAge Software, Williamstown, Massachusetts, USA). This type of decision-analytic model was chosen to allow for time dependency, a particularly important consideration in the context of older people on long-term medicines.12 The base case analysis used a target population of hypothetical 65 year olds who were community-dwelling in Ireland and had no current or previous adverse events relating to these PIP drugs. A health system perspective was used over a time horizon of 35 1-year cycles (ie, to age 100) with a half cycle correction.13 This perspective is recommended in national guidelines on economic evaluation,14 and therefore only direct costs to the health system (including those relating to residential care) were considered. The primary decision maker is therefore Ireland’s Health Service Executive which makes funding allocation decisions relating to health technologies. In each of the three cases, the PIP strategy was compared with an alternative strategy, selected as an appropriate therapeutic option instead of the PIP drug (with respect to effectiveness and safety). The models incorporated the principal adverse drug events relating to each PIP (see table 1). The primary outcomes evaluated were costs and quality-adjusted life years (QALYs). Life years (LYs) and number/rate of adverse events were also quantified as secondary outcomes. A discount rate for costs, QALYs and LYs was applied at 5% per annum and was varied from 0% to 6% in sensitivity analysis, in line with guideline recommendations.14

Table 1.

Description of included criteria from the STOPP

Potentially inappropriate prescription Comparator Prevalence2 Adverse events represented
NSAID >3 months Paracetamol 4.1% Dyspepsia
Gastrointestinal bleed
Myocardial infarction
Benzodiazepine ≥4 weeks No sedative medication 4.3% Hip fracture
Other fall injuries
PPI maximal dose >8 weeks Maintenance dose PPI 23.6% Hip fracture
Clostridium difficile infection

NSAID, non-steroidal anti-inflammatory drug; STOPP, Screening Tool for Older Persons’ Prescriptions.

Supplementary data

bmjopen-2018-021832supp001.pdf (288KB, pdf)

This cohort consisted of healthy community-dwelling older people; therefore in each model, all individuals start in a ‘Well’ state (see online supplementary appendix 2, figure A1 for state transition diagrams for each model). In subsequent cycles, individuals could transition to other states as a result of adverse events relating to the potentially inappropriate medicines of interest. Individuals remain in the adverse event state for one cycle unless they have a further adverse event in the subsequent cycle, and otherwise they transition to the postevent state (if applicable) or the relevant ‘Well’ state. Mortality attributable to adverse events and background age-related mortality were included. An in-depth description of the structure and transitions for each model is included in section 1 of online supplementary appendix 2. The models were populated with parameter estimates (see online supplementary appendix 2, table A1) derived from published sources which are described in detail in section 2 of online supplementary appendix 2. As this study used only previously published data, there was no requirement for ethical approval or patient consent.

Supplementary data

bmjopen-2018-021832supp002.pdf (1.9MB, pdf)

Model inputs

Transition probabilities

Probabilities of transitions between states for the three models were taken from published literature sources which reported rates or probabilities of the adverse events of interest. Population-based epidemiological studies with study samples representative of older community-dwelling adults were used, whenever possible, reflecting the baseline rate of adverse events for individuals in the appropriate alternative models (see Table A1, online supplementary appendix 2). In the PIP models, a measure of the relative risk associated with the PIP drug was applied to the baseline probability for each adverse event. These were taken from meta-analyses of randomised controlled trials for NSAIDs,15–17 meta-analyses of observational studies for benzodiazepines18 19 and for PPIs from a meta-analysis of observational studies20 and a single observational study.21 Annual probability of death from all causes was based on age-specific population rates for 2014 from the Central Statistics Office (CSO).22 Excess mortality estimates following adverse events were taken from observational studies23–28 and were assumed to be independent of PIP exposure (ie, the same postevent mortality was applied in both PIP and alternative scenarios).

Utility values

To increase comparability between the models, the same baseline utility value was applied to all ‘Well’ or no event health states. The source of these values were UK population norms for the EQ-5D visual analogue scale for people aged 65–74 and 75 years and over.29 Utility decrements or disutilities, the annual reduction in utility due to an adverse event were taken from previous economic evaluations or studies that derived these values from patients with the relevant adverse event.9 30–43 These were subtracted from this baseline utility to give the utility value for each state. Further details of these are provided in online supplementary appendix 2, section 2.

Costs

Each state was assigned a cost reflecting the average annual costs to the Irish health system for a patient in that health state, relating to hospital inpatient care, general practitioner, outpatient department and emergency department visits, medicines and long-term (residential) care. Costs in euro from 2014 were used and, where not available, historical costs were inflated using the applicable Consumer Price Index (CPI) Health sub index from the CSO. In the case of C. difficile infection, international estimates of attributable costs were inflated to 2014 costs using the CPI from the origin country and were then converted to Irish costs using the Purchasing Power Parity index.14 Additional healthcare use attributable to adverse events was identified from published studies and Irish unit costs were assigned.44

Assumptions

It was assumed that prescribed medicines were consumed (ie, full adherence) and over-the-counter use was not included in the models. Health states only related to the adverse events of each PIP, so it was assumed that there was no significant differences in efficacy between each PIP and the appropriate alternative, and no significant adverse effects of the appropriate alternative. In the NSAID model, following an adverse event, it was assumed that individuals would be switched to an appropriate alternative. In the other models, it was assumed that individuals remained on therapy regardless of adverse events, due to unlikely attribution of the adverse events in the case of PPIs and dependence and withdrawal effects in the case of benzodiazepines. The effect of this assumption was assessed in structural sensitivity analysis.

Analytic methods

Economic impact of PIP relative to appropriate alternatives

Model structures were assessed for face validity by the research team and models were cross-validated by comparison to other published models concerning these therapeutic areas.45 Models were validated by double-programming in Microsoft Excel 2010 (Microsoft, Redmond, Washington, USA) to detect structural or coding errors, and extreme value testing and comparison of cohort traces between TreeAge Pro and Excel were also conducted.45 Only the base case analyses were programmed in Excel. The models programmed in Excel are available from https://doi.org/10.6084/m9.figshare.5818251.v1, and TreeAge Pro model structures are included as Figures A2-4 in section 3, online supplementary appendix 2.

Base case models were run for the PIP and appropriate scenarios using point estimates for transition probabilities, costs and utilities (as shown in Table A1, online supplementary appendix 2) and results are presented as mean differences in costs, QALYs and LYs. An incremental cost-effectiveness ratio (ICER) was also calculated for each PIP, indicating the expected additional cost per additional QALY in the PIP scenario relative to the appropriate alternative scenario. Differences in the total number of adverse events for the PIP scenario compared with the appropriate scenario were also determined. Uncertainty associated with imprecision of the parameter inputs was incorporated into the model using probabilistic sensitivity analysis (PSA) to allow 95% credible intervals (CIs) to be fitted (see online supplementary appendix 2, section 4 for further details). The impact of varying specific parameter inputs, including costs and discount rates, was assessed in one-way deterministic sensitivity analyses.14 Although not prespecified, we also considered treatment adherence in one-way sensitivity analysis. Up to 20% non-adherence was assessed, which applied a reduction to medication costs and a reduction in the proportion within each state who were exposed to the medication and the associated relative risk of adverse events.

Cost-effectiveness of potential interventions

In the second stage of the analysis, each model was used to evaluate the cost-effectiveness of a potential intervention to reduce prescribing of each PIP drug by switching patients to the more appropriate alternative. This analysis was in the form of a value of implementation analysis,46 and a new decision was framed between implementing an intervention to reduce PIP or usual care, as illustrated for NSAIDs in Figure A5 in online supplementary appendix 2, section 5. The intervention was delivered once at the beginning of the model to all individuals on a long-term NSAID and resulted in a proportion of these people being switched to paracetamol for the duration of the model time horizon. The intervention cost per person and effectiveness (ie, the relative reduction in the proportion on a long-term NSAID) were varied to determine circumstances in which the intervention would be preferred to no intervention at a willingness-to-pay or cost-effectiveness threshold of €45 000/QALY (the conventionally used threshold in Ireland)14 as well as thresholds of €20 000/QALY and €0/QALY. These results were plotted and this was then repeated for benzodiazepine and PPIs. Threshold analysis was conducted using effectiveness estimates from recent primary care trials targeting these PIP drugs which have no published economic evaluation to date to determine maximal costs at which each medicines optimisation intervention would be cost-effective (see section 5 of online supplementary appendix 2 for a description of these trials).47–49

Patient involvement

Patients were not involved in the conception, design or conduct of this research.

Results

Economic impact of PIP relative to appropriate alternatives

Based on the study parameters used (Table A1, online supplementary appendix 2), for all three models the PIP scenarios were dominated by the appropriate treatment scenarios (ie, they generated higher costs and fewer QALYs). The incremental costs and QALYs were largest in the benzodiazepine model, where being on the PIP drug generated an average of €3470 higher costs and 0.07 fewer QALYs per patient compared with the appropriate alternative scenario (table 2). For costs, this was followed by patients on a long-term maximal dose PPI relative to those on a maintenance dose and then being on long-term NSAIDs compared with paracetamol. The QALY loss in the NSAID model was 0.07 QALYs and 0.01 QALYs in the PPI model. The excess adverse events in the PIP scenarios relative to the appropriate alternative scenarios are shown in Table A2 (online supplementary appendix 3). Uncertainty in the outcomes is illustrated in figure 1 showing the distribution of cost and QALY differences for each model in the PSA. The 95% CIs generated from the PSA showed incremental costs and QALY losses were statistically significant for the NSAID (95% CI €415 to €1346 costs; −0.131 to −0.026 QALYs) and benzodiazepine models (95% CI €2434 to €5001 costs; −0.089 to −0.047 QALYs). For the PPI model, the difference in costs and QALYs between maximal dose and maintenance dose prescribing was not statistically significant (95% CI -€69 to €2127 costs; −0.029 to 0.003 QALYs).

Table 2.

Cost, effect and ICER outputs for PIP compared with appropriate scenarios for each model

Strategy Cost, € Incr. cost, € (95% CI) QALYs Incr. QALYs (95% CI) ICER, €/QALY LYs Incr. LYs
NSAID model
 Paracetamol >3 months 2603 8.72 11.54
 NSAID for >3 months 3409 806 (415 to 1346) 8.65 −0.07 (−0.131 to −0.026) −11 511 11.46 −0.08
Benzodiazepine model
 No benzodiazepine 25 158 8.78 11.69
 Benzodiazepine ≥4 weeks 28 628 3470 (2434 to 5001) 8.72 −0.07 (−0.089 to −0.047) −52 672 11.65 −0.04
PPI model
 Maintenance dose >8 weeks 24 831 8.82 11.70
 Maximal dose >8 weeks 25 819 989 (−69 to 2127) 8.81 −0.01 (−0.029 to 0.003) −85 279 11.68 −0.02

CI, credible interval; ICER, incremental cost-effectiveness ratio; LYs, life years; NSAID, non-steroidal anti-inflammatory drug; PPI, proton pump inhibitor; QALYs, quality-adjusted life years.

Figure 1.

Figure 1

Incremental costs and utilities for PIP compared with appropriate from probabilistic sensitivity analysis for each model (northwest quadrant). BDZ, benzodiazepine; NSAID, non-steroidal anti-inflammatory drug; PIP, potentially inappropriate prescribing; PPI, proton pump inhibitor; QALYs, quality-adjusted life years.

Supplementary data

bmjopen-2018-021832supp003.pdf (2.1MB, pdf)

In one-way deterministic sensitivity analysis, the PIP scenario was still dominated by the appropriate alternative scenario in each model across the range of values for the investigated parameters and the rankings of the models by incremental costs and QALYs did not change (see table 3). Similarly, the posthoc sensitivity analysis of treatment non-adherence showed a reduction in both incremental costs and QALYs with increasing non-adherence. Altering the NSAID model structure to assume no switch from the PIP drug to paracetamol after an adverse event (ie, if patients remained on a long-term NSAID regardless of adverse events occurrence, consistent with the benzodiazepine and PPI models) resulted in a larger cost difference (€1494, 95% CI €756 to €2493) and QALY difference (−0.11 QALYs, 95% CI −0.042 to −0.203) between the PIP and appropriate scenarios. The distribution of cost and QALY estimates under this assumption is plotted in figure A6 in online supplementary appendix 3.

Table 3.

One way deterministic sensitivity analysis results

NSAID model Benzodiazepine model PPI model
Incremental effect (QALYs)
 Outcome discount rate
  0 −0.157 −0.175 −0.035
  0.02 −0.111 −0.115 −0.022
  0.04 −0.082 −0.079 −0.014
  0.06 −0.061 −0.056 −0.010
 Non-adherence to treatment
  10% −0.064 −0.059 −0.011
  20% −0.058 −0.052 −0.010
Incremental cost (€)
 Costs discount rate
  0 1145.45 6497.62 1767.79
  0.02 984.56 4978.65 1379.78
  0.04 858.79 3893.76 1099.22
  0.06 758.79 3108.09 893.40
 Inpatient cost of C. difficile
  €4000.00 961.63
  €6398.72 996.79
  €8797.45 1031.94
  €11 196.17 1067.09
 PIP drug cost*
  Low 349.20 3016.20 478.15
  High 1125.73 4474.65 2166.44
 Non-PIP drug cost†
  Low 1192.38 1673.52
  High 660.57 477.64
 Non-adherence to treatment
  10% 740.56 3117.12 900.42
  20% 672.11 2765.54 810.45

*PIP drug cost range (€) NSAID: 74.82–202.00, benzodiazepine: 38.96–164.16, PPI: 117.12–261.60.

†Non-PIP drug cost range (€) NSAID: 38.40–120.00, PPI: 56.56–160.80.

ICER, incremental cost-effectiveness ratio; LYs, life years; NSAID, non-steroidal anti-inflammatory drug; PPI, proton pump inhibitor; QALYs, quality-adjusted life years.

Cost-effectiveness of potential interventions

Applying these models to determine the cost-effectiveness of potential interventions, the relationship between intervention cost, effectiveness and preferred option (intervention or usual care, ie, no intervention) is represented graphically for each PIP drug in figure 2. Additionally, see figure A7 in online supplementary appendix 3 for an example interpretation of these plots. Taking estimates of effectiveness from recently published trials targeting these PIP drugs,47–49 an intervention which reduces potentially inappropriate NSAID use by 49.8% would be cost-effective up to a cost of €1971 per person at a CE threshold of €45 000. For an intervention that resulted in 23% discontinuation among benzodiazepine users, the corresponding threshold cost would be €1480 and for a 55% reduction in potentially inappropriate PPI use it would be €831 (table 4). The rank order of these potential interventions depended on the CE threshold used. Taking the extreme case of a CE threshold of €0 per QALY (ie, willing to pay nothing additional for any QALY gain), cost-effectiveness would be achieved for interventions targeting NSAIDs, benzodiazepines and PPIs up to costs per patient of €401, €798 and €544, respectively (table 4).

Figure 2.

Figure 2

Cost and effectiveness at which interventions would be cost-effective at a cost-effectiveness threshold of €45 000 per QALY for (A) benzodiazepine, (B) PPI and (C) NSAID models. NSAID, non-steroidal anti-inflammatory drug; PIP, potentially inappropriate prescribing; PPI, proton pump inhibitor; QALYs, quality-adjusted life years.

Table 4.

Threshold values across cost-effectiveness thresholds for intervention cost at levels of effectiveness from published trials

NSAIDs Benzodiazepines PPIs
Intervention effectiveness (risk reduction)* 0.498 0.23 0.55
Threshold cost (€) at published intervention effectiveness*
WTP (€ per QALY)
 0 401 798 544
 20 000 1099 1101 671
 45 000 1971 1480 831

*Effectiveness estimates used were taken from Dreishulte et al for NSAIDs,48 Tannenbaum et al for benzodiazepines49 and Clyne et al for PPIs.47

NSAID, non-steroidal anti-inflammatory drugs; PPI, proton pump inhibitor; QALY, quality-adjusted life year; WTP, willingness-to-pay.

Discussion

For the three PIP Markov models considered, the costs were greater and there were fewer QALYs where the potentially inappropriate medicine was prescribed compared with an appropriate alternative strategy (table 2). For PPIs, the differences between the PIP and appropriate alternative did not reach statistical significance due to uncertainty in the risk of adverse events attributable to using maximal doses relative to maintenance doses (figure 1). Of the three PIP drugs considered in this study, benzodiazepines for greater than 4 weeks compared with no sedative medicine had the greatest cost and QALY impact per patient (table 2). In the evaluation of the cost-effectiveness of reducing PIP of these drugs, targeting long-term NSAIDs prescribing would be most cost-effective due to the published effectiveness of the intervention that was evaluated, though the ranking depended on the CE threshold used (table 4).

Context of the literature

No other studies appear to have assessed the economic impact of PIP defined by STOPP beyond direct costs of medicines.3 Several studies have quantified the costs of adverse events relating to drug classes included in this analysis, although in different settings.50 For NSAIDs, the costs associated with no gastroprotection among older patients with peptic ulcer disease in the UK, the excess costs of GI injury among older US Medicaid patients and the comparative costs of harm due to different NSAIDs have been evaluated.6 9 51 Benzodiazepine drug interactions, although not potentially inappropriate benzodiazepine prescribing, were associated with significantly increased healthcare costs in a regression analysis of older patients,7 while a further case-control study considered the attributable fall-related hospitalisation costs.52 They estimated the cost of fall-related hospitalisations attributable to benzodiazepines in the Netherlands as €48.5 million, which is 18.9% of the total cost of fall-related admissions. An economic modelling study comparing benzodiazepines to cognitive behavioural therapy or no treatment among older adults with insomnia considering a time horizon of only 1 year also found substantial falls-related costs associated with sedative drug use.8 While decision-tree analysis has been used to evaluate different PPI treatment strategies, including dose reduction, to manage oesophagitis,53 the economic impact of adverse events or inappropriate prescribing of PPIs has not been evaluated. Comparisons with the present study are difficult, as previous research has often presented results at the population level rather than the incremental cost per person over an extended time horizon.

Despite many studies of interventions to address appropriateness of prescribing in older people in primary care, few economic evaluations have been published.3 10 The PINCER intervention in English GP practices was cost-effective in both the in-trial economic evaluation and the model-based cost-utility analysis over a 5 year time horizon beyond the trial.9 54 However, there was uncertainty in the model-based results due to a lack of precise estimates of harm in the published literature for some of the prescribing/monitoring errors targeted.9 An older study of clinical pharmacist advice to older US veterans on five or more medicines and their doctors reported a cost of $7.50–$30 (€12–€48) per patient per unit improvement in the Medication Appropriateness Index.55 Other published economic evaluations have focused on appropriate prescribing of only specific drug classes, such as benzodiazepines,56 57 psychiatric medicines58 59 or cardiovascular medicines.60 Of all of these interventional studies, only the PINCER trial conducted a model-based economic evaluation presenting results as an ICER (ie, cost per QALY). Several recent trials of primary care interventions have successfully reduced PIP drugs. The OPTI-SCRIPT intervention involved academic detailing by a pharmacist and a computer decision support system for GPs in Ireland and resulted in a reduction in PIP, and in particular in long-term use of PPIs at maximal dosage.47 The Scottish DQIP intervention employing education, informatics and incentives to assist GPs reviewing older patients’ prescribing effectively decreased high-risk prescribing of NSAIDs and other medicines and reduced the rate of hospitalisation for GI bleeding and heart failure.48 Finally, the EMPOWER trial demonstrated that a patient empowerment intervention delivered through Canadian community pharmacies results in greater discontinuation of benzodiazepines than standard care.49 The cost-effectiveness of these interventions has yet to be demonstrated through published economic evaluations, and hence this study illustrates the use of Markov models to assess the cost-effectiveness of reducing PIP and the resulting adverse events.

Strengths and limitations

This is the first study to quantify the economic impact of PIP in older people, considering not just the medication cost but also the adverse consequences. The use of Markov models allowed for available evidence on harm relating to PIP criteria from the published literature to be combined. The analysis also incorporated uncertainty in these estimates and a number of model validation steps were conducted. This study directly compared three types of suboptimal prescribing with distinct adverse effects on a common scale of costs and QALYs. Similarly, it illustrates that the cost-effectiveness of potential interventions to improve prescribing in older people can be assessed using Markov modelling to capture the long-term consequences of medicines optimisation.

This study has several limitations. Only the principal adverse effects of each PIP were included to reduce the complexity and increase transparency of the models. Similarly, although prevalent among older adults, we did not consider drug-drug and drug-disease interactions or exacerbations of underlying conditions within the models. A number of model assumptions were applied to address this study’s aim. First, as the STOPP criteria refer to drug classes, we used pooled estimates for each class for the risk of adverse effects to provide the average economic impact of each PIP, and heterogeneity within drug classes was beyond the scope of this study. Similarly, we did not consider strategies that modify risks, such as gastroprotection with NSAIDs to prevent GI adverse events with NSAIDs. Second, the cohort under consideration were 65 year olds, assumed to be continuous users of each PIP, and in the intervention evaluation, the reduction in PIP was assumed to be sustained over the full time horizon. In reality, patients may spend some time exposed and unexposed; however, these assumptions allowed comparison of the overall effects of each PIP. We considered treatment adherence in sensitivity analysis and although adherence to these medication classes is likely to be high given their symptomatic effects, adherence may be lower in some cases than is considered here. The analyses were performed on a cohort basis to assess the average costs and effects, which does not reflect the variability of these outcomes among individuals, where some patients may incur large costs and have a greater reduction in QALYs. Heterogeneity was also not considered, as the research did not aim to evaluate how the economic impact may vary among patient subgroups. Further research should determine the extent to which differences in individual patient characteristics may alter the economic impact of PIP. This analysis focused only on adverse effects of prescribing deemed to be potentially inappropriate; however, appropriate alternatives were selected on the basis of similar effectiveness and limited adverse effects. Although these types of prescribing are generally regarded as inappropriate for older adults, there may be circumstances where patients and their doctors weigh the benefits and harms and decide that the ‘inappropriate’ prescription is optimal for them individually.

Implications for policy and practice

Trial-based economic evaluations may not always be informative for policy-maker decisions due to, for example, relevant comparators not being included, an insufficient time horizon or measurement of intermediary endpoints (eg, serum cholesterol) or process measures (eg, PIP) rather than final outcomes.44 Modelling approaches can overcome these weaknesses, by allowing all relevant evidence to be synthesised, incorporating alternative treatments not directly compared in a trial and extrapolating beyond the duration of the trial to assess long-term outcomes.12 Adoption of economic modelling approaches could increase the number of informative economic evaluations of prescribing safety interventions, such as in the PINCER trial.9 Such methods may be particularly useful in evaluating services to improve other aspects of medicines use where the benefits may not manifest during the period of a trial, for example, interventions to improve adherence to preventative medicines.61 Future trials of new or expanded services should conduct robust economic evaluations and include long-term consequences to inform policy-makers’ decisions on implementation and funding allocation. Cost-utility analyses presenting results as cost per QALY are most informative, allowing policy-makers to compare interventions and make funding decisions across therapeutic domains. Model-based approaches, as illustrated here, are an effective method to produce these estimates and evaluate interventions which affect outcomes across physiological systems.

Prescribing of potentially inappropriate medicines has significant economic implications, and interventions to reduce PIP are likely to be cost-effective if implemented into primary care for older people. The 95% CIs for cost and QALY differences in the PPI model both included zero, which, similar to the PINCER trial, was due to uncertainty relating to the adverse effects.9 This indicates that more information is needed on the safety of maximal compared with maintenance doses,62 and therefore these results should not deter efforts to deprescribe PPIs where their use is potentially inappropriate.2 47 As illustrated in table 4, the CE threshold being used by policy-makers (ie, the cost they are willing to pay for a QALY) can influence which interventions are funded. Placing a greater monetary value on each QALY will favour interventions that improve quality and quantity of life over those that reduce healthcare costs. While an explicit CE threshold exists for new drugs in the Irish health system, it is less clear whether the same applies to other interventions, such as those to improve prescribing.63 It may be that a lower CE threshold applies to these, for instance, if no additional funding is available for medicines optimisation services and only cost-saving interventions are acceptable to decision-makers. Using a different CE threshold may alter healthcare decisions and potentially result in less net benefit for patients across the health system.63

Conclusion

PIP of benzodiazepines and NSAIDs carry a statistically significant cost, to both the health system and patients, and there is an economic case for implementing effective interventions to improve prescribing of these medications for older people. Maximal dose PPI use is highly prevalent but evidence of harms is less certain, and so further studies should consider whether continuing maximal dose PPI is associated with increased risks compared with maintenance dose prescribing in order to establish whether targeting this is an efficient use of resources. Future research should also evaluate which patient subgroups inappropriate medication use have the greatest economic impact on, and thus, which patients would most benefit from prescribing optimisation interventions to maximise cost-effectiveness.

Supplementary Material

Reviewer comments
Author's manuscript

Footnotes

Twitter: @FrankMoriarty

Contributors: FM, CC, KB and TF contributed to the conception and design of this study. FM collected the data inputs used and carried out the statistical analysis. All authors interpreted the data. The manuscript was drafted by FM and all authors were involved in the critical revision and approval of the final manuscript. FM is the guarantor.

Funding: All authors have declared support from the Health Research Board (HRB) in Ireland through grant no. PHD/2007/16 (FM), grant no. HRC/2014/1 (TF) and grant no. RL/15/1579 (CC and KB) for this work.

Competing interests: None declared.

Patient consent: Not required.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: Markov models coded in Microsoft Excel are available at https://doi.org/10.6084/m9.figshare.5818251.v1 and data inputs are included in the technical appendix (online supplementary appendix table A1).

References

  • 1. Cahir C, Bennett K, Teljeur C, et al. Potentially inappropriate prescribing and adverse health outcomes in community dwelling older patients. Br J Clin Pharmacol 2014;77:201–10. 10.1111/bcp.12161 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Moriarty F, Hardy C, Bennett K, et al. Trends and interaction of polypharmacy and potentially inappropriate prescribing in primary care over 15 years in Ireland: a repeated cross-sectional study. BMJ Open 2015;5:e008656 10.1136/bmjopen-2015-008656 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Hill-Taylor B, Sketris I, Hayden J, et al. Application of the STOPP/START criteria: a systematic review of the prevalence of potentially inappropriate prescribing in older adults, and evidence of clinical, humanistic and economic impact. J Clin Pharm Ther 2013;38:360–72. 10.1111/jcpt.12059 [DOI] [PubMed] [Google Scholar]
  • 4. Chiatti C, Bustacchini S, Furneri G, et al. The economic burden of inappropriate drug prescribing, lack of adherence and compliance, adverse drug events in older people: a systematic review. Drug Saf 2012;35:73–87. 10.1007/BF03319105 [DOI] [PubMed] [Google Scholar]
  • 5. Pohl-Dernick K, Meier F, Maas R, et al. Potentially inappropriate medication in the elderly in Germany: an economic appraisal of the PRISCUS list. BMC Health Serv Res 2016;16:109 10.1186/s12913-016-1366-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Smalley WE, Griffin MR, Fought RL, et al. Excess costs from gastrointestinal disease associated with nonsteroidal anti-inflammatory drugs. J Gen Intern Med 1996;11:461–9. 10.1007/BF02599040 [DOI] [PubMed] [Google Scholar]
  • 7. Dionne PA, Vasiliadis HM, Latimer E, et al. Economic impact of inappropriate benzodiazepine prescribing and related drug interactions among elderly persons. Psychiatr Serv 2013;64:331–8. 10.1176/appi.ps.201200089 [DOI] [PubMed] [Google Scholar]
  • 8. Tannenbaum C, Diaby V, Singh D, et al. Sedative-hypnotic medicines and falls in community-dwelling older adults: a cost-effectiveness (decision-tree) analysis from a US Medicare perspective. Drugs Aging 2015;32:305–14. 10.1007/s40266-015-0251-3 [DOI] [PubMed] [Google Scholar]
  • 9. Elliott RA, Putman KD, Franklin M, et al. Cost effectiveness of a pharmacist-led information technology intervention for reducing rates of clinically important errors in medicines management in general practices (PINCER). Pharmacoeconomics 2014;32:573–90. 10.1007/s40273-014-0148-8 [DOI] [PubMed] [Google Scholar]
  • 10. Clyne B, Fitzgerald C, Quinlan A, et al. Interventions to address potentially inappropriate prescribing in community-dwelling older adults: a systematic review of randomized controlled trials. J Am Geriatr Soc 2016;64:1210–22. 10.1111/jgs.14133 [DOI] [PubMed] [Google Scholar]
  • 11. Husereau D, Drummond M, Petrou S, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ 2013:346 f1049. [DOI] [PubMed] [Google Scholar]
  • 12. Briggs A, Claxton K, Sculpher M. Decision modelling for health economic evaluation. Oxford, UK: Oxford University Press, 2006. [Google Scholar]
  • 13. Siebert U, Alagoz O, Bayoumi AM, et al. State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Med Decis Making 2012;32:690–700. 10.1177/0272989X12455463 [DOI] [PubMed] [Google Scholar]
  • 14. Health Information and Quality Authority. Guidelines for the economic evaluation of health technologies in Ireland. Dublin, Ireland: Health Information and Quality Authority, 2014. [Google Scholar]
  • 15. Straus WL, Ofman JJ, MacLean C, et al. Do NSAIDs cause dyspepsia? A meta-analysis evaluating alternative dyspepsia definitions. Am J Gastroenterol 2002;97:1951–8. 10.1111/j.1572-0241.2002.05905.x [DOI] [PubMed] [Google Scholar]
  • 16. Ofman JJ, Maclean CH, Straus WL, et al. Meta-analysis of dyspepsia and nonsteroidal antiinflammatory drugs. Arthritis Rheum 2003;49:508–18. 10.1002/art.11192 [DOI] [PubMed] [Google Scholar]
  • 17. Bhala N, Emberson J, Merhi A, et al. Vascular and upper gastrointestinal effects of non-steroidal anti-inflammatory drugs: meta-analyses of individual participant data from randomised trials. Lancet 2013;382:769–79. 10.1016/S0140-6736(13)60900-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Woolcott JC, Richardson KJ, Wiens MO, et al. Meta-analysis of the impact of 9 medication classes on falls in elderly persons. Arch Intern Med 2009;169:1952–60. 10.1001/archinternmed.2009.357 [DOI] [PubMed] [Google Scholar]
  • 19. Klotzbuecher CM, Ross PD, Landsman PB, et al. Patients with prior fractures have an increased risk of future fractures: a summary of the literature and statistical synthesis. J Bone Miner Res 2000;15:721–39. 10.1359/jbmr.2000.15.4.721 [DOI] [PubMed] [Google Scholar]
  • 20. Ngamruengphong S, Leontiadis GI, Radhi S, et al. Proton pump inhibitors and risk of fracture: a systematic review and meta-analysis of observational studies. Am J Gastroenterol 2011;106:1209–18. 10.1038/ajg.2011.113 [DOI] [PubMed] [Google Scholar]
  • 21. Howell MD, Novack V, Grgurich P, et al. Iatrogenic gastric acid suppression and the risk of nosocomial Clostridium difficile infection. Arch Intern Med 2010;170:784–90. 10.1001/archinternmed.2010.89 [DOI] [PubMed] [Google Scholar]
  • 22. Central Statistics Office. StatBank Data Dissemination Service. http://www.cso.ie/px/pxeirestat/Statire/SelectVarVal/Define.asp?maintable=PEA01&PLanguage=0 (accessed 20 Dec 2015).
  • 23. Blower AL, Brooks A, Fenn GC, et al. Emergency admissions for upper gastrointestinal disease and their relation to NSAID use. Aliment Pharmacol Ther 1997;11:283–91. 10.1046/j.1365-2036.1997.d01-604.x [DOI] [PubMed] [Google Scholar]
  • 24. Rockall TA, Logan RFA, Devlin HB, et al. Incidence of and mortality from acute upper gastrointestinal haemorrhage in the United Kingdom. BMJ 1995;311:222–6. 10.1136/bmj.311.6999.222 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Montalescot G, Dallongeville J, Van Belle E, et al. STEMI and NSTEMI: are they so different? 1 year outcomes in acute myocardial infarction as defined by the ESC/ACC definition (the OPERA registry). Eur Heart J 2007;28:1409–17. 10.1093/eurheartj/ehm031 [DOI] [PubMed] [Google Scholar]
  • 26. Smolina K, Wright FL, Rayner M, et al. Long-term survival and recurrence after acute myocardial infarction in England, 2004 to 2010. Circ Cardiovasc Qual Outcomes 2012;5:532–40. 10.1161/CIRCOUTCOMES.111.964700 [DOI] [PubMed] [Google Scholar]
  • 27. Haentjens P, Magaziner J, Colón-Emeric CS, et al. Meta-analysis: excess mortality after hip fracture among older women and men. Ann Intern Med 2010;152:380–90. 10.7326/0003-4819-152-6-201003160-00008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Dubberke ER, Butler AM, Reske KA, et al. Attributable outcomes of endemic Clostridium difficile-associated disease in nonsurgical patients. Emerg Infect Dis 2008;14:1031–8. 10.3201/eid1407.070867 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Kind P, Hardman G, Macran S. UK Population norms for EQ-5D. Centre for Health Economics, University of York, York, UK 1999. [Google Scholar]
  • 30. Ebell MH, Warbasse L, Brenner C. Evaluation of the dyspeptic patient: a cost-utility study. J Fam Pract 1997;44:545–55. 0. [PubMed] [Google Scholar]
  • 31. Groeneveld PW, Lieu TA, Fendrick AM, et al. Quality of life measurement clarifies the cost-effectiveness of Helicobacter pylori eradication in peptic ulcer disease and uninvestigated dyspepsia. Am J Gastroenterol 2001;96:338–47. 10.1016/S0002-9270(00)02309-1 [DOI] [PubMed] [Google Scholar]
  • 32. Clarke P, Gray A, Holman R. Estimating utility values for health states of type 2 diabetic patients using the EQ-5D (UKPDS 62). Med Decis Making 2002;22:340–9. 10.1177/027298902400448902 [DOI] [PubMed] [Google Scholar]
  • 33. Smith DW, Davies EW, Wissinger E, et al. A systematic literature review of cardiovascular event utilities. Expert Rev Pharmacoecon Outcomes Res 2013;13:767–90. 10.1586/14737167.2013.841545 [DOI] [PubMed] [Google Scholar]
  • 34. Lindgren P, Buxton M, Kahan T, et al. The lifetime cost effectiveness of amlodipine-based therapy plus atorvastatin compared with atenolol plus atorvastatin, amlodipine-based therapy alone and atenolol-based therapy alone: results from ASCOT1. Pharmacoeconomics 2009;27:221–30. 10.2165/00019053-200927030-00005 [DOI] [PubMed] [Google Scholar]
  • 35. Brennan VK, Colosia AD, Copley-Merriman C, et al. Systematic literature review of costs related to patients with type-2 diabetes mellitus experiencing a stroke or myocardial infarction. Value in Health 2013;16:A162 10.1016/j.jval.2013.03.811 [DOI] [Google Scholar]
  • 36. Borgström F, Zethraeus N, Johnell O, et al. Costs and quality of life associated with osteoporosis-related fractures in Sweden. Osteoporos Int 2006;17:637–50. 10.1007/s00198-005-0015-8 [DOI] [PubMed] [Google Scholar]
  • 37. Hiligsmann M, Ethgen O, Richy F, et al. Utility values associated with osteoporotic fracture: a systematic review of the literature. Calcif Tissue Int 2008;82:288–92. 10.1007/s00223-008-9117-6 [DOI] [PubMed] [Google Scholar]
  • 38. Neumann PJ, Hermann RC, Kuntz KM, et al. Cost-effectiveness of donepezil in the treatment of mild or moderate Alzheimer’s disease. Neurology 1999;52:1138–45. 10.1212/WNL.52.6.1138 [DOI] [PubMed] [Google Scholar]
  • 39. Honkanen LA, Mushlin AI, Lachs M, et al. Can hip protector use cost-effectively prevent fractures in community-dwelling geriatric populations? J Am Geriatr Soc 2006;54:1658–65. 10.1111/j.1532-5415.2006.00939.x [DOI] [PubMed] [Google Scholar]
  • 40. Church J, Goodall S, Norman R, et al. The cost-effectiveness of falls prevention interventions for older community-dwelling Australians. Aust N Z J Public Health 2012;36:241–8. 10.1111/j.1753-6405.2011.00811.x [DOI] [PubMed] [Google Scholar]
  • 41. Nathwani D, Cornely OA, Van Engen AK, et al. Cost-effectiveness analysis of fidaxomicin versus vancomycin in Clostridium difficile infection. J Antimicrob Chemother 2014;69:2901–12. 10.1093/jac/dku257 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Slobogean GP, O’Brien PJ, Brauer CA. Single-dose versus multiple-dose antibiotic prophylaxis for the surgical treatment of closed fractures. Acta Orthop 2010;81:256–62. 10.3109/17453671003587119 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Rodrigues MA, Brady RR, Rodrigues J, et al. Clostridium difficile infection in general surgery patients; identification of high-risk populations. Int J Surg 2010;8:368–72. 10.1016/j.ijsu.2010.05.004 [DOI] [PubMed] [Google Scholar]
  • 44. Gray AM, Clarke PE, Wolstenholme JL, et al. Applied Methods of Cost-Effectiveness Analysis in Health Care. Oxford, UK: Oxford University Press, 2011. [Google Scholar]
  • 45. Vemer P, Corro Ramos I, van Voorn GA, et al. AdViSHE: a validation-assessment tool of health-economic models for decision makers and model users. Pharmacoeconomics 2016;34:349–61. 10.1007/s40273-015-0327-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Fenwick E, Claxton K, Sculpher M. The value of implementation and the value of information: combined and uneven development. Med Decis Making 2008;28:21–32. 10.1177/0272989X07308751 [DOI] [PubMed] [Google Scholar]
  • 47. Clyne B, Smith SM, Hughes CM, et al. OPTI-SCRIPT study team. Effectiveness of a Multifaceted Intervention for Potentially Inappropriate Prescribing in Older Patients in Primary Care: A Cluster-Randomized Controlled Trial (OPTI-SCRIPT Study). Ann Fam Med 2015;13:545–53. 10.1370/afm.1838 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Dreischulte T, Donnan P, Grant A, et al. Safer prescribing--a trial of education, informatics, and financial incentives. N Engl J Med 2016;374:1053–64. 10.1056/NEJMsa1508955 [DOI] [PubMed] [Google Scholar]
  • 49. Tannenbaum C, Martin P, Tamblyn R, et al. Reduction of inappropriate benzodiazepine prescriptions among older adults through direct patient education: the EMPOWER cluster randomized trial. JAMA Intern Med 2014;174:890–8. 10.1001/jamainternmed.2014.949 [DOI] [PubMed] [Google Scholar]
  • 50. Anderson R. Systematic reviews of economic evaluations: utility or futility? Health Econ 2010;19:350–64. 10.1002/hec.1486 [DOI] [PubMed] [Google Scholar]
  • 51. Peris F, Martínez E, Badia X, et al. Iatrogenic cost factors incorporating mild and moderate adverse events in the economic comparison of aceclofenac and other NSAIDs. Pharmacoeconomics 2001;19:779–90. 10.2165/00019053-200119070-00006 [DOI] [PubMed] [Google Scholar]
  • 52. Panneman MJ, Goettsch WG, Kramarz P, et al. The costs of benzodiazepine-associated hospital-treated fall Injuries in the EU: a Pharmo study. Drugs Aging 2003;20:833–9. 10.2165/00002512-200320110-00004 [DOI] [PubMed] [Google Scholar]
  • 53. Goeree R, O’Brien B, Hunt R, et al. Economic evaluation of long-term management strategies for erosive oesophagitis. Pharmacoeconomics 1999;16:679–97. 10.2165/00019053-199916060-00007 [DOI] [PubMed] [Google Scholar]
  • 54. Avery AJ, Rodgers S, Cantrill JA, et al. A pharmacist-led information technology intervention for medication errors (PINCER): a multicentre, cluster randomised, controlled trial and cost-effectiveness analysis. Lancet 2012;379:1310–9. 10.1016/S0140-6736(11)61817-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55. Cowper PA, Weinberger M, Hanlon JT, et al. The cost-effectiveness of a clinical pharmacist intervention among elderly outpatients. Pharmacotherapy 1998;18:327–32. [PubMed] [Google Scholar]
  • 56. Godfrey C, Heather N, Bowie A, et al. Randomised controlled trial of two brief interventions against long-term benzodiazepine use: Cost-effectiveness. Addict Res Theory 2008;16:309–17. 10.1080/16066350801900297 [DOI] [Google Scholar]
  • 57. Morgan JD, Wright DJ, Chrystyn H. Pharmacoeconomic evaluation of a patient education letter aimed at reducing long-term prescribing of benzodiazepines. Pharm World Sci 2002;24:231–5. 10.1023/A:1021587209529 [DOI] [PubMed] [Google Scholar]
  • 58. Patterson SM, Cadogan CA, Kerse N, et al. Interventions to improve the appropriate use of polypharmacy for older people. Cochrane Database Syst Rev 2014:CD008165 10.1002/14651858.CD008165.pub3 [DOI] [PubMed] [Google Scholar]
  • 59. Eveleigh R, Grutters J, Muskens E, et al. Cost-utility analysis of a treatment advice to discontinue inappropriate long-term antidepressant use in primary care. Fam Pract 2014;31:578–84. 10.1093/fampra/cmu043 [DOI] [PubMed] [Google Scholar]
  • 60. Fretheim A, Aaserud M, Oxman AD. Rational prescribing in primary care (RaPP): economic evaluation of an intervention to improve professional practice. PLoS Med 2006;3:e216 10.1371/journal.pmed.0030216 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61. Elliott RA, Boyd MJ, Waring J, et al. Understanding and appraising the new medicines service in the NHS in England (029/0124). Nottingham, UK, 2014. [Google Scholar]
  • 62. Claxton K. The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies. J Health Econ 1999;18:341–64. 10.1016/S0167-6296(98)00039-3 [DOI] [PubMed] [Google Scholar]
  • 63. O’Mahony JF, Coughlan D. The Irish cost-effectiveness threshold: does it support rational rationing or might it lead to unintended harm to Ireland’s health system? Pharmacoeconomics 2016;34:5–11. 10.1007/s40273-015-0336-1 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2018-021832supp001.pdf (288KB, pdf)

Supplementary data

bmjopen-2018-021832supp002.pdf (1.9MB, pdf)

Supplementary data

bmjopen-2018-021832supp003.pdf (2.1MB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES