Challenges with cost-utility analyses of behavioral interventions among older adults at risk for dementia

Abstract

Background/Aim

Cognitive decline is one of the most prominent health care issues of the 21st century. Within the context of combating cognitive decline through behavioural interventions, physical activity is a promising approach. There is a dearth of health economic data in the area of behavioural interventions for dementia prevention. Yet, economic evaluations are essential for providing information to policy makers for resource allocation. It is essential we first address population and intervention specific methodological challenges prior to building a larger evidence base. We use a cost-utility analysis conducted alongside the Exercise for Cognition and Everyday Living (EXCEL) study to illustrate methodological challenges specific to assessing the cost-effectiveness of behavioural interventions aimed at older adults at risk of cognitive decline.

Methods

A cost-utility analysis conducted concurrently with a six-month three arm randomized controlled trial (i.e., the EXCEL study) was used as an example to identify and discuss methodological challenges.

Results

Both the aerobic training and resistance training interventions were less costly than twice weekly balance and tone classes. In critically evaluating the economic evaluation of the EXCEL study we identified four category specific challenges: 1) Analyzing costs, 2) Assessing QALYs 3) Incomplete data and 4) ‘Intervention’ activities of the control group.

Conclusions

Resistance training and aerobic training resulted in health care cost saving and were equally effective to balance and tone classes after only 6 months of intervention. To ensure this population is treated fairly in terms of claims on resources, we first need to identify areas for methodological improvement.

TRIAL REGISTRATION

NCT00958867

INTRODUCTION

In our current landscape of finite and limited resources, policy makers must make informed decisions. This paper considers the general area of economic evaluation of behavioral interventions targeting older adults at risk of dementia, and discusses some of the particular challenges associated with cost-utility analysis in this clinical field.

A large problem in quantity and cost: cognitive impairment and dementia

Cognitive decline is one of the most prominent health care issues of the 21st century. By 2040, over 80 million people will have dementia [1]. Dementia is associated with increased health care resource utilization.[2, 3] In 2010, the worldwide cost of dementia was $US 604 billion with 70% of these costs arising from North America and Western Europe.[3] Thus, the societal value of identifying cost-effective intervention strategies for dementia is considerable, especially given the scarcity and finite nature of health care resources.[4]

Treatment options exist – pharmacologic treatment

The causes of dementia are numerous; Alzheimer’s disease and vascular dementia are the two most common.[5] Using Alzheimer’s disease as an example to discuss the cost-effectiveness evidence currently available from pharmacologic treatment, we know that a number of economic evaluations have been conducted. Specifically, a recent systematic review of the cost-effectiveness of pharmacologic interventions for individuals with Alzheimer’s disease demonstrated wide divergence ranging from concluding cost-effectiveness to not being cost-effective for the same treatment.[6] For example, the incremental cost effectiveness ratio (ICER) for patients treated with donepezil versus no drug treatment ranged from a dominant value (less money and improved outcomes) to 281,416.13 euros per quality-adjusted life-year (QALY).[6] The sheer quantity of estimates for pharmacologic treatment provides is an essential starting platform of evidence, however, reasons for divergence in the ICERs warrant further exploration.

Behavioral interventions

Physical activity is a potentially inexpensive treatment that could have substantial preventative and restorative properties for cognitive function.[7] Evidence from randomized controlled trials demonstrates that targeted exercise training promotes cognitive function in older adults.[8–12] Specifically, three meta-analyses of RCTs of exercise concluded that targeted exercise training has cognitive benefits.[13–15] Further, recent RCTs provide preliminary evidence that the cognitive benefits of targeted exercise training extend to older adults with mild cognitive impairment – a largely understudied population but an ideal target population for prevention.[11, 12, 16] To our knowledge, there are only four studies to date that examined the efficacy of different types of targeted exercise training on cognitive function in older adults with Mild Cognitive Impairment (MCI).[12, 17–19] An essential counterpart to such clinical research is economic evaluation.

Economic evaluations – absent in RCTs of behavioral interventions promoting cognitive function – essential tool informing policy decisions

Economic evaluations guide essential health policy decisions. Critically relevant to the understated need of economic evaluations, is the fact that very few RCTs in the important field-behavioral interventions promoting cognitive function - have included economic analysis as part of the protocol. Further, given the plethora of economic data available for pharmacological interventions, there is a pressing need to generate economic data for potentially equally promising intervention strategies that may act as complements or substitutes to pharmacologic interventions – behavioural (e.g. exercise) interventions. Building the health economic data evidence base for behavioural interventions is an essential component to promote fair claims on relevant resources for this specific population. However, improvement in economic evaluation methodology relating specifically to older adults at risk of cognitive decline is needed before claims on resources can be addressed in an appropriate way.

Challenges in conducting a concurrent economic evaluation within an RCT in a population of older adults at risk of dementia

Trials with older adults with any degree of cognitive decline carry the inherent challenge of defining a valid outcome measurement and ensuring adherence with the intervention of choice (i.e., exercise).[20] The additional component – an economic evaluation – includes a key element essential for health care decision-making. We use the cost-utility analysis presented below as a case study to examine the unique methodological challenges related to conducting economic evaluations alongside clinical trials among older adults at risk of dementia.

We provide a brief description of the cost-utility analysis used as a case study for this paper. We designed a concurrent, prospective economic analysis using individual level data on cost and effectiveness outcomes as part of the EXCEL study; a three arm randomized controlled trial.[18] The main results have been reported elsewhere.[18] Our primary objective for the economic evaluation was to determine the incremental cost utility ratio (cost per QALY gained) of twice weekly resistance training or twice weekly aerobic training compared with twice weekly balance and tone classes (comparator). The comparator program is not specifically designed to combat cognitive decline.

First, we provide a brief overview of the EXCEL study.

METHODS

Ethical approval was obtained from the Vancouver Coastal Health Research Institute and the University of

British Columbia’s Clinical Research Ethics Board. All participants provided written informed consent.

Overview of economic evaluation

We used a Canadian health care system perspective in our cost effectiveness and cost utility analyses and a six-month time horizon for the economic evaluation.

The study sample included 86 community dwelling women aged 70 to 80 years. The interventions for the EXCEL study included three participant groups: twice weekly resistance training (RT), twice weekly aerobic training (AT) and the comparator (control) group, twice weekly balance and tone classes (BAT).

Costs

Briefly, we used a self-report questionnaire to track total healthcare resource utilization prospectively for each participant for 6 months.[21] Further detail on the collection of cost data were previously reported.[22]

Effectiveness

We calculated the total QALYs lost or gained at 6 months for each of the three participant groups using the EQ-5D administered at baseline and trial completion (6 months). We used linear regression to calculate the incremental QALYs for each participant adjusted for baseline utility score. All statistical analyses were carried out using STATA version 11.0.

Handling missing data

For all discrete time points, we used a combination of multiple imputation and bootstrapping to estimate uncertainty caused by missing values.[23, 24] [25, 26] We imputed missing EQ-5D and healthcare resource use.

Cost utility analysis

We also calculated the incremental cost per QALY for both twice-weekly resistance training and twice weekly aerobic training compared with twice weekly balance and tone classes (comparator). We used 5000 bootstrapped replications of mean difference in cost and mean QALY differences.[26]

RESULTS

Healthcare use and costs

Unit costs for healthcare cost items are provided in Appendix 1. The mean total healthcare costs were notably lower in the cost-saving region of the cost-effectiveness plane for the AT and RT groups compared with BAT (p<0.05) (see Table 1 and Figure 1).

Table 1.

Results of base case analysis

	Twice weekly balance and tone (n=28)	Once weekly resistance training (n=30)	Twice weekly resistance training (n=28)
Cost of delivering program (2010 CAD $¶) per person	353.06	352.15	353.06
Mean (SD) healthcare resource use cost (2010 CAD $)	1179 (934)	863 (448)	1146 (636)
Mean incremental costs for total healthcare resource use (2010 CAD $)	Reference	−316	−33
Total healthcare resource use costsÝ	reference||	dominates§ Figure 1	dominates§ Figure 1
Unadjusted QALY mean (SD) based on: EQ-5D	0.442 (0.059)	0.452 (0.053)	0.437 (0.099)
Unadjusted incremental QALY based on: EQ-5Dý	0 (reference)	0.01	−0.005
Adjusted QALY mean (SD) based on: EQ-5D	0.446 (0.049)	0.446 (0.49)	0.440 (0.078)
Adjusted incremental QALY based on: EQ-5Dý	0 (reference)	0.000	−0.006
Incremental cost per QALY based on: EQ-5D and Total healthcare resource use costs	Reference	Figure 1	Figure 1

^*p<0.05

^Ý ICER based on total HRU costs

^ý Incremental QALYs are adjusted for the baseline utility using a linear regression model

^§ For these strategies it was not appropriate to calculate an incremental cost effectiveness ratio because the intervention strategy were less costly and more effective than the balance and tone comparator (i.e., the intervention dominates the comparator in each of these cases)

^|| Reference indicates that the balance and tone group is the comparator

Figure 1.

Health outcomes

Mean QALYs and adjusted mean QALYs calculated from the EQ-5D are provided in Table 1.

Adjusting QALYs for baseline utility in each group

The incremental QALYs after 6 months calculated using the EQ-5D and adjusted for baseline EQ-5D levels, were 0.000 for the AT group and −0.006 for the RT group compared with BAT (Table 1).

Cost utility analysis

Based on the bootstrapped estimates from the cost-effectiveness plane (Figures 1 and 2), we found that RT was less costly and equally effective than BAT classes. Figures 1 and 2 demonstrates that for AT and RT compared with BAT, all of the bootstrapped cycles (nearly 100% of the 5000 cycles) were represented in the south quadrant (i.e., cost-saving region). This indicates that both AT and RT are less costly and equally effective to the comparator (BAT).

Figure 2.

DISCUSSION

Before conclusions can be made from this study, we first need to discuss the challenges that are specific to conducting economic evaluations alongside behavioral interventions among older adults at risk of dementia. Improvement in methodology is needed to facilitate the appropriate allocation of resources. To our knowledge no relevant methodological discussions relating to conducting concurrent economic evaluations alongside clinical trials exist for cognitive impairment and dementia. Below, we categorize these intervention population specific challenges into four themes: 1) Analyzing costs, 2) Assessing QALYs: is there a most appropriate method? 3) Incomplete data: followup time, changes in cognitive status, censoring and 4) ‘Intervention’ activities of the control group.

Analyzing costs

There are unique challenges faced when valuing medical expenditure related costs as well as costs related to loss of productivity among individuals with dementia. A recent study articulating the cost of dementia highlighted that the yearly monetary cost per person that was attributable to dementia ranged from $41,689 to $56,290.[27] The monetary value calculated was dependent on the method used to value informal care.[27] Such a wide variation in cost valuations can have substantial impacts on the results of a cost-utility analysis.

These data highlight the need to consider methodology for accurately articulating dementia related costs. For economic evaluations in the EXCEL study, rather than defining dementia related costs, we chose to report more broadly total health resource utilization costs. A potential benefit of choosing option one is that it eliminates the concern over accurately defining dementia related costs. Further, the use of cost diaries (i.e., self-reporting) to quantify health resource utilization is well supported in the literature as a valid method among a general population.[21] A few limitations of this approach are as follows: 1) wider variation in total health resource utilization costs, 2) larger sample size required due to wider variation, and 3) may fail to see the true impact of the intervention on incremental costs if the dementia specific incremental costs are smaller than the observed variation.

An alternative method would be to develop a consensus on what cost items are directly attributable to dementia. Given that individuals with dementia often have other comorbidities –most commonly stroke or depression[28] – defining dementia related costs is a significant challenge.[27] Another important set of cost items important to value in this population is unpaid time of family or friends spent helping an affected individual achieve their activities of daily living.[29] Of note, this perspective was not valued in the EXCEL study. However, equally important is the issue of how to accurately assign a monetary cost to an informal caregivers time. Specifically, the following items need to be considered: 1) unit cost of caregivers time, 2) loss of work productivity of caregiver and 3) time (travel time and time spent with the individual).

Methodological challenges relating to how to value resources remains a long standing issue.[30] The recommendations to date are broad; researchers should be specific in the reporting and analysis of their cost items and different valuations and inclusions of cost items should be explored in sensitivity analyses. [30] Specific differences in the incremental costs calculated in a systematic review of over 3000 cost-utility ratios indicated differences in costing methodology among the following: 1) disclosure of study perspective; 2) statement of time horizon; 3) use of discounting for future costs and quality-adjusted life years (QALYs); 4) statement of year of currency; 5) reporting of sensitivity analysis; and 6) use of incremental analysis.

In summary, moving forward with economic evaluations alongside clinical trials among individuals with dementia, future research needs to focus on what cost items should be included, accurate valuation of cost items and mode of data collection for such cost items.

Assessing QALYs: is there a most appropriate method?

When dealing with a clinical population with dementia or at high risk of cognitive decline, assessing QALYs becomes a particularly challenging task due to three methodological issues including response rate, recall bias and the validity of a proxy.[31–33] For example, the EQ-5D (also used in the EXCEL study) is a widely used instrument in cost-utility analyses across the globe because of its demonstrated feasibility and reliability. We chose the EQ-5D in the EXCEL study because it contained the appropriate domains we expected the intervention to impact and because of its feasibility. However, these are not adequate reasons to use an instrument and we need to consider potential challenges of such instruments.

Cognitive function is a spectrum and thus an important question to consider is: At what threshold of cognitive function is an individual considered impaired and unable to accurately value their health state utility values?

First, when considering establishing thresholds of cognitive function, it is crucial to consider individuals cognitive function trajectories over time. Evidence indicated that health related quality of life (HRQoL) is relatively stable among Canadians until age 70 at which point it begins to decline.[34] From recently published HRQoL trajectories for Canadians,[34] we speculate that a 6-month time horizon is not long enough to observe any significant incremental changes in QALYs between the AT or RT groups compared with BAT.[35] However, a counter argument may be that we would expect people at risk for dementia to decline at a greater rate; therefore, shorter time horizons may be advisable when not using a proxy.

Second, in the EXCEL study, our study population consisted of individuals with probable MCI. We did not make use of a proxy; however, this may be acceptable for the EXCEL study because the range and mean Montreal Cognitive Assessment (MOCA) scores across the three groups indicate the distributions were comparable across the three participant groups. As such, we did not anticipate that our mean incremental QALY estimates would be largely affected. However, we were unable to ascertain the validity of the mean QALYs. A key challenge this limitation highlights is what ‘gold standard’ should be used to ascertain the validity of participant self-reported health state utility values.

Response rate

Among individuals with at high risk of cognitive decline or with dementia, achieving a high response rate is a key challenge to consider. For example, one study that included 390 patients and their caregivers used the EQ-5D to assess patients’ health state utility values. Data demonstrated that the response rate of caregivers and of patients was higher for those with mild dementia compared with those of moderate dementia.[31]

Recall bias

Three commonly used instruments to assess HRQoL are the SF-36, EQ-5D and Health Utilities Index [36–39] each assume a different recall period ranging from an individual’s perceived health status today, in the past one week or in the past four weeks. Differences in these instruments and the implied recall period are an integral consideration among individuals with dementia or MCI because these individuals may be unable to accurately recall information for such periods. Hence, it is necessary to determine which recall periods are most appropriate to administer along the cognitive status spectrum.[40]

Use of a proxy

Participants in the EXCEL study had MCI. Hence, it is possible that they rated their health state better and thus we may observe a ceiling effect rendering it more difficult to estimate any incremental QALY differences.[32] In considering solutions for this issue, we explore the evidence surrounding the use of proxies. One study examined the inter-rater agreement of patient and proxy completion of the EuroQol EQ-5D instrument among individuals with dementia.[32] Both caregivers and physicians were compared with patient self-report. The results indicated that across all three raters, there was a high degree of variation that did not lead to support of any method.[32] However, the data provided support for interviewer administration of the EQ-5D.[32] Another study supported the use of the EQ-5D by patients and the inter rater reliability between patients and proxies.[31] Given that there are differences in assessment of HRQoL between people with dementia and proxies, a justified decision should be made at study outset about which perspective to take.

In summary, moving forward with economic evaluations alongside clinical trials among individuals with dementia or MCI, future research needs to explore cognitive status thresholds for when a proxy should be considered, what combination of proxies should be used to ensure validity and optimal response rate, and instrument selection based on target intervention domains and recall bias.

Incomplete data: followup time, changes in cognitive status, censoring

A general challenge of economic evaluations that are concurrent with clinical trials is that the followup time usually coincides with the duration of the trial. This followup time for the economic evaluation is often too short to assess full health gains or losses.

Further, censored observations are a key challenge when dealing with frail elderly populations at high risk of cognitive decline. The frequency of censored observations among individuals in this population is often high due to death or loss of competence to provide further data. This argument further supports the use of a shorter time horizon for individuals at high risk of dementia. To our knowledge, this challenge has not been discussed in the economic of economic evaluations among individuals with cognitive impairment.

In summary, moving forward with economic evaluations alongside clinical trials among individuals with dementia or MCI, future research needs to explore methods for determining feasible and appropriate followup times for economic evaluations that may extend beyond the clinical trial and ascertain appropriate methods for handling increased censoring observations.

‘Intervention’ activities of the control group

The activities prescribed for the control group is a particularly important methodological issue for behavioural interventions. In the EXCEL study the control (comparator) group was assigned to active treatment. An active treatment group was thought necessary to prevent contamination (i.e. control participants may feel “left out” and start doing other exercises on their own (i.e., their own intervention) or increased attrition. Hence there is a justified need for an active control group (i.e., the BAT group) for behavioural interventions. However in real life, these people may be “doing nothing”. This issue is particularly relevant to behavioural interventions in comparison with pharmaceutical interventions where participants can remain blinded due to a placebo. Not all RCTs are justified in providing some active control both in terms of ethics and usual practice. This issue does affect the primary outcomes of economic evaluations often making the ICER an overly conservative estimate. In the case of the EXCEL study, the BAT was designed to specifically include exercises not designed to combat cognitive decline. However, benefits of social group interaction may lead to an overly conservative estimate of health outcome benefit. As such, it may be advisable for economic evaluations to model a ‘do-nothing’ alternative.

Conclusions

This critically important population that cannot be ignored for resource allocation. To ensure this population is treated fairly in terms of claims on resources, we identified areas for methodological improvement. Most methodological challenges discussed relate specifically to population specific challenges; however a few examples of behavioural intervention specific challenges. The EXCEL study provided a platform to discuss population and intervention specific methodological challenges in conducting cost-utility analyses alongside clinical trials.

WHAT ARE THE NEW FINDINGS?

Aerobic training and resistance training interventions were less costly than twice weekly balance and tone classes.

This study sheds light on to population (i.e., individuals at high risk of dementia) and behavioural intervention specific methodological challenges. We discuss four category specific methodogical challenges of conducting behavioural interventions among older adults at risk of dementia: 1) Analyzing costs, 2) Assessing QALYs 3) Incomplete data and 4) ‘Intervention’ activities of the control group.

Identifying and discussing such methodological challenges is a key first step to ensure this population is treated fairly in terms of claims on finite resources.

Future research needs to explore cognitive status thresholds for when a proxy should be considered and what combination of proxies should be used to ensure validity and optimal response rate.

HOW MIGHT IT IMPACT ON CLINICAL PRACTICE?

Cognitive decline is one of the most prominent health care issues of the 21st century.

Treatment options for dementia exist – pharmacologic and behavioural interventions – yet, there is a dearth of health economic data in the area of behavioural interventions for dementia prevention.

The societal value of identifying cost-effective intervention strategies for dementia is considerable, especially given the scarcity and finite nature of health care resources.