Skip to main content
PMC home page
Journal of Pediatric Psychology logoLink to Journal of Pediatric Psychology
. 2018 May 15;43(7):758–768. doi: 10.1093/jpepsy/jsy022

Cost-Effectiveness Analysis of an Adherence-Promotion Intervention for Children With Leukemia: A Markov Model-Based Simulation

Meghan E McGrady 1,2,✉, Mark H Eckman 3, Maureen M O’Brien 4, Ahna L H Pai 1,2
PMCID: PMC6054144  PMID: 29771338

Abstract

Objective

Improving medication adherence among children with B-cell precursor acute lymphoblastic leukemia (B-ALL) has the potential to reduce relapse rates but requires an investment in resources. An economic evaluation is needed to understand the potential costs and benefits of delivering adherence-promotion interventions (APIs) as part of standard clinical care.

Methods

A Markov decision analytic model was used to simulate the potential incremental cost-effectiveness per quality-adjusted life year (QALY) to be gained from an API for children with B-ALL in first continuous remission compared with treatment as usual (TAU, no intervention). Model parameter estimates were informed by previously published studies. The primary outcome was incremental cost (2015 US$) per QALY gained for API compared with TAU.

Results

The model predicts the API to result in superior health outcomes (4.87 vs. 4.86 QALYs) and cost savings ($43,540.73 vs. $46,675.71) as compared with TAU, and simulations indicate that, across a range of plausible parameter estimates, there is a 95% chance that the API is more effective and less costly than TAU. The API was estimated to remain more effective and less costly than TAU in situations where the prevalence of nonadherence exceeds 32% and when API improves baseline adherence in at least 3% of patients.

Conclusions

Providing APIs to children with B-ALL may improve health outcomes and save costs over a 6-year period.

Keywords: adherence, behavioral intervention, oncology, pediatric

Each year, over 2,900 children (0–19 years of age) are diagnosed with acute lymphoblastic leukemia (ALL), making ALL the most common type of childhood cancer (Hunger & Mullighan, 2015; National Cancer Institute, 2013). While overall survival rates have increased from 10 to 90% since 1960, 5-year survival rates remain low (21–53%) for the 20% of children with ALL who relapse (Hunger & Mullighan, 2015; Nguyen et al., 2008). As a result, novel efforts to maintain remission may be critical to further improving outcomes in pediatric ALL (Nguyen et al., 2008). One promising avenue for reducing relapse rates is to target nonadherence to oral mercaptopurine during maintenance therapy, which accounts for an estimated 59% of pediatric ALL relapses but can be modified with behavioral intervention (Bhatia et al., 2012; Kahana, Drotar, & Frazier, 2008; Pai & McGrady, 2014). Meta-analyses of adherence-promotion randomized clinical trials (RCTs) for children with a medical condition (i.e., cancer) suggest that monitoring medication adherence and providing behavioral interventions increases adherence, improves health outcomes, and reduces health care utilization (McGrady et al., 2015; Pai & McGrady, 2014).

Given the potential of adherence-promotion efforts to improve health outcomes among children with ALL, multiple professional organizations (i.e., Society of Pediatric Psychology, Children’s Oncology Group [COG]) have endorsed the routine assessment and monitoring of medication adherence throughout cancer treatment as a standard of clinical care (Pai & McGrady, 2015). For many pediatric oncology clinics, implementing this standard of care will require an investment from the hospital and/or insurance company in additional resources to assess adherence (e.g., electronic monitors) and reimburse trained providers (e.g., psychologists, social workers) to deliver interventions (Wu et al., 2013). Observational studies of adults with cancer suggest that increasing medication adherence may reduce medical costs, but the relative magnitude of economic outcomes and their association with clinical benefits among children with ALL remain unknown (Darkow et al., 2007; Dieng, Cust, Kasparian, Mann, & Morton, 2016; McCowan, Wang, Thompson, Makubate, & Petrie, 2013).

One method of advocating for resources to support a new clinical initiative (such as providing an adherence-promotion intervention [API] to children with ALL) is to provide the relevant decision-maker (e.g., hospital, insurance company) with a cost-effectiveness analysis describing the costs and health outcomes of implementing the proposed program. Cost-effectiveness analyses may include data from a single clinical trial (hereafter referred to as “trial-based analyses”) or use decision analytic modeling to synthesize the best available data from multiple sources (Buxton et al., 1997). Trial-based analyses produce reliable estimates of costs and health outcomes with high internal validity (Buxton et al., 1997; Petrou & Gray, 2011). However, trial-based analyses often fail to meet many of the criteria necessary to ensure the resulting data are relevant to the decision-maker (Sculpher, Claxton, Drummond, & McCabe, 2006). Specifically, to be generalizable to the current situation, the clinical trial must have tested an intervention that is identical (i.e., in setting, format, content, efficacy) to the planned intervention; tested the intervention with a patient population identical to the population of interest; and collected data on final end points and for a time period that is sufficient to detect changes in health outcomes (Brennan & Akehurst, 2000; Sculpher et al., 2006). When psychologists are interested in delivering an intervention in a manner or setting that deviates even slightly from the trial protocol or to a population whose demographic and/or clinical characteristics do not match those enrolled in the trial, trial data cannot accurately estimate the costs and health outcomes of implementing such an intervention (Brennan & Akehurst, 2000; Sculpher et al, 2006). Because the full health benefits of interventions delivered by pediatric psychologists are often not realized until several years or decades after the intervention, the trial follow-up period is also often too short to capture the long-term health benefits of the intervention, and thus unable to accurately estimate the costs and health outcomes of the intervention (Sculpher et al., 2006).

To provide the data necessary to advocate for adherence-promotion efforts for children with ALL, an important next step is to conduct a cost-effectiveness analysis to simultaneously consider the costs and health outcomes of integrating adherence-promotion efforts into standard clinical care. While Kato et al. (2008) published a clinical trial of an API for adolescents and young adults with cancer, the trial population differs from the current population of interest (trial includes older patients with a range of cancer diagnoses vs. population of interest, which includes children with ALL) and did not include cost or long-term health outcome data. As a result, it is not currently possible to conduct a trial-based cost-effectiveness analysis. Thus, like the majority of cost-effectiveness analyses published in the broader health economic evaluation literature (Weinstein, 2006), this study used a decision analytic model. Specifically, a Markov model was used to estimate the potential incremental cost-effectiveness per quality-adjusted life year (QALY) to be gained from an API for children with B-cell precursor ALL (B-ALL) compared with treatment as usual (TAU, no intervention).

Because pediatric psychology is a relatively new field in which there remain several gaps regarding intervention efficacy (Palermo, 2014), there will likely be other instances in which pediatric psychologists are interested in evaluating the cost-effectiveness of an intervention in the absence of economic data resulting from an RCT. As a result, while the primary aim of this manuscript is to evaluate the potential costs and health outcomes of integrating an API into standard clinical care for children with B-ALL, it is also hoped that this manuscript can serve as an exemplar as to how pediatric psychologists can synthesize existing data to inform model-based analyses evaluating the cost-effectiveness of various behavioral interventions.

Methods

A decision analytic model was developed to compare the 6-year outcomes associated with two strategies of clinical care (API and TAU) for children with B-ALL receiving care at a U.S. children’s hospital. This model was developed using published data and is exempt from institutional review board approval.

The hypothetical cohort included children (0–19 years of age at diagnosis) with B-ALL in first continuous remission prescribed daily oral mercaptopurine as part of maintenance therapy (Hunger & Mullighan, 2015). As prognosis and treatment protocols differ by ALL lineage type (B-ALL vs. T-cell ALL) and B-ALL accounts for approximately 80–85% of pediatric ALL diagnoses, the proposed cohort was assumed to include only children with B-ALL (Nguyen et al., 2008). The age range was selected to mirror that used by the Surveillance, Epidemiology, and End Results registries (National Cancer Institute, 2013). Maintenance therapy was selected as nonadherence to oral mercaptopurine prescribed during this phase of treatment has been linked with relapse and earlier treatment blocks primarily consist of intravenous medications administered in a hospital setting (Bhatia et al., 2012). The cohort’s sociodemographic and clinical characteristics were assumed to be consistent with those of U.S. children with B-ALL (Hunger & Mullighan, 2015). Based on published clinical trials, it was estimated that 60% of children who relapsed would be classified as “high” risk and 40% would be classified as “low” risk (per relapse site, time to relapse, and minimal residual disease [MRD] presence at the end of reinduction; Nguyen et al., 2008; Oskarsson et al., 2016; Raetz & Bhatla, 2012).

Comparators

In a cost-effectiveness analysis, the two (or more) interventions or strategies being compared are termed “comparators.” In this model, the two comparators are the API and TAU.

The hypothetical first comparator, API, was assumed to be a 6-month intervention similar to those shown to be effective in published meta-analyses (Kahana, Drotar, & Frazier, 2008; Pai & McGrady, 2014) and is reflective of the clinical practices, resources, and structure of our children’s hospital. The API was assumed to reflect a hypothetical intervention rather than any single API, as the only published API in pediatric oncology includes a videogame delivered without intervention by a trained professional to adolescents and young adults with a range of cancer diagnoses (Kato et al., 2008). This intervention is unlikely to reflect the type of intervention for which psychologists would need to advocate for coverage and includes a population that is older and more heterogeneous in terms of diagnosis than our target population. Similar to published pediatric APIs (Kahana, Drotar, & Frazier, 2008; Pai & McGrady, 2014), the hypothetical API was assumed to include 6 monthly intervention sessions with the patient, caregiver(s), and psychologist. Sessions were assumed to include the most common behavior change techniques from published pediatric APIs (McGrady, Ryan, Brown, & Cushing, 2015) and target self-monitoring (via use of an electronic pill box to monitor adherence to oral mercaptopurine), knowledge, problem-solving skills, and environmental and social influences.

The second hypothetical comparator, TAU, represents the current state of adherence-promotion efforts in many pediatric oncology clinics in which adherence is not routinely assessed and APIs are provided only following referral.

Description of Simulation Model

Researchers evaluating the cost-effectiveness of alternative interventions using decision analytic modeling have many modeling approaches from which to choose. For a thorough overview of decision analytic modeling options and a guide for model selection, readers are encouraged to refer to Brennan & Akehurst (2000). A Markov state transition model was selected for this analysis as these models are ideal to capture clinical situations like ALL in which patients progress through multiple health states over time (e.g., induction → consolidation → maintenance), risk of a poor health outcome (e.g., relapse) is ongoing, and a given event (e.g., relapse) may occur more than once (Sonnenberg & Beck, 1993). With each 1-month “tick of the clock,” patients move from one health state to another based on the chance that given events occur (Siebert et al., 2012). Readers interested in an introduction to Markov modeling are referred to Sonnenberg & Beck (1993).

The Markov model for this manuscript was developed in consultation with experts in pediatric ALL, adherence, and decision-making (co-authors) to reflect the health states through which patients can transition in the 6 years following the beginning of maintenance therapy. The model reflects the standard arms of the current comprehensive treatment protocol for first relapse of pediatric B-ALL (COG AALL1331, NCT02101853; Children’s Oncology Group, 2017). A 6-year time horizon was selected to capture the period over which adherence to oral mercaptopurine has been linked to relapse status (Bhatia et al., 2015). Transitions could occur monthly as detailed below and in Figure 1.

Figure 1.

Figure 1.

Open in a new tab

Simplified Markov model. All patients start the Markov simulation in “Maintenance 1.” Arrows represent transitions that may occur with each 1-month “tick of the clock.” Rounded arrows represent health states in which patients can remain for >1 cycle (1 month). Values in parentheses represent the maximum number of months that can be spent in a given health state.

All patients began in maintenance 1 and were assumed to remain in that state for 6 months while they received API or TAU unless they died. Following API or TAU (at the end of Month 6), patients were classified as adherent (≥95% of doses of mercaptopurine) or nonadherent (<95% of doses) (Bhatia et al., 2015). Starting at month 7, patients could remain in maintenance 1 (for up to 24 months), die, or relapse. Patients completing the entire 24 months of maintenance 1 then transitioned to post-treatment, where they could remain or transition to relapse or death. Probability of relapse during maintenance or post-treatment was dependent on adherence status at month 6. Consistent with ongoing therapeutic trials, patients with relapsed B-ALL were assumed to receive re-induction chemotherapy (Blocks 1–3) followed by continuation chemotherapy or stem cell transplant (SCT) dependent on risk status (Parker et al., 2010; Raetz & Bhatla, 2012). Patients with low-risk relapse receiving chemotherapy without SCT could transition through 4 months of continuation, 17 months of maintenance, and to post-treatment.

At any point post-relapse, patients could also die. Patients with high-risk relapse could transition through SCT, an 11-month post-SCT period, and to post-treatment, or die at any point. The probability of death post-relapse was not related to adherence status at month 6 and thus equivalent across API and TAU.

Multiple simplifying assumptions were made in model development. First, given the variation in off-protocol therapy and low 5-year survival rates following a second relapse (9–15%), all patients experiencing second relapse were classified as dead (Ko et al., 2010; Reismüller et al., 2009). In addition, the model assumed that all high-risk patients would be deemed eligible for and undergo SCT. Additional assumptions are detailed in Supplementary Table S1.

Review of Data Used in Model

Once the structure of the Markov model has been developed, model parameter estimates (also termed “base case estimates,” see Base Case column of Table I) are added. This model required estimation of the following parameters: the costs associated with each health state, the health-related quality of life associated with each health state (termed “health utilities” and used to calculate QALYs), the probability of a patient being adherent or nonadherent post-API and TAU (intervention efficacy), and the probability of transitioning from each health state to all other health states (termed “transition probabilities”). Model parameter estimates were informed by manuscripts identified via PubMed searches (Supplementary Table S2). Data were extracted from articles meeting inclusion criteria (Supplementary Tables S3–S7). Articles with the highest level of evidence per the Oxford Centre for Evidence-Based Medicine Levels of Evidence were used for parameter estimates and are described below (Oxford Centre for Evidence-Based Medicine Levels of Evidence Working Group, 2016).

Table I.

Base Case Parameter Values and Clinically Plausible Ranges

Outcome variable Base case Range for sensitivity analyses
Minimum Base case
Monthly transition probabilities
 Adherent post-TAU 0.58 (Bhatia et al., 2015) 0.29a 0.70e
 Adherent post-API (API efficacy) 0.71 (Pai & McGrady, 2014) 0.59b,c 0.80b
 High-risk relapse 0.60d 0.50d 0.70d
 Relapse if non-adherent 0.00208 (Bhatia et al., 2015) 0.00203b 0.00213b
 Relapse if adherent 0.00067 (Bhatia et al., 2015) 0.00064b 0.00069b
 Dying during maintenance therapy 0.00006 (Bhatia et al., 2015) 0.00003a 0.00008a
 Dying during induction post-relapse 0.05273 (Parker et al., 2010) 0.02390c (Ko et al., 2010) 0.32000c (Raetz et al., 2008)
 Dying during chemotherapy post-relapse 0.00371 (Eckert et al., 2013) 0.00235c (Eckert et al., 2013) 0.00796c (Lew et al., 2014)
 Dying following SCT post-relapse 0.01987 (Oskarsson et al., 2016) 0.00464c (Eckert et al., 2013) 0.02568c (Oskarsson et al., 2016)
Health state utilities
 Maintenance 0.87 (Rae et al., 2014) 0.44a 0.90e
 Relapse
  Block 1 0.66 (Rae et al., 2014) 0.33a 0.90e
  Block 2 0.66 (Rae et al., 2014) 0.33a 0.90e
  Block 3 0.66 (Rae et al., 2014) 0.33a 0.90e
  Continuation (Weeks 1–4) 0.79 (Rae et al., 2014) 0.40a 0.90e
  Continuation (Weeks 5–8) 0.79 (Rae et al., 2014) 0.40a 0.90e
  SCT: Initial hospitalization (Mo. 1) 0.60 (Barr et al, 1996) 0.30a 0.90e
  SCT: Short-term follow-up (Mos. 2–3) 0.63 (Felder-Puig et al., 2006) 0.32a 0.90e
  SCT: Long-term follow-up (Mos. 4–12) 0.73 (Felder-Puig et al., 2006) 0.37a 0.90e
  Post-treatment (no relapse) 0.90 (Rae et al., 2014) 0.45a 1.00e
  Post-treatment (relapse) 0.90 (Rae et al., 2014) 0.45a 1.00e
Monthly costs
 Maintenance $803.37f $401.69a $1,205.06a
 Relapse
  Block 1 $49,706.16f $24,853.08a $74,559.24a
  Block 2 $21,871.34f $10,935.67a $32,807.01a
  Block 3 $26,345.30f $13,172.65a $39,517.95a
  Continuation (Weeks 1–4) $610.37f $305.19a $915.56a
  Continuation (Weeks 5–8) $3,264.47f $1,632.24a $4,896.71a
  SCT: Initial hospitalization (Mo. 1) $256,448.00 (Lin et al., 2010) $0.01b,e $664,244.50b
  SCT: Short-term follow-up (Mos. 2–3) $63,458.00 (Lin et al., 2010) $0.01b,e $212,489.90b
  SCT: Long-term follow-up (Mos. 4–12) $16,985.00 (Lin et al., 2010) $0.01b,e $56,324.36b
 Post-treatment (Year 1) $149.79f $74.90a $224.69a
 API $149.30g $74.65a $223.95a
Open in a new tab

Note. API = adherence-promotion intervention; MO = month; SCT = stem cell transplant; TAU = treatment as usual.

Estimate represents: a50% (for minimum) or 150% (for maximum) of base case, bupper or lower 95% CI of base case, cmost extreme (highest or lowest) published value, dexpert synthesis of previously published literature. eTotal 50 or 150% of base case estimate fell outside the minimum (e.g., $0 for costs) or maximum possible value (e.g., 1.00 for health utility), so minimum or maximum possible value was used. fEstimated from treatment protocol (Supplementary Table S8). gSee Supplementary Table S10.

Costs

The analysis was performed from the health care system perspective, and direct medical and induced costs were included (Cohen & Reynolds, 2008). Costs are expressed in 2015 U.S. dollars. Medicare payments for both facility and professional services were used as a proxy for costs (Table I). Direct nonmedical costs (e.g., transportation to and from clinic appointments/hospitalizations, lodging for family during hospitalizations, home health services such as health aides) were assumed to be equivalent across both interventions and were not included.

With the exception of SCT, medical therapy costs for each health state were calculated according to inpatient admissions, outpatient visits, laboratory tests, and medications included in current B-ALL treatment protocols (Supplementary Table S8; Raetz & Bhatla, 2012). Cost data sources included the Centers for Medicare and Medicaid Services (CMS) Physician Fee Schedule, CMS Clinical Laboratory Fee Schedule, and CMS Average Sales Price Drug Pricing Files. When CMS medication cost data were not available, pharmacy estimates were used. Costs associated with the management of adverse events (e.g., hospitalization for febrile neutropenia) were assumed equivalent across treatment arms and not included. Costs associated with SCT were obtained from a cost analysis, including 110 children with ALL who underwent SCT (Supplementary Table S9; Lin et al., 2010). Cost estimates were similar to those in a recent economic evaluation of MRD testing for pediatric ALL (Health Quality Ontario and the Toronto Health Economics and Technology Assessment Collaborative, 2016).

Personnel costs associated with API development and delivery were obtained from National Occupational Employment and Wage Estimates and the CMS Physician Fee Schedule (Supplementary Table S10). Electronic monitor device and adherence data transmission costs were obtained from the manufacturer. Costs were adjusted to 2015 US$ according to the medical care component of the Consumer Price Index, and future costs were discounted at an annual rate of 3% (U.S. Department of Labor, 2016).

Quality-Adjusted Life Years

Health utility assessments (Health Utilities Index, HUI) from a cohort of 317 children with ALL treated on the Dana-Farber Cancer Institute protocol (Supplementary Table S7; Rae et al., 2014) were used to estimate the QALYs associated with the time spent in the maintenance therapy and post-treatment health states. In the absence of studies describing health utilities post-relapse, HUI estimates for post-relapse reinduction, continuation, and post-treatment were also obtained from the Dana-Farber Cancer Institute cohort (Rae et al., 2014; van Litsenburg et al., 2014). HUI estimates for post-relapse SCT were extracted from studies of adults with ALL and children with a range of oncological diagnoses undergoing SCT (Barr et al., 1996; Felder-Puig et al., 2006). QALYs were discounted at an annual rate of 3% (Weinstein, Siegel, Gold, Kamlet, & Russell, 1996).

Prevalence of Nonadherence

Because adherence is a continuous construct ranging from no missed doses to no doses taken, classifying patients as adherent or nonadherent requires a clinically significant cut point. Thus, to inform the parameter estimate, a search was conducted to identify original research articles that reported on the prevalence of nonadherence (as a dichotomous variable) in children with ALL (Supplementary Table S3). Using the cut point of 95% of prescribed doses taken as assessed via electronic monitor, Bhatia et al. (2015) found that 42% of children were nonadherent to oral mercaptopurine during maintenance therapy. These children were at a higher risk of relapse than adherent patients (Bhatia et al., 2015). Thus, 42% was used as the prevalence of nonadherence or the probability of nonadherence post-TAU.

API Efficacy

In the absence of an RCT of an API for children ages 0–19 years with B-ALL, the effect of APIs on electronically monitored medication adherence was obtained from a meta-analysis of pediatric APIs (d = 0.31, Supplementary Table S4; Pai & McGrady, 2014). A 2 × 2 contingency table was used to compute the probability of API success assuming a treatment effect of d = 0.31, resulting in a parameter estimate of 71% adherent post-API.

Transition Probabilities

Transition probabilities were estimated for each possible transition from a given health state to another health state (each arrow included in Figure 1). Beginning at the top of the model, the probability of relapse (arrows from “Maintenance 1” to “Relapse: Block 1 Chemotherapy” and “Post-Treatment: No Relapse”) was obtained from a COG study of 742 children with ALL (Supplementary Table S5; Bhatia et al., 2012, 2015). The cumulative risk of relapse was 2.7 times higher among patients classified as nonadherent (M [SD] risk = 13.9% [2.6%]) than those classified as adherent (4.7% [1.3%]; Bhatia et al., 2015).

The probability of surviving induction post-relapse (arrows from “Relapse: Block 1 Chemotherapy” through subsequent chemotherapy blocks to “Continuation Chemotherapy 1,” “SCT,” and “Death”) was obtained from a study of 103 children with first relapse ALL enrolled on ALLR3 (mitoxantrone arm, Supplementary Table S6; Parker et al., 2010). Patients surviving reinduction were assumed to receive continuation chemotherapy (40%) or undergo SCT (60%) based on risk classification (arrows from “Block 3 Chemotherapy” to “Continuation Chemotherapy 1” and “SCT”; Nguyen et al., 2008; Oskarsson et al., 2016; Parker et al., 2010).

The probability of survival for low-risk patients receiving continuation chemotherapy (70%) was obtained from POG 9412 for late isolated extramedullary relapse, and COG AALL0433 and ALL-REZ BFM 2002 in which patients with intermediate-risk marrow relapse ALL and an MRD level <10−3 at the end of induction therapy received chemotherapy (arrows from “Continuation Chemotherapy 1” through other treatment blocks to “Post-Treatment: Relapse and Chemotherapy” and “Death”; Barredo et al., 2006; Eckert et al., 2013; Lew et al., 2014, Supplementary Table S6). The probability of 5-year event-free survival following SCT was estimated to be 30% based on a study of children with high-risk relapse ALL treated according to NOPHO ALL-92 and ALL-2000 protocols and a retrospective study of children treated on COG frontline trials who experienced first relapse (arrows from “SCT” through other treatment blocks to “Post-Treatment: Relapse and SCT” and “Death”; Nguyen et al., 2008; Oskarsson et al., 2016, Supplementary Table S6). As the impact of nonadherence on outcomes post-relapse remains unknown, transition probabilities post-relapse were assumed to be equivalent for adherent and nonadherent patients.

Analyses

Incremental Cost-Effectiveness Ratio

The primary outcome of a cost-effectiveness analysis (and this study) is the incremental cost-effectiveness ratio (ICER), defined as the differences in costs divided by the differences in health outcomes of the two comparators. The ICER for this study was defined as: ICER = (cost of API −  cost of TAU)(QALYs, API − QALYs, TAU). In the primary analysis, costs and QALYs over a 6-year time horizon were estimated for the API and TAU groups and used to compute the ICER. Costs (numerator) were measured in 2015 US$ as detailed above. Health outcomes (denominator) were assessed using QALYs calculated from health utility values. While the denominator of a cost-effectiveness analysis may include any health outcome, QALYs offer the benefit of capturing both the quality of life associated with a given health state and the time spent in that state (see Weinstein, Torrance, and McGuire, 2009 for an overview of QALYs).

Sensitivity Analyses

Because base case estimates are obtained from multiple sources, uncertainty is inherent in model-based analyses. Deterministic and probabilistic sensitivity analyses were used to examine the impact of uncertainty in parameter estimates on model results. The range of values tested in the sensitivity analysis for each parameter was obtained from the published 95% confidence interval (CI) and when a published CI was not available, from either the extreme values (lowest and highest for each parameter) identified via literature review or upper and lower limits as calculated as 50 and 150% of base case values (see Table I, Range for Sensitivity Analyses).

Deterministic sensitivity analyses were performed for each variable separately by systematically varying each parameter across the clinically plausible range detailed in Table I. For example, as it is possible that d = 0.31 (which translates to a probability of adherent post-API of 0.71) is not an accurate estimate of API efficacy, the model was rerun across the 95% CI of d = 0.02–0.59 (Pai & McGrady, 2014; which translates to a probability of adherent post-API of 0.59–0.80). The results of this deterministic sensitivity analysis indicate the degree to which model results would be expected to change as a result of variations in API efficacy. In sum, deterministic sensitivity analyses were used to explore the robustness of model results to potential variations in any parameter estimate.

Similar to the manner in which p values capture uncertainty within clinical studies, probabilistic sensitivity analyses estimate the likelihood that a given intervention will be “cost-effective” in decision analytic modeling (Cohen & Reynolds, 2008). Results of a probabilistic sensitivity analysis provide a model-level understanding of the impact of uncertainty in parameter estimates by “vibrating” the values of all model parameters simultaneously. We conducted a probabilistic sensitivity analysis by performing 10,000 second-order Monte Carlo simulations to sample parameter values from their distributions and estimate outcomes. The results of the probabilistic sensitivity analysis were used to calculate the percentage of simulations in which the API was predicted to be “cost-effective” relative to TAU (Briggs & Fenn, 1998). Additional details regarding the probabilistic sensitivity analysis (i.e., parameter distributions) are available from the authors on request.

Analyses were conducted using Windows Decision Maker® (Boston, MA). As recommended by the World Health Organization, the annual U.S. per capita gross domestic product was used as the willingness-to-pay threshold (Marseille, Larson, Kazi, Kahn, & Rosen, 2015). Using this criterion, ICER values <$55,200 per QALY are considered “cost-effective” (The World Bank Group, 2016). Results are reported in accordance with Consolidated Health Economic Evaluation Reporting Standards (Husereau et al., 2013).

Results

Base Case

The Markov model predicted that 6 years after beginning API or TAU, 95% of adherent patients were in post-treatment (no relapse), 1% were in post-treatment following chemotherapy for relapse, 1% were in posttreatment following SCT for relapse, and 2% were deceased (<1% in another health state). Of nonadherent patients, 86% were in post-treatment (no relapse), 3% were in post-treatment following chemotherapy for relapse, 3% were in post-treatment following SCT for relapse, and 6% were deceased (≤1% in another health state, Supplementary Figure S1). As the only difference between the API and TAU groups was the probability of being adherent, the different health state distributions across adherent and nonadherent patients led to superior health outcomes in the API group (4.87 QALYs vs. 4.86 QALYs for TAU). The lower rates of relapse in API also resulted in lower average costs compared with TAU ($43,540.73 vs. $46,675.71, Table II). As the API strategy was both superior in health outcomes and less costly, it dominated the TAU strategy (ICER is not calculated, as it would result in a negative value).

Table II.

Base Case Cost-Effectiveness Analysis

Strategy Cost Effectiveness ICER
API $43,540.73 4.87 –
TAU $46,675.71 4.86 Dominated
Open in a new tab

Note. API = adherence-promotion intervention; ICER = incremental cost-effectiveness ratio; TAU = treatment as usual.

Sensitivity Analyses

Results of deterministic sensitivity analyses indicate that the model was not sensitive to variations in outcomes post-relapse (i.e., probability of dying during reinduction, chemotherapy, or SCT), health state utilities, or costs. Across the range of clinically plausible values for each parameter, API remained more effective and less costly than TAU. The model was sensitive to variations in nonadherence prevalence (p adherence post-TAU) and API efficacy (p adherence post-API). The ICER was <0 when the probability of adherence post-TAU was ≤.68, suggesting that the API is more effective and less costly than TAU when the prevalence of nonadherence is at least 32% (1–.68, see Supplementary Figure S2). The probability of adherence post-TAU was .69 at the willingness-to-pay threshold of $55,200, suggesting that the API could be considered “cost-effective” if the prevalence of nonadherence is at least 31% (1–.69). The API was also estimated to be more effective and less costly than TAU when at least 61% of children were adherent post-API (ICER < $0) and “cost-effective” (ICER < $55,200) when at least 60% of children were adherent post-API versus 58% in TAU (see Supplementary Figure S2).

The results of the 10,000 second-order Monte Carlo simulations (probabilistic sensitivity analysis) are depicted in Figure 2. Each diamond in the figure represents the result of a single Monte Carlo simulation. As demonstrated in the scatter plot of differences in health outcomes and costs between API and TAU, API was more effective and less costly than TAU in the 95% of simulations (lower right quadrant). Thus, the API has a 95% probability of being more effective and less costly as compared with TAU. In addition, as 97% of simulations were below the ICER of $55,200, the API has a 97% probability of being either cost saving or “cost-effective” as compared with TAU.

Figure 2.

Figure 2.

Open in a new tab

Cost-effectiveness plane depicting the differences in costs and QALYs between the API and TAU resulting from 10,000 simulations. The black square represents the base case scenario.

Conclusions

To our knowledge, this study is the first to evaluate the cost-effectiveness of an API in any pediatric population and exemplifies the integral role of pediatric psychologists in national efforts to improve health outcomes and reduce costs. Model results predict that investing $150 a month per child with B-ALL to routinely assess and monitor medication adherence, as recommended by the Psychosocial Standards of Care for Children with Cancer and Their Families, has a 95% probability of improving outcomes and saving costs over a 6-year period (Pai & McGrady, 2015). Even though these results were based on a decision analytic model, our results were robust to sensitivity analyses of 29 variables including costs, health state utilities, and outcome probabilities, with API remaining more effective and less costly than TAU across each analysis.

In addition, the API was more effective and less costly when the prevalence of nonadherence among children with B-ALL was as low as 32%, well below documented nonadherence rates of 42–44% (Bhatia et al., 2012, 2015). This suggests that the API may improve outcomes and save costs if delivered to all children as part of standard clinical care versus only those with identified nonadherence. Even if API efficacy is small (61% adherent post-API), compared with usual care (58% adherent), the API remains more effective and less costly than TAU. For example, if the API was delivered to 200 children with B-ALL, only six patients would have to transition from nonadherent to adherent for the API result in improved health outcomes and cost savings as compared with TAU.

In contrast to health care payers in other countries (i.e., England, Australia, Canada), health care payers in the United States do not routinely consider cost-effectiveness analyses when making funding decisions (Eddama & Coast, 2008). The specific data needed to convince a health care payer to cover a service (such as an API) are likely to vary across payers and cost-effectiveness analyses such as this one often represent just one component of the required evidence base. As a result, pediatric psychologists advocating for coverage for their services are encouraged to partner with relevant health care payers to determine the data of interest to the payer. Even when health care payers request data other than those from a cost-effectiveness analysis, it is still imperative that pediatric psychologist continue to produce data resulting from economic models that abide by best practice guidelines (Husereau et al., 2013; Siebert et al., 2012). The methodological and scientific rigor inherent in these guidelines will ensure that our field is producing high-quality data in support of our services in addition to the payer-specific data requested to influence a given coverage decision. Such efforts are particularly timely given the uncertainty regarding the future structure of health care in the United States.

Advancing this clinically significant line of research requires studies addressing limitations of our model. First, in the absence of RCTs of APIs for children ages 0–19 years with B-ALL in maintenance therapy, the API effect size was estimated from a meta-analysis of pediatric APIs (Pai & McGrady, 2014). Sensitivity analyses indicate that the proposed API can demonstrate an effect size (d = 0.04) smaller than the lower 95% CI estimated by multiple meta-analyses of pediatric APIs ([d = 0.23, 95% CI: 0.17–0.19], [d = 0.20; 95% CI: 0.08–0.31]; Kahana et al., 2008; Pai & McGrady, 2014) to result in improved health outcomes and cost savings. However, the duration, complexity, demands, and associated side effects of various treatment regimens differ across chronic illness populations, and the majority of the studies included in the meta-analysis included populations other than cancer (asthma, diabetes). As a result, RCTs of APIs designed specifically for children with B-ALL in maintenance therapy are needed to obtain effect size data to improve model accuracy. These data would also address our limitation of including parameter estimates from samples with different demographic and clinical characteristics. In addition, these data would enable researchers to estimate the cost-effectiveness of the API as compared with TAU for different age groups, a question of particular relevance given the higher prevalence of nonadherence and poorer health outcomes among adolescents and young adults as compared with younger children (Barr, Ferrari, Ries, Whelan, & Bleyer, 2016). Second, to maximize replicability and transparency, only costs for treatment and minimal required laboratory observations on the current COG trial for first relapse B-ALL (COG AALL1331, NCT02101853) were included. Costs for additional routine labs, supportive care measures, red cell or platelet transfusions, anesthesia for invasive procedures, unexpected medical care, or treatments required by only some patients were not included. Third, patients experiencing a second relapse were classified as “dead,” while in reality such patients may receive multiple subsequent treatment attempts (i.e., novel immune and cellular therapies such as chimeric antigen receptor T cells) likely to dramatically increase costs (Maude et al., 2014). Future models including cost data collected in API RCTs, thus, may predict even greater cost savings resulting from the relapses prevented by APIs.

Investing in APIs during maintenance therapy for children with B-ALL may result in improved health outcomes and significant cost savings for health care payers. Further model refinement is needed to inform decisions by clinicians, administrators, and policy makers interested in optimizing health outcomes and maximizing cost containment.

Supplementary Data

Supplementary data can be found at: http://www.jpepsy.oxfordjournals.org/.

Funding

M.E.M. is supported by the National Cancer Institute of the National Institutes of Health (NIH) under grant number K07CA200668. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Conflicts of interest: None declared.

Supplementary Material

Supplementary Data
Click here for additional data file. (571.5KB, doc)

References

  1. Barr R. D., Ferrari A., Ries L., Whelan J., Bleyer W. A. (2016). Cancer in adolescents and young adults: A narrative review of the current status and a view of the future. JAMA Pediatrics, 170, 495–501. [DOI] [PubMed] [Google Scholar]
  2. Barr R., Furlong W., Henwood J., Feeny D., Wegener J., Walker I., Brain M. (1996). Economic evaluation of allogeneic bone marrow transplantation: A rudimentary model to generate estimates for the timely formulation of clinical policy. Journal of Clinical Oncology, 14, 1413–1420. [DOI] [PubMed] [Google Scholar]
  3. Barredo J. C., Devidas M., Lauer S. J., Billett A., Marymont M, Pullen J., Camitta B., Winick N., Carroll W., Ritchey A. K. (2006). Isolated CNS relapse of acute lymphoblastic leukemia treated with intensive systemic chemotherapy and delayed CNS radiation: A pediatric oncology group study. Journal of Clinical Oncology, 24, 3142–3149. [DOI] [PubMed] [Google Scholar]
  4. Bhatia S., Landier W., Hageman L., Chen Y., Kim H., Sun C. L., Kornegay N., Evans W. E., Angiolillo A. L., Bostrom B., Casillas J., Lew G., Maloney K. W., Mascarenhas L., Ritchey A. K., Termuhlen A. M., Carroll W. L., Wong F. L., Relling M. V. (2015). Systemic exposure to thiopurines and risk of relapse in children with acute lymphoblastic leukemia: A children’s oncology group study. JAMA Oncology, 1, 287–295. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bhatia S., Landier W., Shangguan M., Hageman L., Schaible A. N., Carter A. R., Hanby C. L., Leisenring W., Yasui Y., Kornegay N. M., Mascarenhas L., Ritchey A. K., Casillas J. N., Dickens D. S., Meza J., Carroll W. L., Relling M. V., Wong F. L. (2012). Nonadherence to oral mercaptopurine and risk of relapse in hispanic and non-hispanic white children with acute lymphoblastic leukemia: A report from the Children's Oncology Group. Journal of Clinical Oncology, 30, 2094–2101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brennan A., Akehurst R. (2000). Modelling in health economic evaluation. What is its place? What is its value? Pharmacoeconoimcs, 17, 445–459. [DOI] [PubMed] [Google Scholar]
  7. Briggs A., Fenn P. (1998). Confidence intervals or surfaces? Uncertainty on the cost-effectiveness plane. Health Economics, 7, 723–740. [DOI] [PubMed] [Google Scholar]
  8. Buxton M. J., Drummond M. F., Van Hout B. A., Prince R. L., Sheldon T. A., Szucs T., Vray M. (1997). Modeling in economic evaluation: An unavoidable fact of life. Health Economics, 6, 217–227. [DOI] [PubMed] [Google Scholar]
  9. Children’s Oncology Group. (2017). AALL1331: Risk-stratified randomized phase III testing of Blinatumomab in first relapse of childhood b-lymphoblastic leukemia (B-ALL) Retrieved from https://www.childrensoncologygroup.org/aall1331.
  10. Cohen D. J., Reynolds M. R. (2008). Interpreting the results of cost-effectiveness studies. Journal of the American College of Cardiology, 52, 2119–2126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Darkow T., Henk H. J., Thomas S. K., Feng W., Baladi J. F., Goldberg G. A., Hatfield A., Cortes J. (2007). Treatment interruptions and non-adherence with imatinib and associated healthcare costs: A retrospective analysis among managed care patients with chronic myelogenous leukaemia. Pharmacoeconomics, 25, 481–496. [DOI] [PubMed] [Google Scholar]
  12. Dieng M., Cust A. E., Kasparian N. A., Mann G. J., Morton R. L. (2016). Economic evaluations of psychosocial interventions in cancer: A systematic review. Psychooncology, 25, 1380–1392. [DOI] [PubMed] [Google Scholar]
  13. Eckert C., Henze G., Seeger K., Hagedorn N., Mann G., Panzer-Grümayer R., Peters C., Klingebiel T., Borkhardt A., Schrappe M., Schrauder A., Escherich G., Sramkova L., Niggli F., Hitzler J., von Stackelberg A. (2013). Use of allogeneic hematopoietic stem-cell transplantation based on minimal residual disease response improves outcomes for children with relapsed acute lymphoblastic leukemia in the intermediate-risk group. Journal of Clinical Oncology, 31, 2736–2742. [DOI] [PubMed] [Google Scholar]
  14. Eddama O., Coast J. (2008). A systematic review of the use of economic evaluation in local decision-making. Health Policy, 86, 129–141. [DOI] [PubMed] [Google Scholar]
  15. Felder-Puig R., di Gallo A., Waldenmair M., Norden P., Winter A., Gadner H., Topf R. (2006). Health-related quality of life of pediatric patients receiving allogeneic stem cell or bone marrow transplantation: Results of a longitudinal, multi-center study. Bone Marrow Transplant, 38, 119–126. [DOI] [PubMed] [Google Scholar]
  16. Health Quality Ontario and the Toronto Health Economics and Technology Assessment Collaborative. (2016). Minimal residual disease evaluation in childhood acute lymphoblastic leukemia: An economic analysis. Ontario Health Technology Assessment Series, 16, 1–83. [PMC free article] [PubMed] [Google Scholar]
  17. Hunger S. P., Mullighan C. G. (2015). Acute lymphoblastic leukemia in children. New England Journal of Medicine, 373, 1541–1552. [DOI] [PubMed] [Google Scholar]
  18. Husereau D., Drummond M., Petrou S., Carswell C., Moher D., Greenberg D., Augustovski F., Briggs A. H., Mauskopf J., Loder E.; ISPOR Health Economic Evaluation Publication Guidelines-CHEERS Good Reporting Practices Task Force. (2013). Consolidated Health Economic Evaluation Reporting Standards (CHEERS)—Explanation and elaboration: A report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value in Health, 16, 231–250. [DOI] [PubMed] [Google Scholar]
  19. Kahana S., Drotar D., Frazier T. (2008). Meta-analysis of psychological interventions to promote adherence to treatment in pediatric chronic health conditions. Journal of Pediatric Psychology, 33, 590–611. [DOI] [PubMed] [Google Scholar]
  20. Kato P. M., Cole S. W., Bradlyn A. S., Pollock B. H. (2008). A video game improves behavioral outcomes in adolescents and young adults with cancer: A randomized trial. Pediatrics, 122, e305–e317. [DOI] [PubMed] [Google Scholar]
  21. Ko R. H., Ji L., Barnette P., Bostrom B., Hutchinson R., Raetz E., Seibel N. L., Twist C. J., Eckroth E., Sposto R., Gaynon P. S., Loh M. L. (2010). Outcome of patients treated for relapsed or refractory acute lymphoblastic leukemia: A therapeutic advances in childhood leukemia consortium study. Journal of Clinical Oncology, 28, 648–654. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lew G., Lu X., Yanofsky R., Rheingold S. R., Whitlock J., An Q., Devidas M., Hastings C. A., Winick N. J., Carroll W. L., Borowitz M. J., Hunger S., Pulsipher M. A. (2014). Outcomes after intermediate-risk relapse of childhood B-Lymphoblastic Leukemia (B-ALL) and the role of allogeneic Stem Cell Transplantation (SCT): A report from Children's Oncology Group (COG) AALL0433 Paper presented at the ASH Annual Meeting and Exposition, San Francisco, CA. https://ash.confex.com/ash/2014/webprogram/Paper72312.html
  23. Lin Y. F., Lairson D. R., Chan W., Du X. L., Leung K. S., Kennedy-Nasser A. A., Martinez C. A., Gottschalk S. M., Bollard C. M., Heslop H. E., Brenner M. K., Krance R. A. (2010). The costs and cost-effectiveness of allogeneic peripheral blood stem cell transplantation versus bone marrow transplantation in pediatric patients with acute leukemia. Biology of Blood and Marrow Transplantation, 16, 1272–1281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Marseille E., Larson B., Kazi D. S., Kahn J. G., Rosen S. (2015). Thresholds for cost-effectiveness of interventions: Alternative approaches. Bulletin of the World Health Organization, 93, 118–124. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Maude S. L., Frey N., Shaw P. A., Aplenc R., Barrett D. M., Bunin N. J., Chew A., Gonzalez V. E., Zheng Z., Lacey S. F., Mahnke Y. D., Melenhorst J. J., Rheingold S. R., Shen A., Teachey D. T., Levine B. L., June C. H., Porter D. L., Grupp S. A. (2014). Chimeric antigen receptor T cells for sustained remissions in leukemia. New England Journal of Medicine, 371, 1507–1517. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. McCowan C., Wang S., Thompson A. M., Makubate B., Petrie D. J. (2013). The value of high adherence to tamoxifen in women with breast cancer: A community-based cohort study. British Journal of Cancer, 109, 1172–1180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. McGrady M. E., Ryan J. L., Brown G. A., Cushing C. C. (2015). Topical review: Theoretical frameworks in pediatric adherence-promotion interventions: Research findings and methodological implications. Journal of Pediatric Psychology, 40, 721–726. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. McGrady M. E., Ryan J. L., Gutiérrez-Colina A. M., Fredericks E. M., Towner E. K., Pai A. L. H. (2015). The impact of effective paediatric adherence promotion interventions: Systematic review and meta-analysis. Child Care Health and Development, 41, 789–802. [DOI] [PubMed] [Google Scholar]
  29. National Cancer Institute. (2013). SEER Cancer Statistics Review 1975-2010. Bethesda, MD: National Cancer Institute. [Google Scholar]
  30. Nguyen K., Devidas M., Cheng S. C., La M., Raetz E. A., Carroll W. L., Winick N. J., Hunger S. P., Gaynon P. S., Loh M. L.; Children's Oncology Group. (2008). Factors influencing survival after relapse from acute lymphoblastic leukemia: A Children's Oncology Group study. Leukemia, 22, 2142–2150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Oskarsson T., Söderhäll S., Arvidson J., Forestier E., Montgomery S., Bottai M., Lausen B., Carlsen N., Hellebostad M., Lähteenmäki P., Saarinen-Pihkala U. M., Jónsson Ó. G., Heyman M.; Nordic Society of Paediatric Haematology and Oncology (NOPHO) ALL Relapse Working Group. (2016). Relapsed childhood acute lymphoblastic leukemia in the Nordic countries: Prognostic factors, treatment and outcome. Haematologica, 101, 68–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Oxford Centre for Evidence-Based Medicine Levels of Evidence Working Group. (2016). The Oxford levels of evidence 2 Retrieved from http://www.cebm.net/ocebm-levels-of-evidence/
  33. Pai A. L. H., McGrady M. (2014). Systematic review and meta-analysis of psychological interventions to promote treatment adherence in children, adolescents, and young adults with chronic illness. Journal of Pediatric Psychology, 39, 918–931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Pai A. L. H., McGrady M. E. (2015). Assessing medication adherence as a standard of care in pediatric oncology. Pediatric Blood and Cancer, 62, S818–S828. [DOI] [PubMed] [Google Scholar]
  35. Palermo T. M. (2014). Evidence-based interventions in pediatric psychology: Progress over the decades. Journal of Pediatric Psychology, 39, 753–862. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Parker C., Waters R., Leighton C., Hancock J., Sutton R., Moorman A. V., Ancliff P., Morgan M., Masurekar A., Goulden N., Green N., Révész T., Darbyshire P., Love S., Saha V. (2010). Effect of mitoxantrone on outcome of children with first relapse of acute lymphoblastic leukaemia (ALL R3): An open-label randomised trial. Lancet, 376, 2009–2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Petrou S., Gray A. (2011). Economic evaluation alongside randomised controlled trials: Design, conduct, analysis, and reporting. BMJ, 342, d1548. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Rae C., Furlong W., Jankovic M., Moghrabi A., Naqvi A., Sala A., Samson Y., DePauw S., Feeny D., Barr R. (2014). Economic evaluation of treatment for acute lymphoblastic leukaemia in childhood. European Journal of Cancer Care, 23, 779–785. [DOI] [PubMed] [Google Scholar]
  39. Raetz E. A., Bhatla T. (2012). Where do we stand in the treatment of relapsed acute lymphoblastic leukemia? Hematology, 2012, 129–136. doi: 10.1182/asheducation-2012.1.129 [DOI] [PubMed] [Google Scholar]
  40. Raetz E. A., Borowitz M. J., Devidas M., Linda S. B., Hunger S. P., Winick N. J., Camitta B. M., Carrol W. L. (2008). Reinduction platform for children with first marrow relapse of acute lymphoblastic leukemia: A Children's Oncology Group study. Journal of Clinical Oncology, 26, 3971–3978. doi: 10.1200/JCO.2008.16.1414. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Reismüller B., Attarbaschi A., Peters C., Dworzak M. N., Pötschger U., Urban C., Fink F. M., Meister B., Schmitt K., Dieckmann K., Henze G., Haas O. A., Gadner H., Mann G.; Austrian Berlin-Frankfurt-Münster (BFM) Study Group. (2009). Long-term outcome of initially homogenously treated and relapsed childhood acute lymphoblastic leukaemia in Austria–a population-based report of the Austrian Berlin-Frankfurt-Munster (BFM) Study Group. British Journal of Haematology, 144, 559–570. [DOI] [PubMed] [Google Scholar]
  42. Sculpher M. J., Claxton K., Drummond M., McCabe C. (2006). Whither trial-based economic evaluation for health care decsion making? Health Economics, 15, 677–687. [DOI] [PubMed] [Google Scholar]
  43. Siebert U., Alagoz O., Bayoumi A. M., Jahn B., Owens D. K., Cohen D. J., Kuntz K. M. (2012). State-transition modeling: A report of the ISPOR-SMDM modeling good research practices task force–3. Medical Decision Making, 32, 690–700. [DOI] [PubMed] [Google Scholar]
  44. Sonnenberg F. A., Beck J. R. (1993). Markov models in medical decision making: A practical guide. Medical Decision Making, 13, 322–338. [DOI] [PubMed] [Google Scholar]
  45. The World Bank Group. (2016). United States: World development indicators. Retrieved from http://data.worldbank.org/country/united-states
  46. U.S. Department of Labor. (2016). Consumer price index: Medical care datatable. Retrieved from http://data.bls.gov/timeseries/CUUR0000SAM
  47. van Litsenburg R. R., Kunst A., Huisman J., Ket J. C., Kaspers G. J., Gemke R. J. (2014). Health status utilities in pediatrics: A systematic review of acute lymphoblastic leukemia. Medical Decision Making, 34, 21–32. [DOI] [PubMed] [Google Scholar]
  48. Weinstein M. C. (2006). Recent developments in decision-analytic modelling for economic evaluation. Pharmacoeconomics, 24, 1043–1053. [DOI] [PubMed] [Google Scholar]
  49. Weinstein M. C., Siegel J. E., Gold M. R., Kamlet M. S., Russell L. B. (1996). Recommendations of the panel on cost-effectiveness in health and medicine. JAMA, 276, 1253–1258. [PubMed] [Google Scholar]
  50. Weinstein M. C., Torrance G., McGuire A. (2009). QALYS: The basics. Value in Health, 12, S5–S9. [DOI] [PubMed] [Google Scholar]
  51. Wu Y. P., Rohan J. M., Martin S., Hommel K., Greenley R. N., Loiselle K., Ambrosino J., Fredericks E. M. (2013). Pediatric psychologist use of adherence assessments and interventions. Journal of Pediatric Psychology, 38, 595–604. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data
Click here for additional data file. (571.5KB, doc)

Articles from Journal of Pediatric Psychology are provided here courtesy of Oxford University Press

RESOURCES