Abstract
Objective:
As the population with multiple chronic conditions (MCC) increases, it is essential that randomized controlled trials (RCTs) consider MCC. Behavioral interventions have the potential to positively impact MCC patient outcomes, however a comprehensive review of consideration of MCC in these trials has not been conducted. The purpose of this systematic review is to determine the frequency with which participants with MCC are represented in behavioral intervention RCTs targeting chronic illness published 2000–2014.
Methods:
MEDLINE and EMBASE were searched from 2000 to 2014 to identify RCTs testing behavioral interventions among adults with chronic illness. A random sampling selection process was performed to identify 600 eligible studies representative of the literature. Two reviewers independently extracted information on consideration of MCC in eligibility criteria and evaluated the reporting and consideration of MCC in trial analyses. Risk of bias was assessed using the Cochrane Collaboration Risk of Bias Tool.
Results:
In 600 behavioral intervention RCTs, targeting MCC was rare (4.3%). Exclusion of MCC was common (68.3%) and was done through general, specific, or vague exclusion criteria. 218 (36.3%) trials reported presence of MCC through general or condition-specific measures. Comorbidities were only considered in 4.8% of all trial analyses.
Conclusions:
In this comprehensive systematic review of 600 studies published from 2000–2014, RCTs testing behavioral interventions rarely consider individuals with MCC, limiting generalizability. Given the public health relevance and limited evidence base, this work highlights the urgent need to improve the consideration of MCC in clinical trial research.
Keywords: systematic review, multimorbidity, behavioral medicine, randomized clinical trials
Multiple chronic conditions (MCC) are a growing public health challenge. In 2013, 17.6% of people age 18–64 and 60.8% of those 65 and older in the United States had more than one chronic condition, (National Center for Health Statistics, 2015) and 65% of total health care spending in the nation is used for this population (A. K. Parekh & Barton, 2010). As the United States continues to experience considerable growth in its older population, the proportion of people with MCC will undoubtedly expand (Ortman, Velkoff, & Hogan, 2014).
Management of the heterogeneous MCC patient population is complex (Anand K Parekh, Goodman, Gordon, Koh, & Conditions, 2011). Evidence synthesis and generation that specifically addresses MCC is needed to better inform health care decision making at the individual level and resource allocation at the societal level (Weiss et al., 2014). Randomized-controlled trials (RCTs) play a key role in the generation of evidence to inform health care decision making. However, the current clinical research enterprise strives to maximize internal validity by excluding participants with comorbidities which then limits external validity. Results of a previous systematic review limited to high impact journals indicate that up to 81% of RCTs reviewed excluded participants with MCC and only 2% of RCTs explicitly included participants with MCC (Van Spall, Toren, Kiss, & Fowler, 2007). Furthermore, when MCC participants are not excluded, reporting of co-occurring chronic conditions is limited. A survey of clinical trials revealed only a 44% reporting rate of participant comorbidities (Boyd, Vollenweider, & Puhan, 2012). A review of ongoing trials registered through ClinicalTrials.gov targeting chronic conditions found that 79% exclude at least one concomitant chronic condition (du Vaure, Dechartres, Battin, Ravaud, & Boutron, 2016). Exclusion of participants with MCC and underreporting of comorbid conditions results in a limited knowledge base for MCC patients and reduces our ability to improve their outcomes through evidence-based medicine.
It is well established that behavioral factors play a large role in outcomes for numerous chronic conditions, such as cancer, cardiovascular disease, and diabetes (Trask et al., 2002). In addition, the three leading causes of US deaths in the year 2000 were attributable to modifiable behavioral risk factors: tobacco use (18.1%), poor diet and physical inactivity (15.2%), and alcohol consumption (3.5%) (Mokdad, Marks, Stroup, & Gerberding, 2004). Behavioral interventions may be well suited for MCC patients due to their low side effect profile. They have the potential to positively impact MCC patient outcomes yet a comprehensive review assessing the consideration of MCC patients specifically in trials of behavioral interventions has not been conducted. Previously published reviews (du Vaure et al., 2016; Jadad, To, Emara, & Jones, 2011; Van Spall et al., 2007) did not focus on behavioral interventions and limited their sample to trials published in high impact or specialty journals.
To address this evidence gap, we conducted a comprehensive systematic review to assess the consideration of MCC in trials of behavioral interventions published from 2000 to 2014. We specifically focused on MCC in the context of trial eligibility criteria, reporting of MCC and consideration of MCC in trial analyses.
Methods
This review follows a written protocol (available at mccsystematicreview.wustl.edu/).
Data Sources and Searching
A librarian designed search strategies to retrieve all published RCTs in adults that test behavioral interventions and target chronic illness in PubMed Medline and Embase (full search strategy available in the online supplement). Database supplied limits were used to limit by 5- year strata (2000–2004, 2005–2009, and 2010–2014) and searches were limited to English language. Chronic illness was defined using general terms for chronic conditions and any of the following 20 conditions: arthritis, asthma, autism spectrum disorder, all cancer except non-melanoma skin, cardiac arrhythmias, chronic kidney disease, chronic obstructive pulmonary disease, congestive heart failure, coronary artery disease, dementia including Alzheimer’s and other senile dementias, depression, type 2 diabetes, hepatitis, human immunodeficiency virus, hyperlipidemia, hypertension, osteoporosis, schizophrenia, stroke, and substance abuse disorders. The selected conditions were identified from a list compiled by the MCC working group within the Department of Health and Human Services (Goodman, Posner, Huang, Parekh, & Koh, 2013). These conditions were chosen as they meet the definition for chronicity, are prevalent and have the potential to be modifiable by public health or clinical intervention. Searches were completed in February 2015. Results were sent to EndNote and the software-provided duplicate finder was used to identify duplicates.
Study Selection
Sampling strategy.
Given the large amount of literature potentially meeting eligibility criteria (broadly, behavioral RCTs target participants with chronic conditions published from 2000–2014), it was not feasible to include every eligible study in this review. Instead, a sampling strategy was used to produce a representative sample of literature of behavioral intervention RCTs over 15 years from 2000–2014. To examine the impact of calendar time, we sampled equal number of studies from three time periods (2000–2004, 2005–2009, 2010–2014). Three separate literature searches (using identical keywords and databases) were performed within the three defined time periods. Within each time period, search results were then randomly ordered using the RAND function in Microsoft Excel. Study selection was performed on the randomly ordered results within each time period until 200 eligible studies were identified (for a total of 600 eligible studies).
Inclusion criteria.
We a priori defined the following criteria for study inclusion: (1) Primary report of a RCT testing the efficacy or effectiveness of behavioral interventions (2) the study reports original data (protocols, post-trial follow-up studies, secondary or separate subgroup analyses were excluded) (3) the RCT targets chronic illness as defined above (4) the RCT applied eligibility criteria at the individual level (5) the trial was published in English (6) the RCT enrolled only adult subjects (≥18 years).
This review focuses on trials that included participants with at least one diagnosed chronic condition. As the objective of this review is to assess the inclusion of participants with MCC, it was necessary to ensure the participants had at least one condition and then assess for the presence or exclusion of further conditions. Two reviewers (CS, SI) independently screened studies identified through the literature search. Studies that did not meet all pre-specified inclusion criteria were excluded. Disagreements were resolved by consensus.
Data Extraction and Risk of Bias Assessment
Among a pool of 9 trained reviewers, each article was extracted independently by two reviewers using a standardized data extraction form. Reviewers were Master of Public Health students or had equivalent experience and underwent a thorough training process before the beginning of data abstraction. Data were collected and managed using Research Electronic Data Capture (REDCap), a secure, web-based application. Differences between abstractors were identified and disagreements were resolved by a third party (CS).
Study characteristics and risk of bias assessment.
Information on journal type, funding source, region, trial registration, intervention focus and sample size were abstracted to describe overall characteristics of included studies. The risk of bias of each trial was assessed using a modified version of the Cochrane Risk of Bias Tool (Higgins et al., 2011). The tool’s “other bias” item was removed as this element was unnecessary under the scope of this review.
Furthermore, the readers evaluated whether attrition bias was appropriately reported and explained but did not assess the amount of attrition. We created a risk of bias summary score for each article by combining the six individual risk of bias items (random sequence generation, allocation concealment, performance bias, detection bias, attrition bias, and reporting bias). Each bias item was scored as low risk of bias (−1), unclear risk of bias (0), or high risk of bias (+1). The summary score ranged from-6 to +6 with a lower score indicating lower risk of bias.
Eligibility criteria.
We reviewed inclusion criteria and assessed the frequency of the individual chronic conditions targeted and summarized the number of trials that specifically targeted MCC (i.e. targeting participants with > 1 condition). To assess the exclusion of individuals with MCCs, we reviewed the exclusion criteria for each trial and categorized criteria related to chronic conditions as specific, general, or vague (Table 1). An exclusion criteria was defined as ‘specific’ if the individual condition was mentioned by name or diagnostic criteria (i.e. type 2 diabetes or HbA1c>7%). We defined an exclusion criteria as ‘general’ if a general/generic terms was used to describe exclusions (i.e. chronic disease, additional comorbidities). A ‘vague’ exclusion criteria was defined as an exclusion criteria that does not provide sufficient information to classify a condition but is likely to result in exclusion of chronic conditions (i.e. serious medical problem, acute medical conditions, unstable medical conditions, mental illness, too ill). Among those trials that provided specific exclusion criteria, we determined the frequency of the individual 20 chronic conditions among those conditions excluded. Furthermore, among all trials that mentioned either specific, general or vague exclusion criteria, we assessed whether a justification for exclusion was provided. We further evaluated whether trials used a maximum age as exclusion criteria.
TABLE 1.
Definitions and examples of types of exclusions used in eligibility criteria to exclude participants with multiple chronic conditions
| Type of exclusion | Definition | Examples |
|---|---|---|
| Specific | exclusion of individual conditions by name or diagnostic criteria | Type 2 diabetes, HbA1c > 7% |
| General | exclusion of MCC by general term | chronic disease, additional comorbidities |
| Vague | exclusion criteria that is likely to result in exclusion of specific conditions, but do not provide enough information to determine which conditions would be excluded | serious medical problems, acute medical complications, unstable medical conditions, mental illness, too ill |
Reporting of MCC.
We evaluated whether trials reported the presence of MCC by reviewing the participants’ characteristics. If a trial reported any chronic condition in addition to the index chronic condition targeted, we defined this as reporting MCC. As the quality of reporting of MCC and information provided varied greatly across trials, we further distinguished between a general description of MCC in the study population (e.g. mean number of conditions) and specific description of the individual chronic condition (e.g. number of participants with depression). This approach allowed us to distinguish between reporting the overall presence of MCC (i.e. specific or general description provided) and to evaluate the types of measures used among those trials that provided a general description of MCC.
Analysis of MCC.
We assessed whether comorbidities were considered in any analyses of trial outcomes (subgroup analyses, comorbidity-adjusted analyses) among all trials and those that specifically reported MCC.
Data Synthesis and Analysis
Descriptive statistics including mean, median, proportions and frequencies were used to summarize the study characteristics and measures as described above. Analysis of variance (ANOVA) was used to assess the impact of time period on trial quality assessment (risk of bias), eligibility criteria and MCC reporting. All analyses were performed using SAS 9.4 (SAS Institute, Cary, NC).
Results
Study Selection
A total of 343,123 records were identified through the search (Figure 1). After removing duplicate records, 190,554 records remained. This included 52,599 from 2000–2004, 62,221 from 2005–2009, and 75,735 from 2010–2014. In accordance with our sampling strategy, we randomized results within each time period and screened records using eligibility criteria until we reached 200 eligible trials. We first screened titles, then abstracts, and finally full-text articles for eligibility. To achieve the desired 600 (3 X 200 = 600) article sample, we screened 41,186 search results (2000–2004: 18,813; 2005–2009: 12,200; 2010–2014: 75,735). 39,728 of these results were excluded for meeting exclusion criteria based screening title or abstract. The remaining 1,458 records (2000–2004: 534; 2005–2009: 493; 2010–2014: 431), 858 were identified as ineligible based on screening full-text, which left 600 articles meeting eligibility criteria.
Figure 1.
Study selection flow diagram of 600 articles selected from 343,123 identified records
Study Characteristics
Table 2 summarizes the study characteristics of all 600 studies included, stratified by time period. The majority of trials included were published in specialty journals (83.8%) and funded by non-industry sources/sponsors (76.5%). About half of the studies were conducted in North America (51.5%). The interventions varied widely in detail. The most common intervention focus was ‘psychological well-being’ (35.5%), followed by interventions focusing on weight management/diet/physical activity (27.5%). Interventions categorized as psychological well-being could include a variety of methods such as motivational interviewing, cognitive behavioral therapy, or support groups. While trial registration generally increased over time, only 97 out of the 600 trials were registered on a trial registry (e.g. clinicaltrials.gov). The sample size of included trials varied greatly, ranging from 8 – 8,517 participants with a median sample size across trials of 104.5 participants.
Table 2.
Study characteristics of 600 behavioral RCTs targeting chronic illness published from 2000–2014
| 2000–2004 (N=200) |
2005–2009 (N=200) |
2010–2014 (N=200) |
Total (N=600) |
|
|---|---|---|---|---|
| Journal type | ||||
| General Medicine | 29 (14.5) | 38 (19.0) | 30 (15.0) | 97 (16.2) |
| Specialty | 171 (85.5) | 162 (81.0) | 170 (85.0) | 503 (83.8) |
| Funding Source a | ||||
| Industry | 23 (11.6) | 13 (6.5) | 12 (6.0) | 48 (8.0) |
| Non-Industry | 145 (72.9) | 153 (76.5) | 160 (80.0) | 458 (76.5) |
| Not reported | 31 (15.6) | 34 (17.0) | 28 (14.0) | 93 (15.5) |
| Region | ||||
| North America | 121 (60.5) | 101 (50.5) | 87 (43.5) | 309 (51.5) |
| Europe | 50 (25.0) | 55 (27.5) | 61 (30.5) | 166 (27.7) |
| Middle East | 2 (1.0) | 1 (0.5) | 6 (3.0) | 9 (1.5) |
| Asia/Pacific | 23 (11.5) | 34 (17.0) | 41 (20.5) | 98 (16.3) |
| Latin America | 4 (2.0) | 5 (2.5) | 3 (1.5) | 12 (2.0) |
| Africa | 0 (0.0) | 4 (2.0) | 2 (1.0) | 6 (1.0) |
| Trial registrationb | ||||
| No | 200 (100.0) | 174 (87.0) | 129 (64.5) | 503 (83.8) |
| Yes - clinicaltrials.gov | 0 (0.0) | 14 (7.0) | 47 (23.5) | 61 (10.2) |
| Yes - other registry | 0 (0.0) | 12 (6.0) | 24 (12.0) | 36 (6.0) |
| Intervention focusc | ||||
| Weight management/diet/physical activity | 49 (24.5) | 57 (28.5) | 59 (29.5) | 165 (27.5) |
| Tobacco habits | 4 (2.0) | 3 (1.5) | 5 (2.5) | 12 (2.0) |
| Adherence to disease management | 45 (22.5) | 52 (26.0) | 41 (20.5) | 138 (23.0) |
| Psychological well-being | 82 (42.0) | 66 (33.0) | 63 (31.5) | 213 (35.5) |
| Other | 18 (9.0) | 22 (11.0) | 32 (16.0) | 72 (12.0) |
| Sample size (N=596) median (range) |
112 (8 – 2957) |
110 (14 – 3522) |
96.5 (10 – 8517) |
104.5 (8 – 8517) |
Funding source was considered industry if any industry sources were reported, and non-industry if only non-industry sources were reported.
Articles were considered reported if they mentioned registration in the article, including headers and footers. Registries were not searched to confirm registration. Registries other than clinicaltrials.gov included Australian New Zealand Clinical Trials Registry, ISRCTN (international), CCMO (The Netherlands), UMIN (Japan), etc.
The most common intervention focus was psychological well-being which included a wide range of interventions such as motivational interviewing, group-based cognitive-behavioral stress management, mindfulness-based cognitive therapy.
Risk of Bias Assessment
Table 3 summarizes the risk of bias assessment using the modified Cochrane Collaboration Risk of Bias Tool. Risk of bias was often categorized as unclear, meaning that the article did not report sufficient detail to judge the risk of bias in that category. This was seen most commonly for allocation sequence concealment (63.8%) and blinding of participants and personnel (67.8%). We found a significant decrease in risk of bias over time (p=<0.001). The overall mean risk of bias score changed from −2.2 in the first time period to −2.8 in the final time period.
Table 3.
Risk of bias of 600 behavioral RCTs targeting chronic illness published from 2000–2014 assessed by the Cochrane Collaboration Risk of Bias Tool
| Risk of Bias | 2000–2004 (N=200) |
2005–2009 (N=200) |
2010–2014 (N=200) |
Total (N=600) |
|---|---|---|---|---|
| Random sequence generation (selection bias) | ||||
| Low risk of bias | 81 (40.5) | 105 (52.5) | 100 (50.0) | 286 (47.7) |
| High risk of bias | 11 (5.5) | 6 (3.0) | 12 (6.0) | 29 (4.8) |
| Unclear risk of bias | 108 (54.0) | 89 (44.5) | 88 (44.0) | 285 (47.5) |
| Allocation sequence concealment (selection bias) | ||||
| Low risk of bias | 50 (25.0) | 61 (30.5) | 86 (43.0) | 197 (32.8) |
| High risk of bias | 8 (4.0) | 5 (2.5) | 7 (3.5) | 20 (3.3) |
| Unclear risk of bias | 142 (71.0) | 134 (67.0) | 107 (53.5) | 383 (63.8) |
| Blinding of participants and personnel (performance bias) | ||||
| Low risk of bias | 16 (8.0) | 20 (10.0) | 48 (24.0) | 84 (14.0) |
| High risk of bias | 36 (18.0) | 35 (17.5) | 38 (19.0) | 109 (18.2) |
| Unclear risk of bias | 148 (74.0) | 145 (72.5) | 114 (57.0) | 407 (67.8) |
| Blinding of outcome assessment (detection bias) | ||||
| Low risk of bias | 78 (39.0) | 103 (51.5) | 114 (57.0) | 295 (49.2) |
| High risk of bias | 17 (8.5) | 8 (4.0) | 25 (12.5) | 50 (8.3) |
| Unclear risk of bias | 105 (52.5) | 89 (44.5) | 61 (30.5) | 255 (42.5) |
| Incomplete outcome data (attrition bias) | ||||
| Low risk of bias | 153 (76.5) | 169 (84.5) | 154 (77.0) | 476 (79.3) |
| High risk of bias | 30 (15.0) | 20 (10.0) | 34 (17.0) | 84 (14.0) |
| Unclear risk of bias | 17 (8.5) | 11 (5.5) | 12 (6.0) | 40 (6.7) |
| Selective outcome reporting (reporting bias) | ||||
| Low risk of bias | 181 (90.5) | 187 (93.5) | 189 (94.5) | 557 (92.8) |
| High risk of bias | 18 (9.0) | 12 (6.0) | 11 (5.5) | 41 (6.8) |
| Unclear risk of bias | 1 (0.5) | 1 (0.5) | 0 (0.0) | 2 (0.4) |
| Mean risk of bias scorea | −2.2 (1.7) | −2.8 (1.7) | −2.8 (1.8) | −2.6 (1.7) |
p=.0002
Inclusion of Individual Conditions and MCC
The majority of trials (99.5%) included in our sample targeted a specific chronic condition and only 3 trials (0.5%) targeted chronic conditions in general without specifying conditions. Of the specific conditions considered in this review only hepatitis and autism spectrum disorders were not represented in our sample. Cancer was the most common condition of interest (17%), followed by diabetes (13%). The majority of trials (95.7%) only targeted one condition and only 26 (4.3%) of all trials directly targeted MCC (Online Supplement Table A1).
Exclusion of MCC
Over all time periods, 68.3% of trials excluded individuals with MCC using general, specific or vague exclusion criteria. Although the overall use of any of these exclusion criteria did not change over time, the use of vague exclusions decreased from 51.5% in the first time period (2000–2004) to 43.5% in the last time period (2010–2014) (Table 4). 31.2% of trials justified the exclusion by relating it to an individual’s inability to perform the intervention or otherwise mentally or physically participate in the trial, although evidence for this explanation was typically not provided. Among trials that reported specific exclusion criteria, substance abuse disorder (19%), dementia (16.9%), and schizophrenia (14.1%) were the most commonly excluded conditions. Furthermore, 27.8% of trials had an exclusion based on a maximum age with a median exclusion age of 65 years (Table 5). When considering time period, the use of a maximum age as an exclusion criteria slightly decreased over time (28.5% vs. 26.5%).
Table 4.
Exclusion of MCC in 600 behavioral RCTs targeting chronic illness published from 2000–2014
| 2000–2004 N=200 |
2005–2009 N=200 |
2010–2014 N=200 |
All years N=600 |
|
|---|---|---|---|---|
| Specific exclusion | 82 (41) | 89 (44.5) | 84 (42.0) | 255 (42.5) |
| General exclusion | 9 (4.5) | 14 (7.0) | 17 (8.5) | 40 (6.7) |
| Vague exclusion | 103 (51.5) | 99 (49.5) | 86 (43.0) | 288 (48.0) |
| Specific OR general exclusion | 85 (42.5) | 94 (47.0) | 91 (45.5) | 270 (45.0) |
| Specific OR general OR vague exclusion | 137 (68.5) | 134 (67.0) | 139 (69.5) | 410 (68.3) |
Note. An individual article could have reported multiple exclusion criteria that met different definitions
Table 5.
Exclusions by age in 600 behavioral RCTs targeting chronic illness published from 2000–2014
| 2000–2004 N=200 |
2005–2009 N=200 |
2010–2014 N=200 |
All years N=600 |
|
|---|---|---|---|---|
| Maximum age eligibility | ||||
| No | 143 (71.5) | 143 (71.5) | 147 (73.5) | 433 (72.2) |
| Yes | 57 (28.5) | 57 (28.5) | 53 (26.5) | 167 (27.8) |
| Maximum age (years) | ||||
| Mean | 65.8 | 66.1 | 68.8 | 66.8 |
| Median | 65.0 | 65.0 | 70.0 | 65.0 |
| Range | 45.0–80.0 | 25.0–85.0 | 40.0–89.0 | 25.0–89.0 |
We considered if a justification for exclusions was provided in studies. Inability to perform the intervention or otherwise mentally or physically participate in the trial was cited as justification for exclusions for specific exclusions (14.9%), general exclusions (25.0%), and vague exclusions (37.2%). Over all time periods 31.2% of trials with exclusions for specific, general, or vague exclusions justified these exclusions by citing an individual’s ability to perform the intervention or otherwise mentally or physically participate in the trial.
Reporting of MCC
Inclusion of participants with MCC was evaluated using reported participant characteristics (Table 6). A total of 218 trials (36.3%) reported the presence of participants with MCC, either through a condition-specific (n=175) and/or general measure (n=68). Among trials that provided condition-specific information, the mean number of additional conditions reported was 2.1. Of the 68 trials that gave a general description of the presence of MCC, 95.6% reported at least one general measure of MCC. Most commonly used measures of MCC included the mean number of MCC per participant (44.6%) and the number or percentage of participants with MCC (43.1%). The Charlson Comorbidity Index (Charlson, Pompei, Ales, & MacKenzie, 1987), a summary score of 17 comorbidities designed to predict 10 year mortality of individuals, was reported in 15.4% of trials giving a general description of the presence of MCC.
Table 6.
Presence of MCC and type of MCC description reported in patient characteristics in in 600 behavioral RCTs targeting chronic illness published from 2000–2014
| 2000–2004 | 2005–2009 | 2010–2014 | Total | |
|---|---|---|---|---|
| Condition specific description | 49 (24.5) | 58 (29.0) | 68 (34.0) | 175 (29.2) |
| General description | 22 (11.0) | 20 (10.0) | 26 (13.0) | 68 (11.3) |
| Specific OR general description | 63 (31.5) | 72 (36.0) | 83 (41.5) | 218 (36.3) |
Consideration of MCC in Analyses
Comorbidities were considered in analysis in only 4.8% of all trials (n=600). Of the 218 trials that reported MCC, 13.3% of trials used any information related to comorbidities in analyses. Of the 26 studies that targeted MCC, 2 (7.7%) considered comorbidities in analysis.
Conclusions
Summary of Main Results
In this systematic review of a representative sample of 600 RCTs testing behavioral interventions published over the last 15 years, trials rarely target individuals with MCC. Additionally, they frequently exclude individuals with MCC due to mostly vaguely reported exclusion criteria and exclusion criteria based on factors associated with MCC, such as age. Approximately a third of the trials report the presence of MCC with variation in quality and detail of information provided. Furthermore, MCC are often not specifically considered in trial analyses.
Comparison to Other Studies
Our findings regarding RCT eligibility criteria (i.e. MCC inclusion and exclusion) are in line with results of previously published reviews (Boyd et al., 2012; Jadad et al., 2011; Van Spall et al., 2007). Jadad et al. evaluated 284 trials that were published in the top five high impact general medical journals and specialized journals with a focus on highly prevalent chronic conditions. Similar to our results, individuals with MCC were excluded in the majority of RCTs and only a small number of trials (2.1%) explicitly included MCC patients (Jadad et al., 2011). In a structured sampling review, van Spall et al. determined the nature and extent of exclusion criteria among 283 RCTs, published in general medical journals between 1995 and 2005. 81% of the RCTs excluded individuals for medical comorbidities, and, together with age, medical comorbidities were among the two most common categories of poorly justified participant exclusions (Van Spall et al., 2007). du Vaure et al. assessed 319 ongoing trials targeting 10 common chronic conditions and registered at ClinicalTrials.gov, including 57 (17.9%) trials involving behavioral interventions. Among these ongoing registered trials, 79% excluded at least one concomitant chronic condition (du Vaure et al., 2016).
Age is a risk factor for MCC and clinical trials historically tend to exclude older participants (Vogeli et al., 2007; Zulman et al., 2011). Several reviews, including our own, have therefore specifically evaluated age as an exclusion criteria in clinical trials (Jadad et al., 2011; Van Spall et al., 2007; Zulman et al., 2011). The percentages of trials that defined an upper age range as exclusion criteria ranged from 20.2% to 38.5% in these previous reports which aligns well with our results (Jadad et al., 2011; Van Spall et al., 2007; Zulman et al., 2011).
Our finding of poor reporting and limited consideration of MCC in trial analyses are consistent with those reported in other reviews. In a literature survey of trials testing the effectiveness of drugs and non-drug therapies for four common chronic diseases (COPD, diabetes, congestive heart failure, stroke), 43% of the 161 included trials described the prevalence of comorbidities among participants with the index disease. However, the comorbidity reporting was limited in terms of providing operational definitions and methods of comorbidity ascertainment. Very few trials (3.1%) conducted subgroup analyses based on the presence of a comorbidity (Boyd et al., 2012).
Our review differs from previous reviews in several ways by (i) focusing solely on RCTs testing behavioral interventions; (ii) considering a previously defined list of 20 chronic conditions chosen for their chronicity, prevalence and potential to be modifiable by public health and/or clinical interventions (iii) not limiting our search strategy and sample to trials published in high impact journals. However, despite these methodological differences, the current body of evidence appears to elucidate a consistent pattern of poor consideration of MCC in the current clinical trial research enterprise.
Possible Explanations for Observed Findings
The reasons for this observed pattern are possibly multifactorial. First, design choices of each individual trial are informed by its purpose (i.e. efficacy versus effectiveness) which could potentially impact the inclusion and exclusion of individuals with MCC. An efficacy trial with the goal of demonstrating the highest effect estimate possible tends to include a homogeneous population, likely to be responsive to the intervention. In contrast, an effectiveness trial aims to include a more heterogeneous patient population to evaluate the effect of an intervention under real-world circumstances (Loudon et al., 2015; Thorpe et al., 2009). Both efficacy and effectiveness trials were eligible for inclusion in our review but, unfortunately, we were unable to assess the potential relationship between trial purpose and eligibility criteria due to the limited number of trials that included MCC and provided sufficient information to conduct a meaningful evaluation. Second, up to 38.5% of clinical trials tend to exclude participants based on an upper age limit (Jadad et al., 2011; Van Spall et al., 2007; Zulman et al., 2011). As the prevalence of MCC among individuals increases with age, incorporating upper age limits could possibly have a large impact on included patient populations and representativeness of MCC (Zulman et al., 2011). Third, despite the efforts to improve trial reporting, for example through the introduction and endorsement of the CONSORT statement, the quality of trial information provided is often still suboptimal (Chan & Altman, 2005; Gabler et al., 2016; Hopewell, Dutton, Yu, Chan, & Altman, 2010). Particularly in the context of MCC, the overall lack of specific information reported and variation of detail provided regarding eligibility criteria and patient characteristics limited our ability to gain a better overall picture of the presence and extent of MCC in the population for the majority of individual trials. Fourth, defining and measuring MCC is complex. There is currently no standard list of chronic diseases to be considered and numerous MCC measures have been proposed (de Groot, Beckerman, Lankhorst, & Bouter, 2003; Diederichs, Berger, & Bartels, 2011). The lack of a standard framework to capture the multidimensional concept of MCC is a possible barrier for reporting MCC in clinical trials. In our review, the majority of trials that included MCC reported either the mean number of MCC or percentage of participants with MCC. However, it is unclear to date how this information can be translated into clinical guidelines and practices given that individual patients can have various numbers and combinations of individual chronic conditions. Last, although subgroup analyses are one possible strategy to elucidate potential heterogeneity of treatment effect (Varadhan, Segal, Boyd, Wu, & Weiss, 2013), only few RCTs conducted subgroup analyses based on comorbidities. Subgroup analyses often require large sample sizes to assure statistical power and are prone to multiple hypothesis testing so careful consideration of the number and combinations of individual MCC is necessary (Burke, Sussman, Kent, & Hayward, 2015; Weiss et al., 2014). These aspects may have contributed to the limited consideration of MCC in trial analyses. However, looking at outcomes by different combinations of comorbidities could provide useful information regarding which clusters of comorbidities made interventions more or less effective. Recently proposed recommendations and guidelines for analyzing heterogeneity of treatment effects hold the promise to improve the consideration of MCC in future trial analyses (Kent, Rothwell, Ioannidis, Altman, & Hayward, 2010; Varadhan et al., 2013).
Strengths and Limitations
This review is substantially more comprehensive than previous reviews in the scope of literature assessed, including the time period covered (2000–2014), the number of chronic conditions considered, and the sample size of included trials (n=600), as well as the breadth of the assessment of participant inclusion. While other reviews have assessed certain aspects of participant inclusion, this review evaluates inclusion across all phases of the trial, including eligibility, screening, selection, and analysis. In addition, this is the first review to assess inclusion of MCC specifically within behavioral trials. Furthermore, we ensured that our sample was representative of the published literature over the past 15 years by not limiting our sample to high impact journals or specific type of journals. This comprehensive assessment was performed following best practices for systematic reviews, including a comprehensive search strategy and sampling technique and double extraction of data by trained readers to ensure quality of data.
The following limitations should be considered when interpreting our findings. We only included primary reports of trials. It is possible that MCC information was provided in subgroup or secondary analyses. We focused on trials targeting a chronic condition to ensure that all participants had at least one chronic condition and to be able to assess for the presence of further conditions. This inclusion criteria may limit the generalizability of our results to the broader behavioral intervention literature as many of these trials are often focused on primary prevention and may therefore include population at risk for developing chronic conditions. Although focusing on a specific list of 20 chronic conditions was a strength of our study, it may have limited our ability to consider information from trials that did not meet our definition of chronic conditions and/or presented information in a way that did not allow us to apply these criteria.
Conclusions
In this comprehensive systematic review of 600 trials published between 2000 and 2014, RCTs testing behavioral interventions rarely include individuals with MCC, which raises questions regarding the generalizability of their findings. Given the public health relevance and currently limited evidence base, our review highlights the urgent need to improve the consideration of MCC in the clinical trial research enterprise.
Supplementary Material
Acknowledgments
This project has been funded in whole or in part with federal funds from the National Cancer Institute, National Institutes of Health, under Contract No. HHSN261200800001E. The content of this publication does not necessarily reflect the views or polices of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations to imply endorsement by the U.S. Government. Stoll, Izadi, Philpott, Colditz and Winter are supported, in part, by the Foundation for Barnes-Jewish Hospital, St Louis. Colditz is also supported by the Alvin J. Siteman Cancer Center Biostatistics Shared Resource, P30 CA091842.
We thank the following individuals who were instrumental in the performing of the data extraction: Alwiya Ahmed, MPH, Joyce Dieterly, MPH, Nageen Mir, MPH, Jeff Viox, BA, Eva Williams, BA, Daoxin Yin, MPH, and Jingsong Zhao, MPH.
Contributor Information
Carolyn R. T. Stoll, Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, Saint Louis, Missouri
Sonya Izadi, Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, Saint Louis, Misouri.
Susan Fowler, Becker Medical Library, Washington University School of Medicine, Saint Louis, Missouri; Brown School Library, Brown School at Washington University in St. Louis, St. Louis, Missouri..
Sydney Philpott, Department of Surgery, Washington University School of Medicine, Saint Louis, Missouri.
Paige Green, Behavioral Research Program, Division of Cancer Control & Population Sciences, National Cancer Institute, Bethesda, Maryland.
Jerry Suls, Behavioral Research Program, Division of Cancer Control & Population Sciences, National Cancer Institute, Bethesda, Maryland.
Anke C. Winter, Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, Saint Louis, Missouri Fresenius Medical Care..
Graham A. Colditz, Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, Saint Louis, Missouri.
References
- Boyd CM, Vollenweider D, & Puhan MA (2012). Informing evidence-based decision-making for patients with comorbidity: availability of necessary information in clinical trials for chronic diseases. PLoS One, 7(8), e41601. doi: 10.1371/journal.pone.0041601 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burke JF, Sussman JB, Kent DM, & Hayward RA (2015). Three simple rules to ensure reasonably credible subgroup analyses. BMJ, 351, h5651. doi: 10.1136/bmj.h5651 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chan AW, & Altman DG (2005). Epidemiology and reporting of randomised trials published in PubMed journals. Lancet, 365(9465), 1159–1162. doi: 10.1016/S0140-6736(05)71879-1 [DOI] [PubMed] [Google Scholar]
- Charlson ME, Pompei P, Ales KL, & MacKenzie CR (1987). A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. Journal of chronic diseases, 40(5), 373–383. [DOI] [PubMed] [Google Scholar]
- de Groot V, Beckerman H, Lankhorst GJ, & Bouter LM (2003). How to measure comorbidity. a critical review of available methods. J Clin Epidemiol, 56(3), 221–229. [DOI] [PubMed] [Google Scholar]
- Diederichs C, Berger K, & Bartels DB (2011). The measurement of multiple chronic diseases--a systematic review on existing multimorbidity indices. J Gerontol A Biol Sci Med Sci, 66(3), 301–311. doi: 10.1093/gerona/glq208du [DOI] [PubMed] [Google Scholar]
- Vaure CB, Dechartres A, Battin C, Ravaud P, & Boutron I (2016). Exclusion of patients with concomitant chronic conditions in ongoing randomised controlled trials targeting 10 common chronic conditions and registered at ClinicalTrials.gov: a systematic review of registration details. BMJ open, 6(9), e012265. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gabler NB, Duan N, Raneses E, Suttner L, Ciarametaro M, Cooney E, . . . Kravitz RL (2016). No improvement in the reporting of clinical trial subgroup effects in high-impact general medical journals. Trials, 17(1), 320. doi: 10.1186/s13063-016-1447-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goodman RA, Posner SF, Huang ES, Parekh AK, & Koh HK (2013). Defining and Measuring Chronic Conditions: Imperatives for Research, Policy, Program, and Practice. Preventing Chronic Disease, 10, E66. doi: 10.5888/pcd10.120239 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Higgins JP, Altman DG, G0tzsche PC, Jüni P, Moher D, Oxman AD, . . . Sterne JA (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ: British Medical Journal, 343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hopewell S, Dutton S, Yu LM, Chan AW, & Altman DG (2010). The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ, 340, c723. doi: 10.1136/bmj.c723 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jadad AR, To MJ, Emara M, & Jones J (2011). Consideration of multiple chronic diseases in randomized controlled trials. JAMA: the journal of the American Medical Association, 306(24), 2670–2672. [DOI] [PubMed] [Google Scholar]
- Kent DM, Rothwell PM, Ioannidis JP, Altman DG, & Hayward RA (2010). Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials, 11, 85. doi: 10.1186/1745-6215-11-85 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, & Zwarenstein M (2015). The PRECIS-2 tool: designing trials that are fit for purpose. BMJ, 350, h2147. doi: 10.1136/bmj.h2147 [DOI] [PubMed] [Google Scholar]
- Mokdad AH, Marks JS, Stroup DF, & Gerberding JL (2004). Actual causes of death in the United States, 2000. JAMA, 291(10), 1238–1245. doi: 10.1001/jama.291.10.1238 [DOI] [PubMed] [Google Scholar]
- National Center for Health Statistics. (2015). Health, United States, 2014: With Special Feature on Adults Aged 55–64. [PubMed] [Google Scholar]
- Parekh AK, & Barton MB (2010). The challenge of multiple comorbidity for the US health care system. JAMA, 303(13), 1303–1304. doi: 10.1001/jama.2010.381 [DOI] [PubMed] [Google Scholar]
- Parekh AK, Goodman RA, Gordon C, Koh HK, & Conditions HIW o MC (2011). Managing multiple chronic conditions: a strategic framework for improving health outcomes and quality of life. Public health reports, 126(4), 460. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, . . . Chalkidou K (2009). A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol, 62(5), 464–475. doi: 10.1016/j.jclinepi.2008.12.011 [DOI] [PubMed] [Google Scholar]
- Trask PC, Schwartz SM, Deaner SL, Paterson AG, Johnson T, Rubenfire M, & Pomerleau OF (2002). Behavioral medicine: the challenge of integrating psychological and behavioral approaches into primary care. Eff Clin Pract, 5(2), 75–83. [PubMed] [Google Scholar]
- Van Spall HG, Toren A, Kiss A, & Fowler RA (2007). Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA, 297(11), 1233–1240. doi: 10.1001/jama.297.11.1233 [DOI] [PubMed] [Google Scholar]
- Varadhan R, Segal JB, Boyd CM, Wu AW, & Weiss CO (2013). A framework for the analysis of heterogeneity of treatment effect in patient-centered outcomes research. J Clin Epidemiol, 66(8), 818–825. doi: 10.1016/j.jclinepi.2013.02.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vogeli C, Shields AE, Lee TA, Gibson TB, Marder WD, Weiss KB, & Blumenthal D (2007). Multiple chronic conditions: prevalence, health consequences, and implications for quality, care management, and costs. J Gen Intern Med, 22 Suppl 3, 391–395. doi: 10.1007/s11606-007-0322-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weiss CO, Varadhan R, Puhan MA, Vickers A, Bandeen-Roche K, Boyd CM, & Kent DM (2014). Multimorbidity and evidence generation. Journal of general internal medicine, 29(4), 653–660. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zulman DM, Sussman JB, Chen X, Cigolle CT, Blaum CS, & Hayward RA (2011). Examining the evidence: a systematic review of the inclusion and analysis of older adults in randomized controlled trials. Journal of general internal medicine, 26(7), 783–790. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.