Abstract
Objective
To compare different definitions of multimorbidity to identify patients with higher health care resource utilization.
Patients and Methods
We used a multinational retrospective cohort including 147,806 medical inpatients discharged from 11 hospitals in 3 countries (United States, Switzerland, and Israel) between January 1, 2010, and December 31, 2011. We compared the area under the receiver operating characteristic curve (AUC) of 8 definitions of multimorbidity, based on International Classification of Diseases codes defining health conditions, the Deyo-Charlson Comorbidity Index, the Elixhauser-van Walraven Comorbidity Index, body systems, or Clinical Classification Software categories to predict 30-day hospital readmission and/or prolonged length of stay (longer than or equal to the country-specific upper quartile). We used a lower (yielding sensitivity ≥90%) and an upper (yielding specificity ≥60%) cutoff to create risk categories.
Results
Definitions had poor to fair discriminatory power in the derivation (AUC, 0.61-0.65) and validation cohorts (AUC, 0.64-0.71). The definitions with the highest AUC were number of (1) health conditions with involvement of 2 or more body systems, (2) body systems, (3) Clinical Classification Software categories, and (4) health conditions. At the upper cutoff, sensitivity and specificity were 65% to 79% and 50% to 53%, respectively, in the validation cohort; of the 147,806 patients, 5% to 12% (7474 to 18,008) were classified at low risk, 38% to 55% (54,484 to 81,540) at intermediate risk, and 32% to 50% (47,331 to 72,435) at high risk.
Conclusion
Of the 8 definitions of multimorbidity, 4 had comparable discriminatory power to identify patients with higher health care resource utilization. Of these 4, the number of health conditions may represent the easiest definition to apply in clinical routine. The cutoff chosen, favoring sensitivity or specificity, should be determined depending on the aim of the definition.
Abbreviations and Acronyms: AUC, area under the receiver operating characteristic curve; CCI, Chronic Condition Indicator; CCS, Clinical Classification Software; ICD, International Classification of Diseases; IQR, interquartile range; LOS, length of stay; NICE, National Institute for Health and Care Excellence; WHO, World Health Organization
With the increase in life expectancy, multimorbidity affects an increasing number of patients.1, 2, 3, 4, 5 Given its association with higher health care resource utilization, polypharmacy, and bad quality of life, it represents a significant burden for patients and health care systems.6, 7, 8, 9, 10, 11, 12 Its definition remains nonetheless not well standardized.5,7 Although the World Health Organization (WHO) and the National Institute for Health and Care Excellence (NICE) guidelines define multimorbidity as 2 or more chronic conditions,13,14 we still lack selection criteria for the conditions to include, particularly on how to differentiate acute and chronic conditions.5, 6, 7, 8 Consequently, the number and types of conditions assessed vary across most studies, making them difficult to compare.5, 6, 7
The prevalence of multimorbidity and its consequences are unsurprisingly influenced by the definition used. Recent reviews underlined the need of standardizing the assessment of multimorbidity and of conducting studies to test the best cutoffs for the number and types of conditions to identify patients with higher burden of multimorbidity because the cutoffs chosen and the accuracy of specific definitions of multimorbidity may differ according to the outcome assessed.5, 6, 7 For example, definitions of multimorbidity developed to assess mortality, such as the Charlson Comorbidity Index, may not be accurate to assess the risk of other adverse health outcomes, such as hospital readmission.6,15
Our objective for this study was to evaluate and compare the performance of different definitions of multimorbidity to identify patients with higher health care resource utilization, assessed as hospital readmission and prolonged length of stay (LOS), with the goal to standardize multimorbidity definition. Our specific aims were to (1) compare the discriminatory power of the definitions, (2) identify for each definition of multimorbidity a lower cutoff favoring sensitivity and an upper cutoff favoring specificity to classify the patients at low, intermediate, or high risk of higher health care resource utilization and that may be used depending on the context and purpose of using the definition, and (3) compare those definitions with WHO/NICE guidelines’ definition of multimorbidity.
Patients and Methods
Study Design
We used a retrospective multinational cohort including all 147,806 medical inpatients discharged from 11 hospitals in 3 countries (United States, Switzerland, and Israel) between January 1, 2010, and December 31, 2011. The cohort included only patients admitted to a medical ward and discharged home or to a nursing home because the study was designed to investigate hospital readmissions in medical inpatients.16 To minimize the risk of including observation stays, we further included only patients with a hospital LOS of 1 day or more. We randomly selected 4 hospitals in the United States, 2 hospitals in Switzerland, and 1 hospital in Israel to develop the definitions of multimorbidity and the remaining 3 US hospitals and 1 Swiss hospital to validate them. Reporting is in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement.17
The institutional review board of each participating site reviewed the study and determined it to be non–human subjects research, as it involved a secondary analysis of anonymized data.
Classification of Diagnoses
We used International Classification of Diseases (ICD) diagnosis codes to define the different health conditions. First, we used the Clinical Classification Software (CCS) to merge the health conditions into 285 exclusive categories.18 Second, we used the Chronic Condition Indicator (CCI) to classify the health conditions as chronic or not chronic, as well as into 18 exclusive body system categories.19 The CCI defines a condition as chronic if it lasts 12 months or longer and places limitations on self-care, independent living, and social interactions and/or results in the need for ongoing intervention with medical products, services, and special equipment. Both the CCS and the CCI have been developed by the Healthcare Cost and Utilization Project, a federal-state-industry partnership sponsored by the Agency for Healthcare Research and Quality,18,19 and are available for ICD, Ninth Revision (ICD-9) and ICD, Tenth Revision (ICD-10) codes. Finally, we used the Deyo-Charlson Comorbidity Index and the Elixhauser-van Walraven Comorbidity Index based on enhanced ICD-9, Clinical Modification and ICD-10 codes.15,20, 21, 22, 23 During the time of the study, the United States and Israel used ICD-9 and Switzerland used ICD-10.
Definitions of Multimorbidity
We assessed 8 definitions of multimorbidity: (1) 2 or more distinct body system categories and number of health conditions, (2) 2 or more distinct body system categories and number of chronic health conditions, (3) number of distinct body system categories, (4) number of CCS categories, (5) number of health conditions, (6) number of chronic health conditions, (7) Deyo-Charlson Comorbidity Index, and (8) Elixhauser-van-Walraven Comorbidity Index.
Outcomes
Our primary outcome was a composite end point including any readmission to the same hospital within 30 days after discharge and/or a prolonged LOS, defined as a stay longer than or equal to country-specific upper (75%) quartile. Secondary outcomes were the single components of the primary outcome: (1) any readmission to the same hospital within 30 days after discharge and (2) a prolonged LOS. We used country-specific LOS because the LOS differed between the countries included in the study (longer LOS in Switzerland than in the United States or in Israel).24,25
Statistical Analyses
We present baseline characteristics as median with interquartile range (IQR) for continuous variables and proportions for categorical variables. We calculated the discriminatory power of the 8 different definitions of multimorbidity in both the derivation and validation cohorts using the area under the receiver operating characteristic curve (AUC), presented with 95% CIs.26 Then we calculated the sensitivity, specificity, positive predictive value, negative predictive value, and positive and negative likelihood ratios of the different definitions of multimorbidity at all possible cutoff values in the derivation cohort. For each definition, we then identified a lower cutoff optimizing sensitivity and an upper cutoff optimizing specificity, allowing us to build 3 risk categories (low, intermediate, and high) of higher health care resource utilization. We defined the lower cutoff as having a sensitivity of at least 90% in order to minimize the rate of false negatives, ie, the number of patients with multimorbidity who would be missed. If several cutoffs met this criterion, we chose the cutoff with the best specificity. We defined the upper cutoff as having a specificity of at least 60% in order to minimize the rate of false positives and optimize the identification of patients with true multimorbidity (true positives). If several cutoffs met this criterion, we chose the cutoff with the best sensitivity. We then used the validation cohort to validate the lower and upper cutoffs identified in the derivation cohort by computing the test characteristics. We used the DeLong test in the derivation and validation cohorts separately to compare the different definitions of multimorbidity. Finally, we compared the performance of these 8 definitions at the identified cutoffs with the historical definition of multimorbidity, ie, the presence of 2 or more chronic health conditions. The lower and upper cutoffs were determined for the primary outcome first and then used to assess the performance of the definitions for the secondary outcomes in the derivation and validation cohorts.
All analyses were performed using R version 3.4.4 (R Project for Statistical Computing).
Results
The median age of the 147,806 study patients was 63 years (IQR, 50-75 years), 48.2% of whom (71,175) were men (Table 1). The median number of health conditions and chronic health conditions were 9 (IQR, 5-13) and 5 (IQR, 3-7), respectively. After random selection of the hospitals, 92,071 of the 147,806 patients (62.3%) were included in the derivation cohort and the remaining 55,735 patients (37.7%) in the validation cohort. The median age was lower in the validation cohort than in the derivation cohort (60 vs 65 years).
Table 1.
| Characteristic | Total population (N=147,806) | Derivation cohort (n= 92,071) | Validation cohort (n= 55,735) |
|---|---|---|---|
| Age (y) | 63 (50-75) | 65 (51-77) | 60 (47-72) |
| Men | 71,175 (48.2) | 44,090 (47.9) | 27,085 (48.6) |
| Country | |||
| United States | 89,268 (60.4) | 42,306 (45.9) | 46,962 (84.3) |
| Switzerland | 42,739 (28.9) | 33,966 (36.9) | 8773 (15.7) |
| Israel | 15,799 (10.7) | 15,799 (17.2) | 0 (0) |
| Description of multimorbidity | |||
| Number of health conditions | 9 (5-13) | 8 (4-12) | 10 (6-14) |
| Number of chronic health conditions | 5 (3-7) | 4 (2-7) | 5 (3-8) |
| Number of CCS categories | 6 (4-9) | 6 (3-9) | 7 (5-10) |
| Number of body system categories | 4 (3-6) | 4 (2-6) | 5 (3-7) |
| Deyo-Charlson Comorbidity Index | 2 (0-3) | 1 (0-3) | 2 (0-3) |
| Elixhauser-van Walraven Comorbidity Index | 5 (0-12) | 5 (0-11) | 5 (0-12) |
| Hospitalization characteristics | |||
| Length of stay (d) | 4 (3-8) | 4 (3-5) | 4 (3-8) |
| Number of admissions in the past year | 0 (0-2) | 0 (0-2) | 0 (0-2) |
CCS = Clinical Classification Software; IQR = interquartile range.
Data are presented as median (IQR) or No. (percentage) of patients.
Performance of the Different Definitions of Multimorbidity for the Primary Outcome
Definitions had poor to fair discriminatory power, with an AUC of 0.61 to 0.65 in the derivation cohort and 0.64 to 0.71 in the validation cohort (Table 2 and Figure; details in Supplemental Table 1, available online at http://www.mcpiqojournal.org). The definitions based on the number of (1) health conditions with 2 or more body system categories, (2) body system categories, (3) CCS categories, and (4) health conditions performed the best, with AUCs of 0.65 to 0.65 in the derivation cohort and 0.71 to 0.71 in the validation cohort.
Table 2.
Performance of the Different Definitions of Multimorbidity for the Primary Outcome of Readmission and/or Prolonged Length of Stay in the Derivation and Validation Cohortsa,b
| Definition of multimorbidity | AUC (95% CI) in the derivation cohort | AUC (95% CI) in the validation cohort | Cutoff favoring sensitivityc | Cutoff favoring specificityc |
|---|---|---|---|---|
| ≥2 Body system categories and number of health conditions | 0.65 (0.643-0.651)d | 0.71 (0.706-0.715)e | ≥3 | ≥9 |
| ≥2 Body system categories and number of chronic health conditions | 0.61 (0.606-0.614)f | 0.64 (0.634-0.644)e | ≥1 | ≥6 |
| Number of distinct body system categories | 0.65 (0.649-0.656)g | 0.71 (0.700-0.709)h | ≥2 | ≥5 |
| Number of CCS categories | 0.65 (0.648-0.656)g | 0.71 (0.704-0.714)i | ≥2 | ≥7 |
| Number of health conditions | 0.65 (0.645-0.653)j | 0.71 (0.706-0.715)i | ≥3 | ≥9 |
| Number of chronic health conditions | 0.61 (0.605-0.613)k | 0.64 (0.633-0.642)e | ≥1 | ≥6 |
| Deyo-Charlson Comorbidity Index | 0.62 (0.611-0.618)f,k | 0.64 (0.637-0.646)e,l,m | NAn | ≥6 |
| Elixhauser-Van-Walraven Comorbidity Index | 0.62 (0.611-0.619)f,k | 0.65 (0.643-0.653)m | 0 | ≥6 |
AUC = area under the receiver operating characteristic curve; CCS = Clinical Classification Software; NA = not available.
Prolonged length of stay was defined as a length of stay as long as or longer than the country-specific upper quartile (75%).
The cutoff values relate to the number of items in the respective definitions of multimorbidity. We defined the lower cutoff as having a sensitivity of ≥90%. If several cutoffs met this criterion, we chose the cutoff with the best specificity. We defined the upper cutoff as having a specificity of ≥60%. If several cutoffs met this criterion, we chose the cutoff with the best sensitivity.
These AUCs were statistically significantly different (P<.05) according to DeLong test conducted separately in the derivation and validation data sets. P values adjusted for multiple comparisons using Bonferroni correction.
There was no cutoff with ≥90% sensitivity for the Deyo-Charlson Comorbidity Index except for a score of zero, which would have resulted in one group of patients only.
Figure.
Area area under the receiver operating characteristic curve (AUC) of the different definitions of multimorbidity to predict any 30-day hospital readmission and/or a prolonged length of stay (defined as a stay longer than or equal to the country-specific upper quartile [75%]) in the derivation cohort. A, Two or more distinct body system categories and number of health conditions. B, Two or more distinct body system categories and number of chronic health conditions. C, Number of distinct body system categories. D, Number of Clinical Classification Software categories. E, Number of health conditions. F, Number of chronic health conditions. G, Deyo-Charlson Comorbidity Index. H, Elixhauser-van-Walraven Comorbidity Index.
We could identify a lower and an upper cutoff meeting our predefined sensitivity (≥90%) and specificity (≥60%) criteria for all definitions except for the definition based on the Deyo-Charlson Comorbidity Index (sensitivity always <90%). For the 4 definitions that performed best, both sensitivity and specificity were around 60% at the upper cutoff in the derivation cohort. In the validation cohort, the sensitivity was higher (75%-79%), but the specificity was lower (50%-53%). At the lower cutoff, the definition based on the number of chronic health conditions performed best with a sensitivity of 96% in the derivation cohort and 99% in the validation cohort, but at the cost of a very low specificity (8% and 4%, respectively). The historical definition of multimorbidity had a sensitivity of 89% and 95%, for a specificity of 20% and 12% in the derivation and validation cohorts, respectively.
Risk Categories
In the derivation cohort (n=92,071), 6.5% to 15.8% of the patients (5995 to 14,209) were classified in the low-risk category and 30.1% to 45.6% (27,701 to 42,016) in the high-risk category, depending on the definition of multimorbidity (Table 3). In the validation cohort (n=55,735), the proportions of patients in the low-risk category were lower (2.7%-8.0% [1479 to 4483]), while up to 31.4% to 58.9% (17,525 to 32,206) were classified at high risk. In the total cohort (N=147,806), the smallest proportion of patients at low risk was found for the definition using the number of chronic health conditions (5.0% [7474]) and the largest proportion for the definition using the number of health conditions with 2 or more body system categories (12.4% [18,008]). At the opposite, the smallest proportion of patients at high risk was found for the definition using the Deyo-Charlson Comorbidity Index (30.6% [45,226]) and the largest proportion for the definitions using the number of health conditions with 2 or more body system categories or the number of health conditions alone (both 50.0% [72,375 and 72,435]).
Table 3.
Observed Proportions of Patients in Risk Categories According to Definitions of Multimorbidity for the Primary Outcome of Hospital Readmission and/or Prolonged Length of Stay in the Derivation, Validation, and Total Cohortsa,b
| Definition of multimorbidity | Risk category |
||
|---|---|---|---|
| Low | Intermediate | High | |
| Derivation cohort (n=92,071) | |||
| ≥2 Body system categories and number of health conditions | 14,209 (15.8) | 35,782 (39.7) | 40,182 (44.5) |
| ≥2 Body system categories and number of chronic health conditions | 13,245 (14.4) | 45,445 (49.4) | 33,379 (36.2) |
| Number of distinct body system categories | 11,695 (12.7) | 39,632 (43.1) | 40,730 (44.2) |
| Number of CCS categories | 8640 (9.4) | 42,472 (46.1) | 40,949 (44.5) |
| Number of health conditions | 11,083 (12.3) | 38,861 (43.1) | 40,229 (44.6) |
| Number of chronic health conditions | 5995 (6.5) | 52,609 (57.1) | 33,465 (36.4) |
| Deyo-Charlson Comorbidity Index | 64,369 (69.9)c | 27,701 (30.1) | |
| Elixhauser-van-Walraven Comorbidity Index | 6798 (7.4) | 43,256 (47.0) | 42,016 (45.6) |
| Validation cohort (n=55,735) | |||
| ≥2 Body system categories and number of health conditions | 3799 (6.9) | 18,702 (34.2) | 32,193 (58.9) |
| ≥2 Body system categories and number of chronic health conditions | 3613 (6.5) | 26,851 (48.2) | 25,267 (45.3) |
| Number of distinct body system categories | 2893 (5.2) | 20,349 (36.5) | 32,489 (58.3) |
| Number of CCS categories | 1940 (3.5) | 22,419 (40.2) | 31,371 (56.3) |
| Number of health conditions | 2721 (5.0) | 19,767 (36.1) | 32,206 (58.9) |
| Number of chronic health conditions | 1479 (2.7) | 28,931 (51.9) | 25,321 (45.4) |
| Deyo-Charlson Comorbidity Index | 38,209 (68.6)c | 17,525 (31.4) | |
| Elixhauser-Van-Walraven Comorbidity Index | 4483 (8.0) | 23,847 (42.8) | 27,404 (49.2) |
| Total cohort (N=147,806) | |||
| ≥2 Body system categories and number of health conditions | 18,008 (12.4) | 54,484 (37.6) | 72,375 (50.0) |
| ≥2 Body system categories and number of chronic health conditions | 16,858 (11.4) | 72,296 (48.9) | 58,650 (39.7) |
| Number of distinct body system categories | 14,588 (9.9) | 59,981 (40.6) | 73,233 (49.5) |
| Number of CCS categories | 10,580 (7.2) | 64,891 (43.9) | 72,320 (48.9) |
| Number of health conditions | 13,804 (9.5) | 58,628 (40.5) | 72,435 (50.0) |
| Number of chronic health conditions | 7474 (5.0) | 81,540 (55.2) | 58,790 (39.8) |
| Deyo-Charlson Comorbidity Index | 102,578 (69.4)c | 45,226 (30.6) | |
| Elixhauser-van-Walraven Comorbidity Index | 11,281 (7.6) | 67,103 (45.4) | 69,421 (47.0) |
CCS = Clinical Classification Software.
Data are presented as percentages of the cohort. The low-, intermediate-, and high-risk categories were defined using the cutoffs identified in Table 2. Patients with a number of items lower than the lower cutoff were classified at low risk, those with a number higher than or equal to the upper cutoff at high risk, and those with a number between the lower and the upper cutoffs at intermediate risk. For example, for the number of health conditions, 0 to 2 health conditions corresponds to low risk, 3 to 8 health conditions to intermediate risk, and 9 or more health conditions to high risk of multimorbidity.
For the Deyo-Charlson Comorbidity Index, there were only 2 risk categories, as we could not identify a lower cutoff.
Performance of the Different Definitions of Multimorbidity for 30-Day Readmission
In the derivation cohort, the AUC for 30-day readmission was 0.57 to 0.58 (95% CI, 0.56-0.61; Supplemental Table 2 and Supplemental Figure 1, available online at http://www.mcpiqojournal.org). In the validation cohort, the AUC varied between 0.57 (95% CI, 0.56-0.58) and 0.63 (95% CI, 0.62-0.63). The definition based on the Deyo-Charlson Comorbidity Index performed best (AUC, 0.63; 95% CI, 0.62-0.63). Only the definitions based on the number of CCS categories, health conditions, chronic health conditions, and Elixhauser-van-Walraven Comorbidity Index reached a sensitivity of 90% or higher using the lower cutoff in the derivation cohort, while all definitions, except the definition based on Deyo-Charlson Comorbidity Index (no lower cut-off), showed a sensitivity of 94% or higher in the validation cohort. For the upper cutoff, only the definitions based on the (1) number of chronic health conditions with 2 or more body system categories, (2) Deyo-Charlson Comorbidity Index, and (3) number of chronic health conditions in the derivation cohort and only the definition based on the Deyo-Charlson Comorbidity Index in the validation cohort reached the 60% or higher specificity threshold.
Performance of the Different Definitions of Multimorbidity for Prolonged LOS
The AUC for prolonged LOS varied between 0.60 (95% CI, 0.60-0.61) and 0.67 (95% CI, 0.67-0.68) in the derivation cohort and between 0.62 (95% CI, 0.61-0.62) and 0.76 (95% CI, 0.75-0.76) in the validation cohort (Supplemental Table 3 and Supplemental Figure 2, available online at http://www.mcpiqojournal.org). The definition based on the number of CCS categories performed best (AUC, 0.76; 95% CI, 0.75-0.76). All definitions showed a sensitivity of 92% or greater for the lowest cutoff in both the derivation and validation cohorts, except the definition based on Deyo-Charlson Comorbidity Index (no lower cut-off). For the upper cutoff, the specificity was 59% to 73% in the derivation cohort, while it varied more in the validation cohort (47%-72%).
Discussion
In a large multinational cohort, we found that 8 definitions of multimorbidity had poor to fair discriminatory power to identify patients with higher health care resource utilization. Four of these definitions performed similarly well and better than the other 4. To our knowledge, this is the first study comparing definitions of multimorbidity in relationship to health care resource utilization and identifying different cutoffs favoring sensitivity or specificity. Simple definitions performed equally well as more complex ones. The selection of a lower and an upper cutoff allowed classification of the patients into 3 risk categories of health care resource utilization.
Although both the WHO and NICE guidelines defined multimorbidity as 2 or more chronic health conditions, previous reviews found that a lack of standardization remains concerning which and how many conditions to include, particularly concerning the distinction between acute and chronic conditions.6,7 In our study, the lower cutoff, favoring sensitivity, varied between 1 or more and 3 or more, depending on the definition. As expected, it was higher for definitions based on all health conditions than for those using categorizing systems. This finding underlines the importance of clearly defining which health conditions to include in a definition before setting a specific cutoff.
Interestingly, the lower cutoff was 1 or more for the definition based on chronic health conditions, and not 2 or more as defined by the WHO and NICE guidelines.13,14 Considering a single condition may be self-contradictory with the concept of multimorbidity. However, this finding suggests that assessing only chronic health conditions may help improve sensitivity to identify multimorbidity. A definition using only chronic health conditions with a cutoff of 1 or more may thus be preferred when high sensitivity is most important but low specificity not an issue, such as for implementing simple, broadly available and cheap preventive interventions, which should reach all patients who possibly have multimorbidity. With this aim in mind and to allow study comparability, a rigorous differentiation between acute and chronic health conditions is required, but not consequently done.5,7 A standardized classification tool such as the CCI may be useful and minimize subjectivity.19
All lower cutoffs, which were comparable to those previously used to define multimorbidity, were characterized by a particularly poor specificity, whereas the upper cutoffs were far higher than usual cutoffs, up to 9 or more for all health conditions.6,7 This finding suggests that usual definitions of multimorbidity favor sensitivity over specificity in relationship to health care resource utilization and may thus not accurately identify patients with multimorbidity at higher risk of health care utilization. Higher cutoffs probably select patients with greater burden of multimorbidity requiring particular attention.5,7 Because the most effective preventive interventions to lower readmission rates are complex and intensive, using higher and more specific cutoffs may be useful to select patients most likely to benefit.27 A classification of patients into 3 risk categories may help to select patients for specific interventions.
Although the Deyo-Charlson Comorbidity Index remains often used to assess multimorbidity, it may not always be appropriate because of its poor sensitivity.6,28 Furthermore, because this index was developed and validated to predict mortality,15,28 it may not be valid to assess other health outcomes. Although previous studies found an association between the Elixhauser Comorbidity Index and health care resource utilization,29,30 its accuracy was rather poor in our cohort, suggesting that complex assessment of multimorbidity is not better than more simple measurements.
Because combining 2 different measures of multimorbidity may help to improve the accuracy, we tested definitions combining all or only chronic health conditions with 2 or more body systems involved. However, these definitions did not perform better than definitions without body system categories and had the same lower and upper cutoffs for the number of conditions. This finding suggests that making the definition more complex does not improve its accuracy, so more simple definitions may be preferred.
The 4 definitions that performed best for the primary outcome were also those that performed best for the secondary outcomes. However, the different definitions performed better to identify patients at higher risk of prolonged LOS than of readmission. Furthermore, the definition that performed best was different for these 2 outcomes (number of CCS categories for prolonged LOS and number of health categories for 30-day readmission), suggesting that some definitions are better than others for different outcomes. Therefore, a different definition may be preferred to identify patients at higher risk of prolonged LOS or of readmission.
Although different cutoffs and definitions may be used according to the purpose of an assessment, this may not apply to the evaluation of multimorbidity prevalence.6,7,28,31 In fact, heterogeneity in the types and number of conditions assessed (4 to 185 in previous studies) inevitably resulted in poorly comparable results, with reported prevalences of 23% to 99%.1, 2, 3,7,32,33 Further research and experts’ consensus is required to delineate a more uniform definition of multimorbidity for prevalence studies in particular. Doing so, the high sensitivity but poor specificity of common definitions should be challenged to avoid overestimating the prevalence of multimorbidity.
Our study had several limitations. First, using ICD codes is subject to coding quality, possibly leading to underreporting.34,35 However, ICD codes are used most often because they are the simplest and most standardized way to collect diagnoses. Second, Switzerland used ICD-10 codes, while the United States and Israel used ICD-9 codes; nevertheless, because the classification systems were available for ICD-9 and ICD-10, a significant impact is rather unlikely. Third, other relevant components of multimorbidity, such as psychological, social, and environmental factors,7,36 as well as differences in health care systems, may have impacted the effect of multimorbidity. However, such aspects are challenging to assess; a standardized definition to be used independently of these factors and across health care systems may thus be useful. Finally, we included only readmissions to the same hospital, so we may have missed readmissions to other hospitals.
Our study also has several strengths. First, we used 8 different definitions of multimorbidity, as well as standardized classification tools allowing reproducibility.18,19 Second, our primary outcome included both readmissions and prolonged LOS, allowing a more holistic assessment of health care resource utilization. Third, we identified lower and upper cutoffs that may be used depending on the purpose of using a definition of multimorbidity. Finally, we used a large and multinational cohort, increasing the generalizability of our findings.
Conclusion
In this study, we found that 8 definitions of multimorbidity had poor to fair discriminatory power, but 4 of them, based on ICD codes, performed similarly well and better than the other 4 to identify patients with higher health care resource utilization. Therefore, one may favor the use of the definition based on the number of health conditions only, because it is simple to apply as ICD codes are easily available. To allow comparability across studies, standardizing the number and types of conditions to include in the definition of multimorbidity is required when measuring its prevalence. However, when evaluating the relationship of multimorbidity with adverse health outcomes, the cutoff and definition chosen should be determined depending on the specific aim of the study.
Acknowledgments
The funding bodies had no role in the study design, data collection, analysis, and interpretation, decision to publish, or preparation of the submitted manuscript.
Footnotes
Grant Support: Dr Aubert was supported by research grants from the Swiss Society of General Internal Medicine Foundation and from the Clinical Trials Unit from Bern University, Bern, Switzerland. Dr Schnipper has received investigator-initiated grants from Mallinckrodt Pharmaceuticals and Portola Pharmaceuticals, Inc. Dr Donzé was funded by the Swiss National Science Foundation.
Potential Competing Interests: The authors report no competing interests.
Supplemental material can be found online at http://www.mcpiqojournal.org. Supplemental material attached to journal articles has not been edited, and the authors take responsibility for the accuracy of all data.
Supplemental Online Material
References
- 1.Clerencia-Sierra M., Calderón-Larrañaga A., Martínez-Velilla N., et al. Multimorbidity patterns in hospitalized older patients: associations among chronic diseases and geriatric syndromes. PLoS One. 2015;10(7):e0132909. doi: 10.1371/journal.pone.0132909. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Wong A., Boshuizen H.C., Schellevis F.G., Kommer G.J., Polder J.J. Longitudinal administrative data can be used to examine multimorbidity, provided false discoveries are controlled for. J Clin Epidemiol. 2011;64(10):1109–1117. doi: 10.1016/j.jclinepi.2010.12.011. [DOI] [PubMed] [Google Scholar]
- 3.Friedman B., Jiang H.J., Elixhauser A., Segal A. Hospital inpatient costs for adults with multiple chronic conditions. Med Care Res Rev. 2006;63(3):327–346. doi: 10.1177/1077558706287042. [DOI] [PubMed] [Google Scholar]
- 4.Schneider F., Kaplan V., Rodak R., Battegay E., Holzer B. Prevalence of multimorbidity in medical inpatients. Swiss Med Wkly. 2012;142:w13533. doi: 10.4414/smw.2012.13533. [DOI] [PubMed] [Google Scholar]
- 5.Fortin M., Stewart M., Poitras M.E., Almirall J., Maddocks H. A systematic review of prevalence studies on multimorbidity: toward a more uniform methodology. Ann Fam Med. 2012;10(2):142–151. doi: 10.1370/afm.1337. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Diederichs C., Berger K., Bartels D.B. The measurement of multiple chronic diseases—a systematic review on existing multimorbidity indices. J Gerontol A Biol Sci Med Sci. 2011;66(3):301–311. doi: 10.1093/gerona/glq208. [DOI] [PubMed] [Google Scholar]
- 7.Johnston M.C., Crilly M., Black C., Prescott G.J., Mercer S.W. Defining and measuring multimorbidity: a systematic review of systematic reviews. Eur J Public Health. 2019;29(1):182–189. doi: 10.1093/eurpub/cky098. [DOI] [PubMed] [Google Scholar]
- 8.van den Akker M., Buntinx F., Knottnerus J.A. Comorbidity or multimorbidity: what's in a name? a review of literature. Eur J Gen Pract. 1996;2(2):65–70. [Google Scholar]
- 9.Bähler C., Huber C.A., Brüngger B., Reich O. Multimorbidity, health care utilization and costs in an elderly community-dwelling population: a claims data based observational study. BMC Health Serv Res. 2015;15:23. doi: 10.1186/s12913-015-0698-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Centers for Medicare and Medicaid Services . Centers for Medicare and Medicaid Services; Baltimore, MD: 2012. Chronic Conditions Among Medicare Beneficiaries, Chartbook: 2012 Edition. [Google Scholar]
- 11.Hopman P., Heins M.J., Korevaar J.C., Rijken M., Schellevis F.G. Health care utilization of patients with multiple chronic diseases in the Netherlands: differences and underlying factors. Eur J Intern Med. 2016;35:44–50. doi: 10.1016/j.ejim.2016.08.025. [DOI] [PubMed] [Google Scholar]
- 12.Vogeli C., Shields A.E., Lee T.A., et al. Multiple chronic conditions: prevalence, health consequences, and implications for quality, care management, and costs. J Gen Intern Med. 2007;22(suppl 3):391–395. doi: 10.1007/s11606-007-0322-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.World Health Organization (2016). Multimorbidity. World Health Organization website. http://www.who.int/iris/handle/10665/252275 Published 2016. Accessed January 28, 2019.
- 14.National Institute for Health and Care Excellence Multimorbidity: clinical assessment and management. NICE guideline [NG56]. NICE website. https://www.nice.org.uk/guidance/ng56 Published September 2016. Accessed January 28, 2019.
- 15.Charlson M.E., Pompei P., Ales K.L., MacKenzie C.R. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 16.Donzé J.D., Williams M.V., Robinson E.J., et al. International validity of the HOSPITAL score to predict 30-day potentially avoidable hospital readmissions. JAMA Intern Med. 2016;176(4):496–502. doi: 10.1001/jamainternmed.2015.8462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.von Elm E., Altman D.G., Egger M., Pocock S.J., Gøtzsche P.C., Vandenbroucke J.P., STROBE Initiative The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg. 2014;12(12):1495–1499. doi: 10.1016/j.ijsu.2014.07.013. [DOI] [PubMed] [Google Scholar]
- 18.Healthcare Cost and Utilization Project, Agency for Healthcare Research and Quality, Clinical Classifications Software (CCS) for ICD-9-CM. HUCP website. https://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp Published March 2017. Accessed June 23, 2018.
- 19.Healthcare Cost and Utilization Project, Agency for Healthcare Research and Quality. Chronic Condition Indicator (CCI) for ICD-9-CM. HCUP website. https://www.hcup-us.ahrq.gov/toolssoftware/chronic/chronic.jsp Published May 2016. Accessed June 23, 2018. [PubMed]
- 20.Elixhauser A., Steiner C., Harris D.R., Coffey R.M. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27. doi: 10.1097/00005650-199801000-00004. [DOI] [PubMed] [Google Scholar]
- 21.Deyo R.A., Cherkin D.C., Ciol M.A. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619. doi: 10.1016/0895-4356(92)90133-8. [DOI] [PubMed] [Google Scholar]
- 22.van Walraven C., Austin P.C., Jennings A., Quan H., Forster A.J. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47(6):626–633. doi: 10.1097/MLR.0b013e31819432e5. [DOI] [PubMed] [Google Scholar]
- 23.Quan H., Sundararajan V., Halfon P., et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130–1139. doi: 10.1097/01.mlr.0000182534.19832.83. [DOI] [PubMed] [Google Scholar]
- 24.Weiss A., Elixhauser A. 2012. Healthcare Cost and Utilization Project - Agency for Healthcare Research and Quality. Overview of Hospital Stays in the United States. Statistical brief #180. 2014, https://www.hcup-us.ahrq.gov/reports/statbriefs/sb180-Hospitalizations-United-States-2012.pdf. [PubMed] [Google Scholar]
- 25.Thommen D., Weissenberger N., Schuetz P., et al. Head-to-head comparison of length of stay, patients' outcome and satisfaction in Switzerland before and after SwissDRG-Implementation in 2012 in 2012: an observational study in two tertiary university centers. Swiss Med Wkly. 2014;144:w13972. doi: 10.4414/smw.2014.13972. [DOI] [PubMed] [Google Scholar]
- 26.Pencina M.J., D'Agostino R.B., Sr. Evaluating discrimination of risk prediction models: the C statistic. JAMA. 2015;314(10):1063–1064. doi: 10.1001/jama.2015.11082. [DOI] [PubMed] [Google Scholar]
- 27.Leppin A.L., Gionfriddo M.R., Kessler M., et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7):1095–1107. doi: 10.1001/jamainternmed.2014.1608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.de Groot V., Beckerman H., Lankhorst G.J., Bouter L.M. How to measure comorbidity. a critical review of available methods. J Clin Epidemiol. 2003;56(3):221–229. doi: 10.1016/s0895-4356(02)00585-1. [DOI] [PubMed] [Google Scholar]
- 29.Menendez M.E., Neuhaus V., van Dijk C.N., Ring D. The Elixhauser comorbidity method outperforms the Charlson index in predicting inpatient death after orthopaedic surgery. Clin Orthop Relat Res. 2014;472(9):2878–2886. doi: 10.1007/s11999-014-3686-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Moore B.J., White S., Washington R., Coenen N., Elixhauser A. Identifying increased risk of readmission and in-hospital mortality using hospital administrative data: the AHRQ Elixhauser comorbidity index. Med Care. 2017;55(7):698–705. doi: 10.1097/MLR.0000000000000735. [DOI] [PubMed] [Google Scholar]
- 31.Huntley A.L., Johnson R., Purdy S., Valderas J.M., Salisbury C. Measures of multimorbidity and morbidity burden for use in primary care and community settings: a systematic review and guide. Ann Fam Med. 2012;10(2):134–141. doi: 10.1370/afm.1363. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Schellevis F.G., van der Velden J., van de Lisdonk E., van Eijk J.T., van Weel C. Comorbidity of chronic diseases in general practice. J Clin Epidemiol. 1993;46(5):469–473. doi: 10.1016/0895-4356(93)90024-u. [DOI] [PubMed] [Google Scholar]
- 33.Fortin M., Bravo G., Hudon C., Vanasse A., Lapointe L. Prevalence of multimorbidity among adults seen in family practice. Ann Fam Med. 2005;3(3):223–228. doi: 10.1370/afm.272. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Resslar M.A., Ivanitskaya L.V., Perez M.A., III, Zikos D. Sources of variability in hospital administrative data: clinical coding of postoperative ileus. Health Inf Manag. 2019;48(2):101–108. doi: 10.1177/1833358318781106. [DOI] [PubMed] [Google Scholar]
- 35.Romano P.S., Chan B.K., Schembri M.E., Rainwater J.A. Can administrative data be used to compare postoperative complication rates across hospitals? Med Care. 2002;40(10):856–867. doi: 10.1097/00005650-200210000-00004. [DOI] [PubMed] [Google Scholar]
- 36.Le Reste J.Y., Nabbe P., Manceau B., et al. The European General Practice Research Network presents a comprehensive definition of multimorbidity in family medicine and long term care, following a systematic review of relevant literature. J Am Med Dir Assoc. 2013;14(5):319–325. doi: 10.1016/j.jamda.2013.01.001. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.