Abstract
Background
Identifying patients at high risk of drug‐related hospital admission (DRA) may help to efficiently target preventive interventions. We developed a score to predict DRAs in older patients with multimorbidity and polypharmacy.
Methods
We used participants from the multicenter European OPERAM trial (“Optimising PharmacothERapy in the Mutlimorbid Elderly”). We assessed the association between easily identifiable predictors and 1‐year DRAs by univariable logistic regression. Variables with p‐value< 0.20 were taken forward to backward regression. We retained all variables with p < 0.05 in the model. We assessed the C‐statistic, calibration (observed/predicted proportions), and overall accuracy (scaled Brier score, <0.25 indicating a useful model) of the score, and internally validated it by tenfold cross‐validation.
Results
Within 1 year, 435/1879 (23.2%) patients (mean age 79.4 years) had a DRA. The score included seven variables: previous hospitalizations, non‐elective admission, hypertension, cirrhosis with portal hypertension, chronic kidney disease, diuretic, oral corticosteroid. The C‐statistic was 0.64 (95% CI 0.61–0.67). Patients with <1 point had a 12.4% predicted and observed risk of DRA, while those with >3 points had a 40.4% predicted and 38.9% observed risk of DRA. The scaled Brier score was 0.05. Calibration showed an adequate match between predicted and observed proportions.
Conclusion
Comorbidities related to drug metabolism, specific medications, non‐elective admission, and a history of hospitalization, were associated with a higher risk of DRA. Awareness of these associations and the score we developed may help identify patients most likely to benefit from preventive interventions.
Keywords: drug‐related admission, older adult, readmission, score
Key points
We developed and internally validated a score including seven readily available variables to predict DRAs in older hospitalized patients with multimorbidity and polypharmacy.
The score showed good overall accuracy and calibration and moderate discrimination.
Why does this paper matter?
The new score we developed and internally validated may help to identify patients at higher risk of DRA, which may help to focus preventive measures on the patients most likely to benefit.
INTRODUCTION
Due to multiple competing comorbidities and usually single‐disease‐focused guidelines, patients with multimorbidity frequently receive multiple medications. 1 , 2 , 3 This leads to an increased risk of drug‐related admissions (DRAs).
While several parameters associated with adverse drug events (ADEs) have been identified, 4 , 5 , 6 , 7 , 8 , 9 DRA‐specific risk factors have not been studied in depth. While a few models have been developed to predict ADEs, we are aware of only one score to predict DRAs in older adults: the PADR‐EC Score. 9 , 10 , 11 However, this score was developed in a cohort including only patients hospitalized in a medical department and did not focus on patients with multimorbidity and polypharmacy. Furthermore, the variable “drug changes in the preceding 3 months” may not be readily available, and it is not clear which medications were defined as “anticholinergics”.
To prevent DRAs, it is important to better understand DRA risk factors and identify patients at higher risk. In this study, we developed a model to predict 1‐year DRAs.
METHODS
Study design and population
We used the participants (≥70 years, ≥3 chronic conditions, ≥5 long‐term medications, 4 European countries) of the OPERAM trial, described previously. 12 , 13 The intervention did not significantly reduce 1‐year DRAs (primary outcome, HR 0.95, 95% confidence interval [CI] 0.77 to 1.17). 13 Participants who died during follow‐up were included in this analysis.
Variables of interest
For inclusion in the model, we considered variables previously associated with ADEs 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 14 and easily identifiable in clinical practice (so the model can be implemented straightforwardly), including (1) demographics (age, sex); (2) comorbidities included in the Charlson Comorbidity Index; 15 , 16 (3) other comorbidities: anemia, atrial fibrillation, hypertension, depression, according to International Classification of Diseases version 10 (ICD‐10) codes (Text S1) 17 ; anemia (ICD‐10 code for acute or chronic anemia and/or admission hemoglobin <120 g/L [women] and < 130 g/L [men]) 18 ; (4) number of individual brand products and specific medication classes at admission (Text S2); (5) hospitalizations within 1 year and ER visits within 6 months before admission; (6) index hospitalization characteristics (urgent/elective, surgical/medical ward, length of stay, discharge destination). Although hyperlipidemia was associated with DRAs in the BADRI model, this variable was missing in most patients and thus not considered for inclusion (reflecting also variable availability in real‐life practice). 11
Outcome
The outcome of interest was the first 1‐year DRA, defined as a hospitalization >24 h, with an ADE as a main or contributory reason for admission, including harm due to non‐preventable adverse drug reactions and preventable medication errors related to over‐, mis‐, and underuse, or over‐, mis‐ and underprescribing. 5 , 19 , 20 DRAs were assessed by an independent blinded and trained adjudication committee, following specific adjudication guidelines. 21
Statistical analyses
Firstly, we assessed the association between each predictor and DRA by logistic regression adjusted for the intervention group because the OPERAM trial intervention aimed at reducing DRAs. All variables with a p‐value <0.20 (to avoid excluding potentially useful predictors by setting a lower significance level) 9 , 10 , 11 , 22 were taken forward to a multivariable backward logistic regression. We retained only variables statistically significant at a p‐value <0.05 in the final model. We then attributed a number of points to each variable by dividing its coefficient by the lowest coefficient (absolute value). To simplify interpretation, we defined two risk categories: lower (number of points < upper quartile), and higher (number of points > upper quartile). We assessed the following performance parameters: (1) Discrimination, using the C‐statistic with bootstrapping with 1000 replications to compute the 95% confidence interval (CI). We internally validated the C‐statistic using ten‐fold cross‐validation. 23 (2) Calibration, comparing predicted and observed risks for each number of points, and for the lower‐risk and higher‐risk categories, and evaluating statistical significance with the Hosmer‐Lemeshow goodness‐of‐fit C‐statistic (significant p‐value indicating overall lack of fit). 24 (3) Overall accuracy of the model using the scaled Brier score (the lower the better; <0.25 indicating a useful model). 24 , 25 We performed all analyses using Stata/MP 16.0 (StataCorp LP, College Station, Texas).
RESULTS
Baseline characteristics
In the OPERAM trial, 119 patients withdrew consent, and 10 were lost of follow‐up, yielding a sample of 1879 patients with data on 1‐year DRA. Baseline characteristics according to the presence of DRA are described in Table 1. Within 1 year, 375 (20.0%) of the patients died, among which 101 (26.9%) with a previous DRA.
TABLE 1.
Characteristic | DRA (N = 435) | No DRA (N = 1444) |
---|---|---|
Age, years | 79.8 (6.5) | 79.3 (6.2) |
Female | 179 (41.2) | 656 (45.4) |
Site of inclusion | ||
Belgium | 73 (16.8) | 265 (18.4) |
Ireland | 77 (17.7) | 253 (17.5) |
The Netherlands | 85 (19.5) | 321 (22.2) |
Switzerland | 200 (46.0) | 605 (41.9) |
Chronic comorbidities, number | 4.7 (2.2) | 4.1 (2.1) |
Chronic medications, number | 11.1 (4.6) | 9.9 (4.1) |
≥1 hospitalization <1 year | 267 (61.4) | 697 (48.3) |
Non‐elective admission | 360 (82.8) | 1078 (74.7) |
Length of stay, days | 11.5 (10.4) | 12.2 (15.1) |
Surgical ward | 49 (11.3) | 254 (17.6) |
Comorbidities | ||
Anemia | 282 (64.8) | 832 (57.6) |
Atrial fibrillation | 108 (43.2) | 498 (34.5) |
Congestive heart failure | 149 (34.3) | 362 (25.1) |
Chronic kidney disease | 157 (36.1) | 390 (27.0) |
Stroke | 126 (29.0) | 415 (28.7) |
Diabetes | 111 (25.5) | 309 (21.4) |
Chronic pulmonary disease | 140 (32.2) | 364 (25.2) |
Hypertension | 283 (65.1) | 1040 (72.0) |
Cirrhosis with portal hypertension | 9 (2.1) | 9 (0.6) |
Medications | ||
Antihypertensive | 374 (86.0) | 1197 (82.9) |
Antithrombotic a | 359 (82.5) | 1112 (77.0) |
Proton pump inhibitor | 264 (60.7) | 806 (55.8) |
Psychotropic b | 181 (41.6) | 545 (37.7) |
Antidiabetic | 108 (24.8) | 306 (21.2) |
Diuretic | 247 (56.8) | 690 (47.8) |
Oral corticosteroid | 86 (19.8) | 168 (66.1) |
Note: Categorical variables are presented as n (%), and continuous variables as mean with standard deviation.
Including both antiplatelets and anticoagulants.
Including hypnotics, antidepressants, antipsychotics.
Drug‐related admission rates and predictors
Within 1 year after discharge, 435 (23.2%) of the 1879 patients had a DRA. In univariable analysis, the number of hospitalizations in the year before index admission, non‐elective admission type, admission to a medical ward, the number of comorbidities, congestive heart failure, atrial fibrillation, chronic kidney disease (CKD), CKD stage 3, cirrhosis with portal hypertension, anemia, blood malignancy, chronic pulmonary disease, the number of medications, diuretic, anticoagulants (not including antiplatelets), and oral corticosteroids were associated with a statistically significantly increased risk of DRA (Table S1). Hypertension, calcium channel blocker, and non‐steroidal anti‐inflammatory drugs were associated with a statistically significantly reduced risk of DRA. Nine additional variables were associated with DRA at a p‐value between 0.05 and 0.20, and thus also selected for entering into the backward regression analysis.
Score to predict drug‐related admissions
The final model included seven variables (Table 2). The coefficient for hypertension was the smallest in absolute value and thus used to compute the score points. The score ranged from −1 to 12 points. The mean number of points in the cohort was 1.9 (SD 1.7), with a median of 2 points. The upper (75%) quartile threshold was at 3 points, so the lower‐risk category was defined as <3 points, and the higher‐risk category as ≥3 points. A DRA was observed in 194/573 (33.9%) higher‐risk patients and 241/1306 (18.4%) lower‐risk patients (Figure 1A).
TABLE 2.
Variables | OR (95%CI) | Coefficient (SE) | p‐value | Points |
---|---|---|---|---|
Hospitalizations <365 days, n | ||||
0 | Reference | Reference | Reference | 0 |
1–2 | 1.46 (1.15–1.86) | 0.38 (0.12) | 0.001 | 1 |
≥3 | 2.43 (1.74–3.38) | 0.89 (0.17) | <0.001 | 3 |
Non‐elective admission | 1.59 (1.20–2.11) | 0.46 (0.14) | 0.001 | 1 |
Hypertension | 0.73 (0.57–0.92) | −0.32 (0.12) | 0.009 | −1 |
Chronic kidney disease | 1.41 (1.11–1.79) | 0.34 (0.12) | 0.005 | 1 |
Cirrhosis with portal hypertension | 3.61 (1.38–9.42) | 1.28 (0.49) | 0.009 | 4 |
Diuretic | 1.36 (1.09–1.71) | 0.31 (0.12) | 0.006 | 1 |
Oral corticosteroid | 1.68 (1.25–2.26) | 0.52 (0.15) | 0.001 | 2 |
Score range | — | — | — | −1 to 12 |
Note: Variables with a p‐value <0.20 in univariable logistic regression were used for backward selection in the multivariable model, with a p‐value set at <0.05 for maintenance in the model. The model was adjusted for the intervention vs. control arm of the OPERAM trial, since the intervention aimed at reducing drug‐related admissions. The number of points for the score were attributed by dividing the coefficient by the lowest coefficient.
Abbreviations: CI, confidence interval; OR, odds ratio; SE, standard error.
The C‐statistic was 0.64 (95% CI 0.61–0.67). Ten‐fold cross‐validation showed good internal validity, with a C‐statistic of 0.63 (95% CI 0.59–0.66). Calibration was good, as depicted by the well‐matching predicted and observed risks of DRAs (Figure 1), and a Hosmer‐Lemeshow goodness‐of‐fit p‐value of 0.82 (leading to rejecting the null hypothesis of lack of fit). Patients with <1 point had a 12.4%‐predicted and observed risk of 1‐year DRA, while those with >3 points had a 40.4%‐predicted and 38.9%‐observed risk of DRA. The overall accuracy of the score was good, with a scaled Brier score of 0.05.
DISCUSSION
In this multicenter European study, CKD, cirrhosis with portal hypertension, diuretics, oral corticosteroids, non‐elective admission, and the number of hospitalizations within the year before admission, were independent predictors of 1‐year DRAs in older adults with multimorbidity and polypharmacy. The score had moderate discriminatory power, but was very well calibrated and showed excellent overall accuracy. The seven included variables are readily extractable from medical records, providing a simple tool that could be easily implemented in clinical practice during hospitalization, to target DRA preventive interventions at patients most likely to benefit, with potentially important and positive implications to reduce DRA burden.
Given their importance in drug metabolism, renal and liver dysfunction would be expected to increase the risk of ADEs, as previously documented. 10 , 14 , 26 , 27 Liver disease was also included as a significant predictor in the GerontoNet ADR Risk Score, and CKD in both the GerontoNet ADR Risk Score and the PADR‐EC Score, previously developed in older hospitalized adults. 10 , 14 This emphasizes the importance of monitoring renal function to adjust prescriptions accordingly. Cirrhosis with portal hypertension is less frequent than CKD but was most strongly associated with DRAs. This may reflect a lack of clear guidance on how best to adjust medication regimens in liver disease, in contrast with well‐developed recommendations in CKD.
Diuretics are frequently started or their dosage modified in hospital, and have been identified as a major cause of ADE and preventable DRA. 6 , 8 , 14 While patients are bed‐bound for about 80% of hospital stay, symptoms related to hypovolemia and orthostatic hypotension may not become apparent during hospitalization. 28 To avoid adverse consequences after discharge, such symptoms should be carefully assessed during hospital stay and medications adequately adapted.
Systemic corticosteroids were previously associated with ADEs in previous studies, 8 , 14 and may also trigger decompensation of pre‐existing comorbidities such as hypertension, diabetes and heart failure. It is important to restrict their use as much as possible, to closely monitor for ADEs, and provide co‐medications as indicated. 29
Hypertension was negatively associated with DRAs. This was unexpected since hypertension is frequently treated with multiple medications that have a high potential of ADE. This association remained significant after removing diuretics from the multivariable model or after adjusting for medication count, suggesting that low‐dose diuretics used for hypertension are not associated with DRA, however, diuretics at a higher dose for heart failure are more frequently associated with DRA.
The discrimination of our score was moderate and worse than for the PADR‐ED Score (C‐statistic 0.70). 10 However, solely relying on discrimination to assess the usefulness of a predictive model is not advisable, since some important risk factors may have only a marginal impact on the C‐statistic, and a perfectly calibrated model may have a C‐statistic as low as 0.63. 30 Our score showed very good calibration, and excellent overall accuracy, and has several advantages. Firstly, it does not include laboratory parameters not routinely measured at the hospital, unlike the BADRI model for example. 11 Secondly, it does not require a medication history, as in the PADR‐EC Score, 10 which may require contacting the primary care provider or the dispensing pharmacy. Thirdly, our score includes only three commonly‐assessed comorbidities, and not relatively ill‐defined conditions such as dementia as in the PADR‐EC Score. 10 Finally, the medications considered in the score are easily and clearly defined unlike in the PADR‐EC Score (not clear what should be defined as anticholinergic). 10 We thus provide a tool that could be broadly and easily implemented in real‐life clinical practice.
This new score has potentially important implications. First, it could help for better risk stratification in future clinical trials testing an intervention to prevent DRAs. Second, it could be applied to identify the differential effects of such interventions tested in past trials. Third, it could be used for benchmarking to compare the predicted versus observed DRAs across health systems.
Limitations and strengths
This study has some limitations. Firstly, ICD codes may be subject to underreporting or overcoding, and are available only after discharge; however, the diagnoses included in the score are easily identifiable without extended coding, so we think it does not prevent using the score during hospitalization. Secondly, the OPERAM trial included only patients aged ≥70 years with multimorbidity and polypharmacy, so our findings may not be generalizable to younger or healthier patients. However, those patients represent the most vulnerable population for ADEs and DRAs and a large proportion of hospitalized patients. Thirdly, medications causing the DRAs were not specifically adjudicated, as most patients had multiple medications that might have contributed to DRAs. Fourthly, our model could not account for post‐discharge factors that may have influenced DRA risk. Fifthly, we did not assess its performance in different subpopulations (e.g., medical vs. surgical). Sixthly, the 1‐year time horizon might explain a lower accuracy of the score. Finally, the OPERAM trial intervention aimed at reducing DRAs, so that DRA risk may have differed between patients in the intervention and control arms. However, this is unlikely because the OPERAM intervention did not significantly reduce 1‐year DRAs, 13 and we adjusted for the study arm (variable not significant in any model).
This study has some significant strengths. Firstly, DRAs were adjudicated by a blinded trained committee. Secondly, all data were collected prospectively and systematically according to a predefined protocol. Thirdly, the sample size and number of measured outcomes allowed for the assessment of multiple predictors. Finally, our sample included medical and surgical patients, increasing the generalizability of our results.
Conclusion
In multimorbid older patients with polypharmacy, comorbidities affecting drug metabolism, diuretics, oral corticosteroids, non‐elective admission, and the number of previous hospitalizations, were significantly associated with a higher risk of 1‐year DRA. Our findings should raise prescriber awareness for patients with chronic renal or liver disease, as well as for those receiving high‐risk medications, such as diuretics and systemic corticosteroids. The simple DRA prediction score we developed, which showed very good calibration and excellent overall accuracy, could be easily implemented in clinical practice during hospitalization as a support to identify higher‐risk patients who may most benefit from preventive interventions to reduce the risk of DRAs.
CONFLICT OF INTEREST
The author declares that there is no conflict of interest that could be perceived as prejudicing the impartiality of the research reported.
AUTHOR CONTRIBUTIONS
Conception and design of the study: Carole E. Aubert, Jacques Donzé. Data acquisition: Nicolas Rodondi, Olivia Dalleur, Anne Spinewine, Clara Drenth‐van Maanen, Wilma Knol, Denis O'Mahony. Analysis and interpretation of data: Carole E. Aubert, Jacques Donzé. Manuscript drafting: Carole E. Aubert. Revising the manuscript critically for important intellectual content: Nicolas Rodondi, Seraina Netzer, Olivia Dalleur, Anne Spinewine, Clara Drenth‐van Maanen, Wilma Knol, Denis O'Mahony, Drahomir Aujesky, Jacques Donzé. Approval of the version of the manuscript to be published: All authors.
SPONSOR'S ROLE
The Sponsor had no role in the conception/design of the study; data acquisition; analysis and interpretation of data; manuscript drafting; manuscript revision; approval of the manuscript to be published; decision to submit the manuscript.
Supporting information
ACKNOWLEDGMENT
Open access funding provided by Universitat Bern.
Aubert CE, Rodondi N, Netzer S, et al. Predictors of 1‐year drug‐related admissions in older multimorbid hospitalized adults. J Am Geriatr Soc. 2022;70(5):1510‐1516. doi: 10.1111/jgs.17667
Funding informationDr. Aubert was supported by an Early Postdoc. Mobility grant from the Swiss National Science Foundation (grant P2LAP3_184042). Prof Donzé was funded by the Swiss National Science Foundation. This work is part of the project “OPERAM: OPtimising thERapy to prevent Avoidable hospital admissions in the Multimorbid elderly” supported by the European Union's Horizon 2020 research and innovation program under the grant agreement No 6342388, and by the Swiss State Secretariat for Education, Research and Innovation (SERI) under contract number 15.0137. The OPERAM study is also funded by the Swiss National Scientific Foundation (SNSF 320030_188549). The opinions expressed and arguments employed herein are those of the authors and do not necessarily reflect the official views of the EC and the Swiss government. The funders had no roles in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
REFERENCES
- 1. Hughes LD, McMurdo ME, Guthrie B. Guidelines for people not for diseases: the challenges of applying UKclinical guidelines to people with multimorbidity. Age Ageing. 2013;42:62‐69. [DOI] [PubMed] [Google Scholar]
- 2. Leendertse AJ, Egberts AC, Stoker LJ, van den Bemt PM. Frequency of and risk factors for preventable medication‐related hospital admissions in The Netherlands. Arch Intern Med. 2008;168:1890‐1896. [DOI] [PubMed] [Google Scholar]
- 3. Aubert CE, Streit S, Da Costa BR, et al. Polypharmacy and specific comorbidities in university primary care settings. Eur J Intern Med. 2016;35:35‐42. [DOI] [PubMed] [Google Scholar]
- 4. Field TS, Gurwitz JH, Avorn J, et al. Risk factors for adverse drug events among nursing home residents. Arch Intern Med. 2001;161:1629‐1634. [DOI] [PubMed] [Google Scholar]
- 5. Krähenbühl‐Melcher A, Schlienger R, Lampert M, Haschke M, Drewe J, Krähenbühl S. Drug‐related problems in hospitals: a review of the recent literature. Drug Saf. 2007;30:379‐407. [DOI] [PubMed] [Google Scholar]
- 6. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. ADE prevention study group. Arch Intern Med. 1999;159:2553‐2560. [DOI] [PubMed] [Google Scholar]
- 7. Wawruch M, Zikavska M, Wsolova L, et al. Adverse drug reactions related to hospital admission in Slovak elderly patients. Arch Gerontol Geriatr. 2009;48:186‐190. [DOI] [PubMed] [Google Scholar]
- 8. Chan M, Nicklason F, Vial JH. Adverse drug events as a cause of hospital admission in the elderly. Intern Med J. 2001;31:199‐205. [DOI] [PubMed] [Google Scholar]
- 9. Onder G, Petrovic M, Tangiisuran B, et al. Development and validation of a score to assess risk of adverse drug reactions among in‐hospital patients 65 years or older: the GerontoNet ADR risk score. Arch Intern Med. 2010;170:1142‐1148. [DOI] [PubMed] [Google Scholar]
- 10. Parameswaran Nair N, Chalmers L, Connolly M, et al. Prediction of hospitalization due to adverse drug reactions in elderly community‐dwelling patients (the PADR‐EC score). PLoS One. 2016;11:e0165757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Tangiisuran B, Scutt G, Stevenson J, et al. Development and validation of a risk model for predicting adverse drug reactions in older people during hospital stay: Brighton adverse drug reactions risk (BADRI) model. PLoS One. 2014;9:e111254. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Adam L, Moutzouri E, Baumgartner C, et al. Rationale and design of OPtimising thERapy to prevent avoidable hospital admissions in multimorbid older people (OPERAM): a cluster randomised controlled trial. BMJ Open. 2019;9:e026769. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Blum MR, Sallevelt B, Spinewine A, et al. Optimizing therapy to prevent avoidable hospital admissions in multimorbid older adults (OPERAM): cluster randomised controlled trial. BMJ. 2021;374:n1585. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Onder G, Pedone C, Landi F, et al. Adverse drug reactions as cause of hospital admissions: results from the Italian Group of Pharmacoepidemiology in the elderly (GIFA). J Am Geriatr Soc. 2002;50:1962‐1968. [DOI] [PubMed] [Google Scholar]
- 15. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373‐383. [DOI] [PubMed] [Google Scholar]
- 16. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD‐9‐CM and ICD‐10 administrative data. Med Care. 2005;43:1130‐1139. [DOI] [PubMed] [Google Scholar]
- 17. German institute for medical documentation and information (DIMDI) . International statistic classificaiton of diseases and associated health problems 10th revision german modification. https://www.dimdi.de/static/de/klassifikationen/icd/icd-10-gm/kode-suche/htmlgm2020/. Accessed October 31 2020.
- 18. World Health Organization . Haemoglobin concentrations for the diagnosis of anaemia and assessment of severity. Vitamin and Mineral Nutrition Information System. Geneva, World Health Organization, 2011 (WHO/NMH/NHD/MNM/11.1) http://www.who.int/vmnis/indicators/haemoglobin. Accessed October 31 2020.
- 19. Edwards IR, Aronson JK. Adverse drug reactions: definitions, diagnosis, and management. Lancet. 2000;356:1255‐1259. [DOI] [PubMed] [Google Scholar]
- 20. van den Bemt PM, Egberts TC, de Jong‐van den Berg LT, Brouwers JR. Drug‐related problems in hospitalised patients. Drug Saf. 2000;22:321‐333. [DOI] [PubMed] [Google Scholar]
- 21. Thevelin S, Spinewine A, Beuscart JB, et al. Development of a standardized chart review method to identify drug‐related hospital admissions in older people. Br J Clin Pharmacol. 2018;84:2600‐2614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. In Lee K, Koval JJ. Determination of the best significance level in forward stepwise logistic regression. Commun Statis—Simul Comput. 1997;26(2):559‐575. [Google Scholar]
- 23. LeDell E, Petersen M, van der Laan M. Computationally efficient confidence intervals for cross‐validated area under the ROC curve estimates. Electron J Stat. 2015;9:1583‐1607. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128‐138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Arkes HR, Dawson NV, Speroff T, et al. The covariance decomposition of the probability score and its use in evaluating prognostic estimates. SUPPORT investigators. Med Decis Mak Int J Soc Med Decis Mak. 1995;15:120‐131. [DOI] [PubMed] [Google Scholar]
- 26. Helldén A, Bergman U, von Euler M, Hentschke M, Odar‐Cederlöf I, Ohlén G. Adverse drug reactions and impaired renal function in elderly patients admitted to the emergency department: a retrospective study. Drugs Aging. 2009;26:595‐606. [DOI] [PubMed] [Google Scholar]
- 27. Corsonello A, Pedone C, Corica F, Mussi C, Carbonin P, Antonelli IR. Concealed renal insufficiency and adverse drug reactions in elderly hospitalized patients. Arch Intern Med. 2005;165:790‐795. [DOI] [PubMed] [Google Scholar]
- 28. Brown CJ, Redden DT, Flood KL, Allman RM. The underrecognized epidemic of low mobility during hospitalization of older adults. J Am Geriatr Soc. 2009;57:1660‐1665. [DOI] [PubMed] [Google Scholar]
- 29. Mundell L, Lindemann R, Douglas J. Monitoring long‐term oral corticosteroids. BMJ Open Qual. 2017;6:e000209. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115:928‐935. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.