Abstract
Introduction
Falling on level ground is now the most common cause of traumatic intracranial bleeding worldwide. Older adults frequently present to the emergency department (ED) after falling. It can be challenging for clinicians to determine who requires brain imaging to rule out traumatic intracranial bleeding, and often head injury decision rules do not apply to older adults who fall. The goal of our study is to derive a clinical decision rule, which will identify older adults who present to the ED after a fall who do not have clinically important intracranial bleeding.
Methods and analysis
This is a prospective cohort study enrolling patients aged 65 years or older, who present to the ED of 11 hospitals in Canada and the USA within 48 hours of having a fall. Patients are included if they fall on level ground, off a chair, toilet seat or out of bed. The primary outcome is the diagnosis of clinically important intracranial bleeding within 42 days of the index ED visit. An independent adjudication committee will determine the primary outcome, blinded to all other data. We are collecting data on 17 potential predictor variables. The treating physician completes a study data form at the time of initial assessment, prior to brain imaging. Data extraction is supplemented by an independent, structured electronic medical record review. We will perform binary recursive partitioning using Classification and Regression Trees to derive a clinical decision rule.
Ethics and dissemination
The study was initially approved by the Hamilton Integrated Research Ethics Committee and subsequently approved by the research ethics boards governing all participating sites. We will disseminate our results by journal publication, presentation at international meetings and social media.
Trial registration number
Keywords: accident & emergency medicine, trauma management, diagnostic radiology, geriatric medicine
Strengths and limitations of this study.
This cohort study aims to derive a clinical decision rule, which identifies older adults at risk of intracranial bleeding after a fall.
This is a large study enrolling patients from 11 hospitals in two countries.
Potential predictor variables are recorded by emergency physicians prior to CT scanning.
The primary outcome, clinically important intracranial bleeding, is determined by an independent adjudication committee.
The main limitation is that not all patients will have CT imaging of the head at their initial emergency department visit.
Introduction
In contrast to the younger population, the incidence of traumatic intracranial bleeding in older adults is rising1 and has a worse prognosis.2 3 Older adults are at higher risk of traumatic intracranial bleeding because there can be loss of the elastic integrity of the cerebral bridging veins and brain atrophy, allowing rapid movements of the brain within the cerebral spinal fluid with trauma. Older adults may be less able to withstand intracranial bleeding because of pre-existing comorbidity, frailty and polypharmacy.
Falling on level ground is now the most common cause of traumatic intracranial bleeding worldwide, accounting for up to 80% of cases.4–8 Fall-associated intracranial bleeding in older adults is increasing in incidence.9 10 The mortality rate for fall-associated intracranial bleeding is 15%7 11 (accounting for half of all fall-associated deaths).12 13 Rather than seeing a decrease in these deaths, this mortality rate is rising.10 Emergency departments (EDs) are managing an increasing number of older adults who have fallen14 and ED visits for fall-related head injuries in older adults have increased year after year.9 13 15–17 There is a paucity of evidence to guide neuroimaging for intracranial bleeding in older adults.
The Canadian CT Head Rule can determine the need for head CT in head-injured patients who experienced loss of consciousness, disorientation or amnesia after their injury.18 However, older ED patients who present after a fall cannot always give a history of what happened, falls are frequently unwitnessed and many older adults who fall do not sustain a head injury. Ordering a CT scan of the head of every older adult who has fallen would be an inefficient and costly way to diagnose intracranial bleeding when only approximately 5% have intracranial bleeding.19 Patients awaiting a CT scan will typically occupy an ED bed. CT overuse in this population not only causes prolonged ED visits, but it also contributes to ED overcrowding, which may result in worse outcomes for other patients.20 Older adults are at greater risk of developing delirium the longer they stay in the ED.21 There is a need for a simple bedside tool that can rapidly stratify the risk of intracranial bleeding in older ED patients who present after falling. Our aim is to derive a clinical decision rule that will identify older adults who present to the ED after a fall who do not have clinically important intracranial bleeding, and therefore do not require a CT of the head.
Methods and analysis
Study design
This is a prospective cohort study designed to develop a unique clinical decision rule for ED physicians evaluating older adults who have fallen. Clinical decision rules are a commonly applied method of clinical diagnostic decision-making in the ED. The rules incorporate the standardised collection and interpretation of multiple predictor variables from the patient’s history, physical examination and test results to optimise evidence-based clinical decision-making. For example, clinical decision rules are used to determine which patients should have cervical spine imaging in trauma,22 thoracic imaging for pulmonary embolism23 and admission after syncope.24 Our study follows the methodological standards for clinical decision rules in emergency medicine25 and the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis guidelines.26
Patient and public involvement
Prior to the protocol development, we conducted a qualitative study with older adults who were waiting in the ED for CT of the head after a fall. We found that diagnosing intracranial bleeding was important to the participants, that they valued testing tailored to their personal risk and shorter ED visits. This protocol was designed with feedback and input from our patient partners.
Study population
This study is conducted at 11 hospitals in Canada and the United States and enrolls patients aged 65 years or older who present to the ED within 48 hours of having a fall. Patients are eligible if they fall on level ground (either inside or outside), off a chair, toilet seat or out of bed. Patients are included regardless of whether they hit their head. Patients are excluded if they fell down steps, fell from a height, were knocked down by a car/bike/pedestrian or other mechanism of injury. Patients who live outside the hospital catchment area, who have previously been enrolled in this study, who are transferred from another hospital and who leave the ED prior to completion of their medical assessment are also excluded. Recruitment commenced on 30 January 2019. Patients are recruited 24 hours a day, 7 days a week.
Patient assessment
Each patient is assessed at their index ED visit by an emergency physician who decides on the need for CT of the head based on clinical history and examination. It would be impractical to perform a CT of the head on all older adults who have fallen, for example, after a simple trip, because there is not always an indication for CT, hospitals have limited resources and ordering a CT delays discharge home. However, if participants return to the ED within 42 days of enrolment with new confusion, headache, loss of balance, repeat falls, change in behaviour, reduced Glasgow Coma Score (GCS) or other neurological symptoms, they will undergo CT of the head.
Outcome definition and measurement
The primary outcome is ‘clinically important intracranial bleeding’ diagnosed within 42 days of the index ED presentation. Our definition was derived after surveying specialists (including neurosurgeons, neurologists, trauma physicians, geriatricians, thrombosis and emergency physicians) who determined that symptoms from intracranial bleeding might develop as late as 6 weeks after a fall. ‘Clinically important intracranial bleeding’ is defined as bleeding within the cranial vault (including subdural, intracerebral, intraventricular, subarachnoid, epidural blood and cerebral contusion), which requires medical or surgical treatment. Medical treatment is defined as any of the following: temporary or permanent discontinuation of anticoagulant or antiplatelet medication; administration of an antifibrinolytic drug; reversal of anticoagulation; or admission to hospital for neurological observation. Clinically important intracranial bleeding will be determined by independent adjudication of CT scans of the head by the centralised outcome adjudication committee consisting of a study neurologist, neurosurgeon, trauma surgeon and radiologist. The adjudicators will be blinded to all ED baseline data. Each scan will be adjudicated independently by two reviewers. In the case of a disagreement, a third adjudicator, blinded to the prior reviews, will determine the classification. Agreement between the adjudicators will be reported. Secondary outcomes relate to the ‘severity’ of the intracranial bleeding: (1) neurosurgical intervention; (2) intensive care admission; (3) hospital length of stay; (4) in-hospital death as determined by medical record review.
We found poor sensitivity (37%, 95% CI: 21% to 56%) for patient-reported diagnosis of intracranial bleeding.27 Furthermore, our experience of personal follow-up in this population28 is that it is frequently not feasible because of residence in nursing homes or baseline cognitive impairment. Therefore, the current study follow-up is restricted to systematic medical record review with independent validation and enrolment is restricted to patients who reside within the hospital catchment area.
Predictor variables
Demographic and predictor variables are collected in two ways: (1) the treating physician completes a standardised data collection form at the time of initial patient assessment, and before the results of the CT of the head are available (therefore blinded to outcome); (2) data are collected by trained on-site research assistants using standardised medical record review protocols, following detailed data definitions and instructions for systematic medical record review. We follow standardised validation procedures for all medical record review data points: deidentified source documentation is uploaded for validation by the coordinating centre. A query is sent to the site research assistant to resolve each discrepancy. The study site investigator resolves discrepancies which persist after research assistant review. Table 1 details the demographic and predictor variables collected.
Table 1.
Predictor variables | Data collected by treating physician at initial assessment | Data collected by medical record review | Comment on predictor choice for rule derivation |
Age | x | No association found* but will be included | |
Sex | x | Trend towards association with male sex* | |
Head injury (as reported by patient or carer) | x | Plausible higher risk | |
Loss of consciousness | x | Marker for head injury severity | |
New amnesia about events of fall | x | Marker for head injury severity | |
History of previous major bleed28 | x | Trend towards association* and biologically plausible | |
Cirrhosis | x | Biologically plausible | |
Previous diagnosis of ischaemic stroke | x | Biologically plausible | |
Chronic renal impairment | x | x | Association demonstrated* |
Reduced Glasgow Coma Score from normal (as indicated by caregiver or family) | x | Association demonstrated* | |
Bruise or laceration on the head (any size) | x | Association demonstrated* | |
New abnormality on neurological examination | x | Association demonstrated * | |
Haemoglobin | x | Biologically plausible | |
Platelet count | x | Biologically plausible | |
Anticoagulation medication | x | x | Commonly held dogma |
Antiplatelet medication | x | x | Commonly held dogma |
Clinical Frailty Score30 | x | Biologically plausible | |
Descriptive variables | |||
Living circumstances | x | No association found* | |
Diabetes | x | No association found* | |
Hypertension | x | No association found* | |
Active cancer within past 2 years | x | No association found* | |
Dementia | x | No association found* | |
History of frequent falls | x | Not previously assessed* | |
Congestive heart failure | x | No association found* | |
Mechanism of injury | x | No association found* | |
Weight | x | No association found* | |
Glasgow Coma Score at time of physician assessment | x | Reduced Glasgow Coma Score from normal has a stronger association* | |
Vomiting (once/more than once) | x | No association found* | |
Signs of basal skull fracture | x | Too rare to assess* | |
Suspected open or depressed skull fracture | x | Too rare to assess* | |
Retrograde amnesia for >30 min | x | Not previously assessed* | |
Creatinine | x | No association found* | |
International normalised ratio | x | Anticipated missing data |
*According to the results of our prior study,28 N=1753.
We initially identified potential predictor variables by a systematic review of prior evidence. We then assessed the frequency among our population and the association between predictor and intracranial bleeding in a study of 1753 older ED patients who had fallen.28 We selected 17 candidate predictor variables, which are considered to be biologically plausible and related to the outcome of intracranial bleeding, and are routinely collected in the ED: age; sex; head injury; loss of consciousness; amnesia; history of previous major bleed (International Society of Thrombosis and Haemostasis criteria)29; cirrhosis; prior ischaemic stroke; chronic renal impairment; GCS reduced from baseline; bruise or laceration on the head; abnormal neurological examination; haemoglobin, platelet count; anticoagulant therapy; antiplatelet therapy; and Clinical Frailty Score.30
Analysis
Variables with large amounts of missing data will be excluded from the models as they would be missing in clinical practice. Likewise, continuous variables whose distributions are too narrow will also be excluded. We will perform binary recursive partitioning using Classification and Regression Trees to develop a decision rule. A clinical decision rule for a life-threatening event such as intracranial bleeding requires very high sensitivity. The model with a sensitivity of >99% and the highest specificity will be selected. We will assess the derived decision rule by comparing the classification of each patient with his or her actual status for the primary outcomes. In addition, 1000 bootstrap iterations will be performed to assess the internal classification performance and overfitting of the selected decision rule.
We will also develop a predictive risk model using multivariable logistic regression. Continuous variables may be transformed and will be fit using restricted cubic splines to relax the linearity assumption. First, a full model with all variables will be fit. To further reduce the model, we will perform backward elimination without model refitting with p<0.5, which has shown to have valid inference.31 32 Clinically and biologically plausible interactions will be tested within the model. Internal validation to obtain unbiased and optimism corrected estimation of model performance will be done using 1000 bootstrap samples. Model discrimination will be reported using the C-statistic and a calibration plot of observed versus predicted probabilities.
Sample size
The current guidelines suggest that we would require at least 10 events per included variable.33 34 We expect that 5% of patients will be diagnosed with clinically important intracranial bleeding,20 and we assume that our initial model will consist of 17 candidate variables. Based on this assumption, a sample size of 4000 should include 200 cases of intracranial bleeding (12 events per variable).
Sources of bias
Intracranial bleeding will be adjudicated blind to all baseline and predictor data. Predictor data are collected before the primary outcome data are collected. However, it is possible that we do not identify every case of intracranial bleeding during the 42-day follow-up period. In our prior study, only 60% of patients had a CT of the head during the index ED visit and 6/738 participants without a CT of the head (0.8%) were subsequently diagnosed with intracranial bleeding within 42 days.28 In comparison, 6/939 (0.6%) with a negative CT of the head were diagnosed with intracranial bleeding within 42 days, suggesting emergency physicians may correctly identify lower risk patients who do not require a scan. However, this evidence is indirect and hypothesis-generating only. Given that not all participants in this study will have a CT scan of the head at baseline, we may underdiagnose intracranial bleeding in this subpopulation which will comprise around 40% of the cohort. Although patients are advised to return if they develop neurological symptoms, it is possible that a patient may die of an intracranial bleed or else fully recover without testing for intracranial bleeding. Furthermore, 42-day follow-up involves institutional electronic medical record review. If a patient attended an unrelated hospital during follow-up and was diagnosed with an intracranial bleed, we might miss this diagnosis. To reduce the chance of this happening, we are restricting study enrolment to patients who reside within the hospital catchment area and most sites have access to records from regional neurosurgical centres. In our prior study where we performed in-person follow-up, no patient was diagnosed with an intracranial bleed at another hospital. The imperfect reference standard bias introduced with differential testing depending on the emergency physician CT request, might inflate the strength of association between predictor variables which are commonly used to determine the need for CT of the head in this population (such as a history of loss of consciousness and anticoagulation use).
Study oversight
The coordinating centre is McMaster University. Electronic data and deidentified source documents are uploaded to a Research Electronic Data Capture database35 36 and stored on a secure server at McMaster University. The coordinating centre validates all data and supervises the adjudication committee activities. The study steering committee consists of the site investigators.
Ethics and dissemination
The study was approved by the Hamilton Integrated Research Ethics Board, Ottawa Health Science Network Research Ethics Board, Mount Sinai Hospital Research Ethics Board, Comité d’éthique du CHU de Québec-Université Laval, Providence Health Care Research Ethics Board and the Institutional Review Board of St. Luke’s University Health Network. Research ethics approval has been obtained from each enrolling site local research ethics board. In our previous study on the same population,28 we obtained patient consent. An interim analysis showed a number of patients were confused (144/890, 16%) or died before a researcher could ask for their consent (39/890, 4%). Family were often not available in the ED. In all, we were unable to obtain consent from 204/890 (23%) patients. To address this problem, we obtained research ethics board approval to include patients who were unable to give informed consent. It is essential we include patients who cannot consent, since they are often the most frail patients who are challenging to evaluate in the ED and frequently excluded from studies. Excluding these patients could limit the generalizability of our clinical decision rule. The current study has research ethics approval at all sites to include patients without obtaining informed consent.
The study results will be submitted for publication in a peer-reviewed journal and presented at national and international emergency medicine meetings.
Supplementary Material
Footnotes
Twitter: @kerstindewit, @DavidBarbic
Deceased: Deceased 3rd June 2020
Contributors: The study was conceived by KdW, MM, CK, SP and AW. The protocol was designed with input from all authors (KdW, MM, CK, SP, AW, NC, EM, ME, IS, DE, DB, RJ, JM, CV, SM, AP, YK, AS, SS and PE) has been endorsed by the Network of Canadian Emergency Researchers. The study is being conducted by KdW, NC, EM, CV, DE, DB, RJ and JM. YK, AS, SS and PE are the study adjudicators. SP will oversee the analysis.
Funding: This work was supported by the Canadian Institute of Health Research grant number PJT-159545.
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review: Not commissioned; externally peer reviewed.
Ethics statements
Patient consent for publication
Not required.
References
- 1. Van den Brand CL, Karger LB, Nijman ST, et al. Traumatic brain injury in the Netherlands, trends in emergency department visits, hospitalization and mortality between 1998 and 2012. Eur J Emerg Med 2017;06:06. 10.1097/MEJ.0000000000000457 [DOI] [PubMed] [Google Scholar]
- 2. Haring RS, Narang K, Canner JK, et al. Traumatic brain injury in the elderly: morbidity and mortality trends and risk factors. J Surg Res 2015;195:1–9. 10.1016/j.jss.2015.01.017 [DOI] [PubMed] [Google Scholar]
- 3. McIntyre A, Mehta S, Aubut J, et al. Mortality among older adults after a traumatic brain injury: a meta-analysis. Brain Inj 2013;27:31–40. 10.3109/02699052.2012.700086 [DOI] [PubMed] [Google Scholar]
- 4. Chan V, Colantonio A, Chen A. A population based perspective of acquired brain injury in older adults: how do they happen? Brain Injury 2012;26:548–9. [Google Scholar]
- 5. Kerr ZY, Harmon KJ, Marshall SW, et al. The epidemiology of traumatic brain injuries treated in emergency departments in North Carolina, 2010-2011. N C Med J 2014;75:8–14. 10.18043/ncm.75.1.8 [DOI] [PubMed] [Google Scholar]
- 6. Albrecht JS, Hirshon JM, McCunn M, et al. Increased rates of mild traumatic brain injury among older adults in US emergency departments, 2009-2010. J Head Trauma Rehabil 2016;31:E1–7. 10.1097/HTR.0000000000000190 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Fu WW, Fu TS, Jing R, et al. Predictors of falls and mortality among elderly adults with traumatic brain injury: a nationwide, population-based study. PLoS One 2017;12:e0175868. (no pagination). 10.1371/journal.pone.0175868 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Peeters W, van den Brande R, Polinder S, et al. Epidemiology of traumatic brain injury in Europe. Acta Neurochir 2015;157:1683–96. 10.1007/s00701-015-2512-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Hastings DL, Brieding M, Lee R. Falls and traumatic brain injuries in older adults: a worsening trend. J Am Geriatr Soc 2017;65:S8. [Google Scholar]
- 10. Sung K-C, Liang F-W, Cheng T-J, et al. Trends in unintentional fall-related traumatic brain injury death rates in older adults in the United States, 1980-2010: a joinpoint analysis. J Neurotrauma 2015;32:1078–82. 10.1089/neu.2014.3509 [DOI] [PubMed] [Google Scholar]
- 11. Fletcher AE, Khalid S, Mallonee S. The epidemiology of severe traumatic brain injury among persons 65 years of age and older in Oklahoma, 1992-2003. Brain Inj 2007;21:691–9. 10.1080/02699050701426873 [DOI] [PubMed] [Google Scholar]
- 12. Chisholm KM, Harruff RC. Elderly deaths due to ground-level falls. Am J Forensic Med Pathol 2010;31:350–4. 10.1097/PAF.0b013e3181f69c87 [DOI] [PubMed] [Google Scholar]
- 13. Brazinova A, Mauritz W, Majdan M, et al. Fatal traumatic brain injury in older adults in Austria 1980-2012: an analysis of 33 years. Age Ageing 2015;44:502–6. 10.1093/ageing/afu194 [DOI] [PubMed] [Google Scholar]
- 14. Shankar KN, Liu SW, Ganz DA. Trends and characteristics of emergency department visits for fall-related injuries in older adults, 2003-2010. West J Emerg Med 2017;18:785–93. 10.5811/westjem.2017.5.33615 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Hartholt KA, Van Lieshout EMM, Polinder S, et al. Rapid increase in hospitalizations resulting from fall-related traumatic head injury in older adults in the Netherlands 1986-2008. J Neurotrauma 2011;28:739–44. 10.1089/neu.2010.1488 [DOI] [PubMed] [Google Scholar]
- 16. Korhonen N, Niemi S, Parkkari J, et al. Incidence of fall-related traumatic brain injuries among older Finnish adults between 1970 and 2011. JAMA 2013;309:1891–2. 10.1001/jama.2013.3356 [DOI] [PubMed] [Google Scholar]
- 17. Verma SK, Willetts JL, Corns HL, et al. Falls and fall-related injuries among community-dwelling adults in the United States. PLoS One 2016;11:e0150939. no pagination. 10.1371/journal.pone.0150939 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Stiell IG, Wells GA, Vandemheen K, et al. The Canadian CT head rule for patients with minor head injury. The Lancet 2001;357:1391–6. 10.1016/S0140-6736(00)04561-X [DOI] [PubMed] [Google Scholar]
- 19. de Wit K, Merali Z, Kagoma YK, et al. Incidence of intracranial bleeding in seniors presenting to the emergency department after a fall: a systematic review. Injury 2020;51:157–63. 10.1016/j.injury.2019.12.036 [DOI] [PubMed] [Google Scholar]
- 20. Kelen G, Peterson S, Pronovost P. In the name of patient safety, let's burden the emergency department more. Ann Emerg Med 2016;67:737–40. 10.1016/j.annemergmed.2015.11.031 [DOI] [PubMed] [Google Scholar]
- 21. Émond M, Grenier D, Morin J, et al. Emergency department stay associated delirium in older patients. Can Geriatr J 2017;20:10–14. 10.5770/cgj.20.246 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Stiell IG, Wells GA, Vandemheen KL, et al. The Canadian C-spine rule for radiography in alert and stable trauma patients. JAMA 2001;286:1841–8. 10.1001/jama.286.15.1841 [DOI] [PubMed] [Google Scholar]
- 23. Wells PS, Anderson DR, Rodger M, et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and D-dimer. Ann Intern Med 2001;135:98–107. 10.7326/0003-4819-135-2-200107170-00010 [DOI] [PubMed] [Google Scholar]
- 24. Thiruganasambandamoorthy V, Stiell IG, Sivilotti MLA, et al. Risk stratification of adult emergency department syncope patients to predict short-term serious outcomes after discharge (RiSEDS) study. BMC Emerg Med 2014;14:8 10.1186/1471-227X-14-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999;33:437–47. 10.1016/S0196-0644(99)70309-4 [DOI] [PubMed] [Google Scholar]
- 26. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162:55–63. 10.7326/M14-0697 [DOI] [PubMed] [Google Scholar]
- 27. Selvanayagam N, Mowbray F, Clayton N, et al. Reliability of patient-reported outcome measures: hemorrhage, anticoagulant, antiplatelet medication use 2021;5:e12501. 10.1002/rth2.12501 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. de Wit K, Parpia S, Varner C, et al. Clinical predictors of intracranial bleeding in older adults who have fallen: a cohort study. J Am Geriatr Soc 2020;68:970–6. 10.1111/jgs.16338 [DOI] [PubMed] [Google Scholar]
- 29. Schulman S, Kearon C. The SOCOAOTS, standardization Committee of the International Society on T, haemostasis. Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J Thromb Haemost 2005;3:692–4. 10.1111/j.1538-7836.2005.01204.x [DOI] [PubMed] [Google Scholar]
- 30. Rockwood K, Song X, MacKnight C, et al. A global clinical measure of fitness and frailty in elderly people. CMAJ 2005;173:489–95. 10.1503/cmaj.050051 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Lawless JF, Singhal K. Efficient screening of Nonnormal regression models. Biometrics 1978;34:318–27. 10.2307/2530022 [DOI] [Google Scholar]
- 32. Harrell F. Regression modeling strategies with applications to linear models, logistic regression, and survival analysis. New York: Springer, 2001. [Google Scholar]
- 33. Mallett S, Royston P, Dutton S, et al. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med 2010;8:20. 10.1186/1741-7015-8-20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Pavlou M, Ambler G, Seaman SR, et al. How to develop a more accurate risk prediction model when there are few events. BMJ 2015;351:h3868. 10.1136/bmj.h3868 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81. 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Harris PA, Taylor R, Minor BL, et al. The REDCap Consortium: building an international community of software platform partners. J Biomed Inform 2019;95:103208. 10.1016/j.jbi.2019.103208 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.