Predicting incident delirium diagnoses using data from primary-care electronic health records

Abstract

Importance

risk factors for delirium in hospital inpatients are well established, but less is known about whether delirium occurring in the community or during an emergency admission to hospital care might be predicted from routine primary-care records.

Objectives

identify risk factors in primary-care electronic health records (PC-EHR) predictive of delirium occurring in the community or recorded in the initial episode in emergency hospitalisation. Test predictive performance against the cumulative frailty index.

Design

Stage 1: case-control; Stages 2 and 3: retrospective cohort.

Setting

clinical practice research datalink: PC-EHR linked to hospital discharge data from England.

Subjects

Stage 1: 17,286 patients with delirium aged ≥60 years plus 85,607 controls. Stages 2 and 3: patients ≥ 60 years (n = 429,548 in 2015), split into calibration and validation groups.

Methods

Stage 1: logistic regression to identify associations of 110 candidate risk measures with delirium. Stage 2: calibrating risk factor weights. Stage 3: validation in independent sample using area under the curve (AUC) receiver operating characteristic.

Results

fifty-five risk factors were predictive, in domains including: cognitive impairment or mental illness, psychoactive drugs, frailty, infection, hyponatraemia and anticholinergic drugs. The derived model predicted 1-year incident delirium (AUC = 0.867, 0.852:0.881) and mortality (AUC = 0.846, 0.842:0.853), outperforming the frailty index (AUC = 0.761, 0.740:0.782). Individuals with the highest 10% of predicted delirium risk accounted for 55% of incident delirium over 1 year.

Conclusions

a risk factor model for delirium using data in PC-EHR performed well, identifying individuals at risk of new onsets of delirium. This model has potential for supporting preventive interventions.

Key points

• Information on predictors of incident episodes of delirium in individuals in primary-care or during emergency hospital admission is scarce.

• Using primary-care records, we identified 55 risk factors in routine electronic health records that predict incident delirium diagnoses, in 60+ year olds.

• A regression model validated using independent data was a good predictor of 1- and 2-year incident delirium and mortality, and outperformed the electronic frailty index.

• The model has potential to help target clinical interventions.

Background

Delirium is a disturbance of consciousness, cognitive function or perception, with acute onset and fluctuating course [1]. Episodes are common in individuals aged over 65, (prevalence 1–2%), resulting in approximately 200,000 cases per year in the UK [2, 3]. Delirium is associated with high levels of morbidity and mortality [2, 4], including prolonged hospitalisation, institutional placement and high health care costs [4, 5].

Delirium in hospital inpatients is well-studied with many risk factors identified [6]. Interventions to prevent delirium in hospital inpatients through improved case management have been tested [7]. However, studies investigating primary-care recorded risk factors for delirium recorded in the community or during admission to an emergency hospital visit have been limited by small samples sizes and few cases [2, 8]. Additional research into the primary-care recorded risk factors and prediction of delirium remains necessary if we hope to identify those at risk [9, 10].

Work evaluating primary-care electronic health record (PC-EHR) alerts for older patients is encouraging [11]. Continuously updated software alerts for practitioners to help highlight the likely need to assign care packages to frail individuals have been implemented in England, using the electronic frailty index (eFI) [12]. The eFI is a Rockwood-based cumulative deficit model based on 36 unweighted conditions. Those classified as frail by the electronic health record software can be assigned a falls risk assessment and an annual medication review. This study explores a delirium prediction model to be used in similar fashion.

Candidate risk factors from PC-EHR for whole older populations in England are available. This study aimed to: (i) identify risk factors for delirium diagnoses from PC-EHR, (ii) calibrate regression models for 1 and 2 year delirium (in primary-care or emergency admission) and (iii) compare model performance against the eFI in an independent validation sample.

Methods

The Clinical Practice Research Datalink (CPRD) links PC-EHR and secondary-care health records and national death certification [13].

Delirium in primary-care and in emergency hospital admissions

Delirium diagnoses were identified from primary-care and hospital inpatient records (diagnosed at admission), to improve recording accuracy [14]. Records of delirium were included if patients were diagnosed first in primary-care or during the first consultant episode-of-care for an emergency admission to hospital. Delirium is often recorded multiple times during a hospital stay; therefore, only episodes of 60 days or more apart were considered independent diagnoses. In line with study aims, our definition of delirium aims to identify delirium occurring in the community or recorded in the initial episode in emergency hospitalisation, while excluding in-patient delirium and delirium tremens.

Risk factors

We identified potential risk factors from previous studies of delirium in the community and in hospital inpatients [3, 11], markers of frailty as defined by the eFI [9], chronic conditions and syndromes common in old age and pharmacological interventions.

The list contains 110 candidate clinical risk factors including medical diagnosis, frailty markers, abnormal diagnostic tests and medication and medical referrals (Supplementary Table S1). All risk factors were recorded prior to an index-date and as dichotomous variables. We used the most recent risk factor code or measurement present within the specified lead-in time. Lead-in time for each risk factor was specified with clinician input to ensure clinical relevance while maximising the use of available data. The same risk factor rules were applied to all three stages, using stage specific index-date (below). Electronic health records are used to record clinically important information only; therefore, the absence of diagnoses or prescription codes were considered to indicate an absence of disease. Similarly, absent medical test data were considered to be normal. Under this approach, all records were considered to have complete information for clinical risk factors for statistical analysis purposes.

Stage 1: Identifying risk factors

A case-control analysis (Stage 1) identified factors associated with a delirium event. Cases were from records dated 1 January 2001 to 31 November 2014 in individuals registered with a practice that is up-to-standard for at least 6 months and alive and registered at the time of diagnosis. Index-date was the date delirium was first recorded in EHR. Individuals were used once in the analysis with the latest case of delirium selected when repeated cases were available. Cases were matched with age, sex and year of study entry matched controls (1:4 ratio; Supplementary Figure S1). Association between risks and incident delirium was estimated using conditional logistic regression. First, a univariate analysis identified factors independently associated with delirium. A backwards deletion process using multivariate regression identified the final list of risk factors. Significance was set at P-values ≤ 0.01.

Stage 2: Calibration of risk factor weights and initial tests of predictive performance

Retrospective cohort design was used for model development and to test predictive performance. Records from all individuals aged ≥60 years, alive, registered with the practice that is up to standard for at least 1 year on 1 January 2015 (index-date for Stages 2 and 3) and followed up for up to 2 years were identified. We randomly split this sample into two subsets using an 80/20 ratio (Supplementary Figure S1). In Stages 2 and 3, additional factors were included in the model alongside the risk factors identified in Stage 1, including age, gender, UK region and quintiles of index of multiple deprivation (IMD) [15]. Values for IMD were missing for 0.02% of the sample and included as a separate category.

The 80% subsample was allocated for generating risk factor weights using multivariate logistic regression [16]. A single run of the logistic regression model provided risk factor weights and risk factors were not excluded based on P-value performance. The predictive model also includes age, gender, IMD and UK regions as predictors.

The primary outcome was incident delirium (first occurrence) in the 1 or 2 years (separately) since beginning of follow-up. To ascertain delirium we used all cases identified between 1 January 2015 and 31 December 2016, using the methods defined above. Additional outcomes include incident all-cause mortality, falls and hospitalisation.

Stage 3: Independent validation of performance

Internal validation of predictive performance used an independent subsample (Supplementary Figure S1). We used the area under the curve (AUC) statistic for receiver operating characteristic (ROC) curve [17] in the validation data, to measure predictive performance. We evaluated the ROC curves from 200 bootstrap samples, retrieved with replacement. The 95% confidence interval (CI) was obtained from 2.5th to 97.5th percentiles of the bootstrap distribution. In Stage 3, model performance was also compared with that of the eFI, in the same validation subsample (Supplementary Figure S1) and for primary and additional outcomes. We also included a sensitivity analysis focusing on delirium diagnosis from PC-EHR alone.

Results

Stage 1: identifying risk factors

In total, 17,286 individuals diagnosed with delirium aged 60 years and older and 68,321 controls (97.4% of cases matched with four controls) were eligible for analyses. Females accounted for 61.1% of the population and the mean age was 82.4 years (sd 7.9) (Supplementary Table S2).

From 110 candidate factors, 99 were independently associated with delirium in univariate models (Supplementary Table S3). The backwards deletion method excluded 44 factors, with 55 statistically independent factors remaining in the multivariate model (Figure 1). Risk factors with the strongest association with delirium were: serious mental illness (odds ratio (OR), 6.9; 95% CI, 6.2–7.6), abnormal white blood cell counts (OR, 2.7; 95% CI, 1.5–4.7), referral to geriatric services (OR, 2.5 95%; CI, 2.3–2.8), prior delirium (OR, 2.5; 95% CI, 2.0–3.0), smoking (OR, 2.3; 95% CI, 2.2–2.5), cirrhosis (OR, 2.1; 95% CI, 1.5:2.9), hospital visit (OR, 2.0; 95% CI, 1.9–2.1), dementia (OR, 2.0; 95% CI, 1.8–2.1), neutrophilia (OR, 1.9; 95% CI, 1.6–2.1) and hyponatraemia (OR, 1.9; 95% CI, 1.7–2.1) (Supplementary Table S3). Interestingly, 14 of the 55 factors were components, or proxies of components of the eFI (Figure 1) [3].

Figure 1.

Forrest plot showing odds ratio (95% CI) estimates for delirium for risk factors identified in the Stage 1 case-control analysis.

Conversely, some markers were associated with lower delirium risk (nursing home residency (ascertained through doctor consultation in a nursing home (OR, 0.71; 95% CI, 0.62–0.82)), individuals receiving specialised care (OR, 0.83; 95% CI, 0.75–0.92), being overweight (OR, 0.79; 95% CI, 0.75–0.82), osteoarthritis (OR, 0.94; 95% CI, 0.90–0.98), all cardiovascular drugs (with the exception of beta-blockers (OR, 1.11; 95% CI, 1.06–1.17)) and referral to cardiology services (Figure 1)).

Stage 2: fitting the model in primary-care population representative records

Model development used a population representative sample of 343,548 individuals (mean age 73.5, sd 8.5), and 53.6% female. Incident studied outcomes were: n = 1920 (0.6%) developed delirium, n = 11,974 (3.5%) died, n = 10,169 (3.0%) had one or more falls and n = 91,785 (26.7%) visited hospital at least once in the first year. Incidence of outcomes over 1 and 2 years are provided in Supplementary Table S4. The full prediction model with all regression coefficients, and model intercept or baseline incidence rate produce in Stage 2 have been included in the supplemental material (Supplementary Table S5), and results by quintiles of risk in Table 1.

Table 1.

Odds ratio for incident delirium by quintiles of delirium index, and frailty index, in the validation sample: Stage 3

	Stage 2: training sample			Stage 3: validation sample
	Delirium index			Delirium index			Frailty index
	Odds ratio	95% CI	P-value	Odds ratio	95% CI	P-value	Odds ratio	95% CI	P-value
One year prediction of delirium
Quintile
1	1.0	(1.0:1.0)	–	1.0	(1.0:1.0)	–	1.0	(1.0:1.0)	–
2	2.3	(1.1:5.1)	0.03	3.0	(0.8:11.1)	0.1	1.8	(1.1:3.2)	0.03
3	7.8	(3.9:15.6)	<0.01	6.0	(1.8:20.4)	<0.01	3.7	(2.4:5.8)	<0.01
4	27.9	(14.3:54.2)	<0.01	24.4	(7.7:77.5)	<0.01	5.9	(3.6:9.6)	<0.01
5	178.5	(92.7:343.7)	<0.01	118.4	(38.0:368.9)	<0.01	15.1	(9.9:22.9)	<0.01
Two year prediction of delirium
Quintile
1	1.0	(1.0:1.0)	–	1.0	(1.0:1.0)	–	1.0	(1.0:1.0)	–
2	3.8	(2.3:6.3)	<0.01	2.6	(1.2:5.5)	0.02	2.3	(1.6:3.3)	<0.01
3	10.4	(6.5:16.7)	<0.01	5.9	(2.9:12.0)	<0.01	4.2	(3.1:5.8)	<0.01
4	33.2	(21.0:52.4)	<0.01	19.4	(9.9:37.9)	<0.01	7.3	(5.3:10.2)	<0.01
5	172.7	(110.0:271.2)	<0.01	85.6	(44.4:165.3)	<0.01	15.1	(11.3:20.2)	<0.01

Stage 3: independent sample validation of predictive performance

Model validation used an independent population representative sample of 85,887 individuals (mean age 73.5, sd 8.5) (Supplementary Table S4). Incidence of studied outcomes over 1 and 2 years are presented in Supplementary Table S4. A logistic analysis of associations with delirium by quintiles of predicted delirium index (Table 1) showed a dose–response relationship. Individuals in the highest risk quintile were more likely to develop delirium (OR = 118.4 (CI 38.0:368.9)) compared to the lowest quintile. Similar analysis on 2 year prediction produced similar results (Table 1). Estimates for the eFI yielded more modest risk differentials (e.g. odds ratio, 15.1; CI, 9.9:22.9 for the highest quintile versus lowest quintile).

C-statistics measured the predictive ability of the individual scores produced using the independent validation sample. Based on the equation fitted for 1 year delirium, ROC curves were plotted for incident delirium, all-cause mortality, falls and hospitalisation for 1 and 2 years of follow-up (Table 2). The analysis of ROC curves indicated good predictive power for most outcomes (Table 2): for delirium, 1 year AUC = 0.87 (CI 0.85:0.88) and 2 year predictions AUC = 0.85 (CI 0.84:0.86), as well as 1 year mortality AUC = 0.85 (CI 0.84:0.85). The delirium specific model outperformed the eFI (AUC = 0.74; CI 0.74:0.78) for prediction of delirium, mortality and falls, but not for hospitalisation. A sensitivity analyses focusing on PC-EHR ascertained delirium found similar results (Supplementary Table S6).

Table 2.

AUC values for incident outcomes in specified follow-up time for the delirium index, and frailty index, in the validation sample: Stage 3

	Stage 3: validation sample
	Delirium index		Frailty index
One year follow-up	AUC	CI	AUC	CI
Delirium	0.87	0.85:0.88	0.76	0.73:0.78
Death	0.85	0.84:0.85	0.74	0.74:0.75
Fall	0.75	0.75:0.76	0.71	0.71:0.73
Hospitalisation	0.68	0.67:0.68	0.67	0.67:0.67
Two year follow-up
Delirium	0.85	0.84:0.86	0.74	0.74:0.77
Death	0.84	0.83:0.84	0.74	0.73:0.74
Fall	0.75	0.74:0.76	0.71	0.71:0.72
Hospitalisation	0.68	0.67:0.68	0.67	0.67:0.68

Finally, we explored cut points for interventions based on the ROC curve (Supplementary Table S7). At the Youndex point (specificity: 72.7%; sensitivity: 86.5%), the optimal point in the curve [18], the model highlights 29.1% of population as at risk of a future delirium event, predicting correctly 86.5% of future cases. A more stringent alternative cut point, focusing on higher specificity (specificity: 90.8%), identified 10.3% at high risk and this group went on to develop 55.4% of all the delirium diagnoses.

Discussion

Risk factors for delirium in hospital inpatients have been extensively studied [2], but less is known about the large number of community-dwelling older people who develop delirium in the community or during admission to an emergency hospital visit [8]. Our analyses identified an extensive list of risk factors from PC-EHR for delirium, and we have validated them by estimating their predictive ability in an independent sample that is broadly representative of England’s older population. All risk factors were ascertained before the outcomes, and it should be noted that the items included do not necessarily have causal significance.

A case-control analysis (Stage 1), with over 17,000 cases of delirium and up to four matched controls identified 55 risk factors falling into seven domains: cognitive impairment or mental illness, prior delirium, psychoactive drugs, frailty/related conditions, infection markers, metabolic disturbance, hyponatraemia and high anticholinergic burden. Interestingly, the majority of risk factors identified showed overlap with those identified for delirium in hospital inpatients, suggesting shared mechanisms [5, 6, 19–21]. Diagnoses for depression, diabetes and kidney failure were not directly identified but through proxies, such as prescription drugs and laboratory test results.

A number of factors were statistically associated with lower risk of delirium diagnosis. Prescription of steroids and most cardiovascular drugs (except beta-blockers) were associated with lower risk of delirium diagnoses (calcium channel blockers, nitrates and antiarrhythmics), in line with findings from previous studies [22]. Other associations with lower risk of delirium likely mark better health states, access to care or of poor recording in EHR, rather than being causal. Lower risks of diagnosed delirium were found with nursing home residency and or receiving specialised care, which contrast with finding from previous studies showing higher rates of delirium in these groups [23]; a possible explanation might be poor recording of delirium in these settings, but more work is needed to clarify these findings including systematic ascertainment of delirium. Additionally, apparently protective associations with overweight and osteoarthritis (a condition common in overweight individuals [24]) likely result from reverse causation, with higher weights identifying healthier individuals who have not suffered weight loss from serious co-morbidity.

The identified 55 risk factors represent the most current and comprehensive set of risk factors of delirium in PC-EHR produced to date [5, 9]. We validated the risk factors by testing predictive performance using independent development and validation data. The model produced AUC scores in excess of 0.85 for predicting delirium and mortality in 1 year and delirium in 2 years of follow-up, indicating good performance.

Limitations

Inevitably there are limitations to this analysis. First, delirium is regarded as underreported in electronic health records; however, high specificity is expected with the most severe episodes recorded [25]. Combining primary- and secondary-care data, as applied to delirium has shown to improve diagnosis ascertainment [13]. Second, ascertainment of delirium recorded in the community or during admission in emergency hospitalisations based on electronic health records is challenging, particularly for secondary-care where some misclassification with delirium starting post-admission may occur. However, a sensitivity analysis using only PC-EHR ascertained delirium produced similar results to the main analysis. Further work with systematically ascertained delirium is necessary to assess biases from poor recording. Third, in analysing research questions or measures, the absence of information for required items is defined as missing data. However, electronic health records are a special case, where absence of information represents a negative value for disease presence and not missing data. For statistical purposes, we assume no missing data in the studied electronic health records. This may overestimate the completeness of record-keeping; however, any other assumption, necessarily without supporting evidence, could introduce biases and could not provide a practical basis for using the existing PC-EHRs for delirium prediction. Finally, we selected a common and robust design of randomly splitting the available data for creating training and validation datasets [26], but alternative methods for validation (e.g. time or geographical) should be tested in future work.

Potential applications and future work

This study demonstrates the potential to identify individuals at risk of delirium using routinely collected ‘real life’ PC-EHR, in a sample that is broadly representative of the older population in England [13]. Classifying the top 10.3% of the older primary-care population as at risk would correctly identify 55.4% of cases of delirium occurring within 1 year (Supplementary Table S6). For clinical context in routine primary care practice, considering 12 million individuals age 65 and older living in the UK, a 2% prevalence of delirium in primary-care equates to over 240,000 cases of delirium per year. Such a threshold would identify as high risk 1.3 million individuals, potentially predicting correctly over 130 thousand cases of delirium in the UK. Similarly, the model’s predictive ability for 1 year mortality means it is likely that within the same at risk group are included over half of the expected 420,000 yearly deaths.

Transferability into routine primary-care practice could follow the approach UK health services adopted for the frailty index [27]. This computer based algorithm would need periodical reviews to reweigh risk factors, adjusting for changes in delirium recording and services provided. Potential intervention strategies targeting high risk individuals aiming to encourage early recognition of delirium may lead to improvements to delirium management. In hospital settings, preventive strategies have proven effective at reducing delirium [28], reduced mortality rates, improved quality of life and led to savings in health care costs [29]. Ultimately, we hope similar prevention strategies are feasible for those at high risk of delirium living in the community. Intervention studies will be needed to define the content and impact of preventive efforts in primary-care.

Future work remains necessary on the causal role of predictors, how these interact and accumulate to cause delirium. It would also be helpful to establish whether risk of delirium is driven by cognitive impairment or higher degrees of deficit accumulation. Additional work must also test the robustness of the model (e.g. validation in an external primary-care data) as well as of alternative modelling techniques (e.g. machine learning) [30]. Regarding community acquired delirium in general, future work should aim to understand the extent to which it can be prevented.

Conclusions

We identified a large set of risk factors for delirium routinely collected in primary-care records. These risk factors overlap with predictors of delirium in hospital inpatients. A derived predictive model proved a good predictor of delirium and all-cause mortality in an independent population representative validation sample. It outperformed the frailty index for delirium, all-cause mortality and falls. These risk factors and models have potential to support family physicians and inform interventions to reduce delirium associated impacts in community-dwelling older people.

Ethical and Scientific Approval

The CPRD has been granted Multiple Research Ethics Committee approval (05/MRE04/87) to undertake observational studies. The work of CPRD is also covered by NIGB-ECC approval ECC 5-05 (a) 2012. This study was approved by the Independent Scientific Advisory Committee for MHRA database research (ISAC) under protocol number 17_189.

Declaration of Conflict of Interests

None.

Funding

This work was funded by the National Institute for Health Research (NIHR), Research for Patient Benefit (RfPB) Programme: PB-PG-1215-20022.

David Melzer is supported by the Medical Research Council and the universities of Exeter and Connecticut. The Funders played no role in design, execution, analysis and interpretation of data, or writing of the study.

Jane Masoli is supported by an NIHR Doctoral Research Fellowship (DRF-2014-07-177). The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the National Institute for Health Research or the Department of Health.