Using Natural Language Processing on Electronic Health Records to Enhance Detection and Prediction of Psychosis Risk

Abstract

Background

Using novel data mining methods such as natural language processing (NLP) on electronic health records (EHRs) for screening and detecting individuals at risk for psychosis.

Method

The study included all patients receiving a first index diagnosis of nonorganic and nonpsychotic mental disorder within the South London and Maudsley (SLaM) NHS Foundation Trust between January 1, 2008, and July 28, 2018. Least Absolute Shrinkage and Selection Operator (LASSO)-regularized Cox regression was used to refine and externally validate a refined version of a five-item individualized, transdiagnostic, clinically based risk calculator previously developed (Harrell’s C = 0.79) and piloted for implementation. The refined version included 14 additional NLP-predictors: tearfulness, poor appetite, weight loss, insomnia, cannabis, cocaine, guilt, irritability, delusions, hopelessness, disturbed sleep, poor insight, agitation, and paranoia.

Results

A total of 92 151 patients with a first index diagnosis of nonorganic and nonpsychotic mental disorder within the SLaM Trust were included in the derivation (n = 28 297) or external validation (n = 63 854) data sets. Mean age was 33.6 years, 50.7% were women, and 67.0% were of white race/ethnicity. Mean follow-up was 1590 days. The overall 6-year risk of psychosis in secondary mental health care was 3.4 (95% CI, 3.3–3.6). External validation indicated strong performance on unseen data (Harrell’s C 0.85, 95% CI 0.84–0.86), an increase of 0.06 from the original model.

Conclusions

Using NLP on EHRs can considerably enhance the prognostic accuracy of psychosis risk calculators. This can help identify patients at risk of psychosis who require assessment and specialized care, facilitating earlier detection and potentially improving patient outcomes.

Introduction

The burden of psychotic disorders is substantial: for example, schizophrenia accounted for 13 million years lived with disability in 2017,¹ with the most recent estimate reporting a European economic burden of €93.9 billion.² Existing treatments have little impact on illness course in established psychosis.^3,4 Primary indicated prevention in individuals at clinical high risk for psychosis (CHR-P), however, has the potential to reduce the duration of untreated psychosis and alter its course.^5,6 Effective preventive intervention is reliant on the successful identification of individuals at risk for psychosis for referral to specialized CHR-P clinical services; ^7–9 these individuals tend to present with attenuated psychotic symptoms and overall functional impairment.^10,11 Detection of at-risk individuals currently relies on help-seeking behaviours¹² and idiosyncratic referral pathways initiated on suspicion of psychosis risk.¹³ Emerging evidence suggests that current detection strategies are highly inefficient,¹³ with only 5%¹⁴–12%¹⁵ of first episode cases intercepted by CHR-P clinical services. To tackle these challenges, we previously developed an individualized, clinically based transdiagnostic risk calculator, using clinical and demographic predictors widely available in electronic health records (EHRs): age, sex, age by sex, ethnicity, and index ICD-10 diagnosis or CHR-P designation.¹⁶ The transdiagnostic risk calculator has been externally validated in two separate large EHR datasets, demonstrating adequate prognostic performance (n = 54 716, Harrell’s C = 0.79; ¹⁴n = 13 702, Harrell’s C = 0.73¹⁷). This transdiagnostic risk calculator has already undergone pilot testing for clinical implementation.¹⁸ The calculator’s potential for implementation, combined with its unique position to enhance large-scale detection of at-risk individuals, underscores the importance of improving its prognostic accuracy. Given the replication crisis in psychiatry and science,^19,20 improving existing, previously validated risk prediction models represents a more efficient approach than redeveloping new models.²¹

While EHRs offer some information (notably on sociodemographic characteristics) in structured fields, the majority of information is recorded in free text such as event notes and uploaded attachments, representing an enormous reservoir of untapped information.²² For example, information on symptomatology and substance use is not routinely recorded in a structured way.²² Natural language processing (NLP) techniques have recently been developed to mine structured data from free text. These offer an unprecedented opportunity to incorporate more granular predictors closer to the pathophysiology of psychosis onset into a model. By applying NLP to EHRs, this study aims to improve on the prognostic accuracy achieved by the previously published transdiagnostic risk calculator,¹⁶ further supporting the efficient detection of individuals at risk for psychosis.

Methods and Materials

Setting

The South London and Maudsley (SLaM) NHS Trust is one of Europe’s largest secondary mental health-care providers.²³ Its main catchment area covers four socioeconomically diverse South London boroughs: Croydon, Lambeth, Lewisham, and Southwark, alongside tertiary referrals from the rest of London and the United Kingdom. The Clinical Record Interactive Search (CRIS) system facilitates interrogation of de-identified EHRs held by the Trust, which adopted a system of electronic recording in 2007.^22,23 SLaM now has EHRs for over 400 000 individuals, providing data on their sociodemographic and clinical characteristics.

Study Population

We extracted data for all individuals accessing the SLaM NHS Trust between January 1, 2008 and July 28, 2018. Inclusion and exclusion criteria followed that of the original analysis and development of the psychosis risk calculator: namely, all individuals who received a first index primary diagnosis of any nonorganic and nonpsychotic mental disorder.¹⁶

Model Specifications

Original Model

As detailed in Fusar-Poli et al,¹⁶ the original transdiagnostic, clinically based, and individualized risk calculator was developed using a retrospective cohort study leveraging EHRs from the SLaM boroughs of Lambeth and Southwark. Cox regression was used to predict the hazard ratio (HR) of developing any psychotic disorder over time (defined in Supplementary methods S1). Predictors included age (at the time of index diagnosis), sex, age by sex, self-assigned ethnicity, and cluster index diagnosis or CHR-P designation (defined in Supplementary tables S2 and S3). The model was externally validated first in the SLaM boroughs of Croydon and Lewisham and later in Camden and Islington NHS Trust.^16,17 In this retrospective version of the risk calculator, individuals who developed psychosis within 3 months following their index diagnosis were excluded. However, implementing this diagnostic lag prospectively in the subsequent implementation study would have resulted in delays for referral to assessment. Therefore, a refined version of the risk calculator without the lag period, which demonstrated similar external prognostic accuracy, was optimized for prospective use and is considered in the current study (for details, see Supplementary table S4).

Model Refinement with NLP Data

In the present study, we refined the prospective version of the transdiagnostic model using all original predictors plus additional NLP-derived predictors. NLP tools were used to extract symptom and substance use data from free text recorded by clinicians within the 6 months prior to the index diagnosis. This time period was chosen to ensure that predictor data did not overlap with our outcome variables, and because symptom assessment tends to precede formal diagnosis. We employed CRIS-specific NLP algorithms that convert unstructured information from free text into structured and quantifiable data. Details on symptom algorithm development can be found in Jackson et al; ²² in general, these were developed using cross-validated support vector machines on a gold-standard, human-annotated training corpus for each symptom. A regularly updated algorithm library, with comprehensive detail on keywords used and validation efforts, can be found on the CRIS website.²⁴ NLP algorithm performance is mainly measured in terms of precision (proportion of true positive instances of total NLP-labelled positive instances) and recall (proportion of true positive instances of all positive instances available in the text). As EHRs provide multiple opportunities for term detection, we favored precision over recall, using only NLP algorithms with at least 80% precision (see Supplementary methods S2 and Supplementary table S5). We also excluded predictors with near-zero variance, which can cause model instability across validation folds.²⁵ This resulted in 14 NLP symptom and substance use predictors with a mean (SD) precision of 0.91 (0.06): tearfulness, poor appetite, weight loss, insomnia, cannabis use, cocaine use, guilt, irritability, delusions, hopelessness, disturbed sleep, poor insight, agitation, and paranoia. We dichotomized NLP symptom and substance use predictors as trigger terms tend to be repeated within and across records; treating predictors as continuous would otherwise have resulted in the model erroneously interpreting them as a linear reflection of severity.^26–28 The value “0” indicated that a given symptom or substance was not mentioned in a patient’s EHR. We retained individuals without NLP-derivable symptom or substance data prior to index diagnosis, treating NLP data as bonus information where available.

Statistical Analysis

This EHR clinical register-based study is reported according to the RECORD and STROBE statements (see Supplementary table S1).²⁹ Model development and validation followed the methodological guidelines of Royston and Altman,³⁰ Steyerberg et al,³¹ and the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD).³²

We performed descriptive analyses of baseline clinical and sociodemographic characteristics of the sample, obtaining means and frequencies for continuous and categorical variables, respectively. The Kaplan–Meier failure function (1-survival) and Greenwood 95% confidence intervals were used to describe the cumulative risk of psychosis onset in SLaM patients. The primary outcome measure was model prognostic discrimination performance measured through Harrell’s C, a recommended measure for external validation of Cox models.³³ A Harrell’s C value of 0.9–1.0 is considered outstanding, 0.8–0.9 excellent and 0.7–0.8 acceptable.³⁴ Model development and validation analyses were conducted in R version 3.6.1.

Model Development

We first divided our cohort into derivation and validation samples using nonrandom, geographic split-sampling, which is one of the recommended approaches to model building.³² Mirroring the original analysis, the derivation sample comprised all cases from Lambeth and Southwark until December 31, 2015.¹⁶ The validation sample comprised all cases from the same two boroughs from January 2016 plus cases from all other boroughs, constituting temporal and geographic forms of external validation. These samples differ on several sociodemographic characteristics.²³

We then trained a Cox proportional-hazards model on our derivation sample using the refined transdiagnostic model. Since adding large numbers of predictors can result in overfitting,²¹ we regularized our model using the Least Absolute Shrinkage and Selection Operator (LASSO) penalty, implemented via the glmnet package in R. The LASSO algorithm performs feature selection by shrinking the coefficients of redundant predictors toward zero. This penalty requires selection of a tuning parameter lambda, which controls the number of coefficients estimated to be non-zero. We used 10-fold cross-validation to select the optimal lambda, choosing the minimum lambda value that maximized partial likelihood from a range of possible values. With the resultant model coefficients, we developed a prognostic index in the derivation dataset by generating prognostic risk scores for each individual.

External Model Validation

We applied the regression coefficients from the derivation data set to each case in the external validation set to generate the prognostic index for the validation dataset. Model prognostic performance (Harrell’s C, which captures discrimination)³¹ was the primary outcome measure. We further assessed overall model performance using the Brier score (the average mean squared difference between predicted probabilities and actual outcomes), which captures calibration and discrimination aspects.³¹ A lower score indicates higher precision and less bias. Calibration (the agreement between observed outcomes and predictions) was assessed with the regression slope of the prognostic index.³⁵ Finally, we performed a sensitivity analysis to assess whether our model would perform better in temporal or geographic external validation by splitting our external validation set into these two groups.

Results

Sociodemographic and Clinical Characteristics

Of 108 211 individuals receiving a first SLaM diagnosis of nonorganic and nonpsychotic mental disorder within SLaM in the period between January 1, 2008 and July 28, 2017, 92 151 had complete data across all original predictors (see figure 1). Mean (SD) age was 33.6 (19.0); individuals were almost evenly split by sex (female 50.7%, male 49.3%), most were of White ethnicity (67.0%), and anxiety disorders were the most frequent index diagnoses (27.5%, table 1). With respect to the new NLP predictors, 44 368 (41%) individuals had no symptom or substance data in the 6 months prior to their index diagnosis. Derivation (n = 28 297) and validation sets (n = 63 854, figure 1) showed notable differences in terms of ethnic make-up and the spread of index diagnoses (eg, substance use disorder was more prevalent in the derivation set, see Supplementary table S6). Mean (SD) follow-up was 1590 (721) days with a significant difference between derivation and validation sets (derivation: mean [SD], 1896 [463]; validation: mean [SD], 1455 [772]). Overall 6-year risk of developing a psychotic disorder was 0.034 (95% CI, 0.033–0.036). Cumulative incidence (Kaplan–Meier failure function) for risk of development of psychotic disorders is presented in Supplementary figure S1.

Fig. 1.

Flowchart of study population.

Table 1.

Sample Characteristics

	No (%)
Variable	Study Population (n = 92 150)	Derivation Dataset (n = 28 297)	Validation Dataset (n = 63 853)
Age, mean (SD)a	33.6 (19.0)	34.8 (18.8)	33.0 (19.1)
Sexa
Female	46 741 (50.7%)	13 861 (49.0%)	32 880 (51.5%)
Male	45 410 (49.3%)	14 436 (51.0%)	30 974 (48.5%)
Ethnicitya
White	61 711 (67.0%)	16 700 (59.0%)	45 011 (70.5%)
Asian	4549 (4.94%)	1030 (3.64%)	3519 (5.51%)
Black	15 187 (16.5%)	6029 (21.3%)	9158 (14.3%)
Mixed	3805 (4.13%)	1183 (4.18%)	2622 (4.11%)
Other	6899 (7.49%)	3355 (11.9%)	3544 (5.55%)
Index diagnosis
CHR-P	445 (0.48%)	238 (0.84%)	207 (0.32%)
Anxiety disorders	25 323 (27.5%)	6765 (23.9%)	18 558 (29.1%)
Acute and transient psychotic disorders	1568 (1.70%)	552 (1.95%)	1016 (1.59%)
Bipolar disorders	3149 (3.42%)	1018 (3.60%)	2131 (3.34%)
Childhood onset disorders	12 332 (13.4%)	3351 (11.8%)	8981 (14.1%)
Developmental disorders	4645 (5.04%)	923 (3.26%)	3722 (5.83%)
Mental retardation	1640 (1.78%)	609 (2.15%)	1031 (1.61%)
Nonbipolar affective disorders	15 965 (17.3%)	5240 (18.5%)	10 725 (16.8%)
Personality disorders	3524 (3.82%)	1071 (3.78%)	2453 (3.84%)
Physiological disorders	6806 (7.39%)	1958 (6.92%)	4848 (7.59%)
Substance use disorders	16 754 (18.2%)	6572 (23.2%)	10 182 (15.9%)
Symptoms
Tearfulness	20 214 (21.9%)	13 835 (21.7%)	6379 (22.5%)
Appetite loss	13 653 (14.8%)	9322 (14.6%)	4331 (15.3%)
Weight loss	8623 (9.36%)	6002 (9.40%)	2621 (9.26%)
Insomnia	5115 (5.55%)	3401 (5.33%)	1714 (6.06%)
Poor insight	17 089 (18.5%)	12 000 (18.8%)	5089 (18.0%)
Guilt	9953 (10.8%)	6665 (10.4%)	3288 (11.6%)
Irritability	9049 (9.82%)	6259 (9.80%)	2790 (9.86%)
Delusions	5352 (5.81%)	3649 (5.71%)	1703 (6.02%)
Hopelessness	8883 (9.64%)	6117 (9.58%)	2766 (9.77%)
Disturbed sleep	25 786 (28.0%)	17 576 (27.5%)	8210 (29.0%)
Agitation	12 916 (14.0%)	9054 (14.2%)	3862 (13.6%)
Paranoia	13 212 (14.3%)	9201 (14.4%)	4011 (14.2%)
Substance use
Cannabis	13 604 (14.8%)	9271 (14.5%)	4333 (15.3%)
Cocaine	10 229 (11.1%)	6554 (10.3%)	3675 (13.0%)

^aMissingness values for ethnicity, sex, and age were 11.9%, 0.06%, and 0.02%, respectively.

Model Development

There were 1060 transitions to psychosis in the derivation dataset (raw counts stratified per index diagnosis are available in Supplementary table S7), of which 55 were observed in the CHR-P group (5%). The refined risk prediction model significantly predicted psychosis onset (likelihood ratio χ² test, 2769; P  < .001, table 2). No variables were selected out of the model via LASSO regularization. The LASSO penalty improves model performance at the expense of bias in parameter estimates (which reduces coefficient interpretability), therefore significance testing for individual predictors would not be appropriate.³⁶ Paranoia, delusions, and agitation were the strongest positive NLP-derived predictors of psychosis while hopelessness was the strongest negative one. The transdiagnostic model refined with NLP predictors showed good apparent prognostic performance (Harrell’s C index, 0.86, table 3), an increase of 0.05 from the Harrell’s C of the original model.

Table 2.

Characteristics of the Refined, Individualized, and Transdiagnostic Clinically Based Risk Prediction Model Employing NLP Predictors to Detect Individuals at Risk for Psychosis in EHRs

	Predictor	Hazard Ratio
Original model	Male sex	1.29
	Age	1.01
	Sex * Age	0.99
	Ethnicity
	White	Ref
	Asian	1.57
	Black	2.16
	Mixed	1.20
	Other	1.18
	Primary diagnosis
	CHR-P	Ref
	Anxiety disorders	0.16
	Acute and transient psychotic disorders	1.26
	Bipolar disorders	0.38
	Childhood-onset disorders	0.06
	Developmental disorders	0.07
	Mental retardation	0.07
	Nonbipolar affective disorders	0.22
	Personality disorders	0.17
	Physiological disorders	0.11
	Substance use disorders	0.15
New NLP symptoms and substance use
	Agitation	1.64
	Appetite loss	1.06
	Cannabis	1.13
	Cocaine	0.87
	Delusions	2.10
	Disturbed sleep	1.12
	Guilt	0.93
	Hopelessness	0.70
	Insomnia	1.05
	Irritability	1.05
	Loss of insight	1.02
	Paranoia	2.62
	Tearfulness	0.93
	Weight loss	1.14

Coefficients obtained via LASSO-regularised, multivariable Cox proportional hazards regression using the derivation dataset (n = 28 297).

Table 3.

Performance Measures for the Original Transdiagnostic Individualized Risk Prediction Model vs the NLP Model Refinement

	Original Transdiagnostic Model		Refined Model Including NLP Predictors
Performance Measure	Derivation	Validation	Derivation	Validation
Overall
Brier	0.028	0.021	0.085	0.061
R2	0.746	0.719	0.885	0.885
Discrimination
Harrell’s C (95% CI)	0.809 (0.795–0.822)	0.790 (0.775–0.806)	0.861 (0.849–0.873)	0.848 (0.838–0.858)
Calibration
Calibration slope	1	0.968	1	1.059

External Model Validation

There were 1662 transitions to psychosis in the external validation dataset, which far exceeds the minimum value of 100 events required for robust external validation.³⁷ The transdiagnostic model refined with NLP predictors still retained good prognostic performance when applied to unseen data (Harrell’s C 0.85), an increase of 0.06 from the Harrell’s C of the original model. The calibration slope coefficient of 1.06 (95% CI 1.03–1.09) indicated no major miscalibration issues. A sensitivity analysis found stronger model discriminatory performance in temporal external validation (Harrell’s C = 0.91) than geographic (Harrell’s C = 0.86; Supplementary table S8).

Discussion

To our knowledge this is the first study demonstrating the potential of applying automated methods such as NLP to EHRs to detect individuals at risk for psychosis. By incorporating NLP-derived data on symptoms and substance use, we refined a previously validated transdiagnostic, individualized, and clinically based risk prediction model. Model refinement considerably improved the external prognostic accuracy of the model to a good level (Harrell’s C 0.85) compared with an adequate level when using structured field data alone (Harrell’s C 0.79).

Efficient detection of individuals at CHR-P has been a neglected area of research despite its necessity for successful early intervention. This study progresses the field by demonstrating, for the first time, that advanced data-mining NLP methods³⁸ can improve prognostication of psychosis risk at large scale. Our NLP-refined model confers a substantial increase in external prognostic accuracy, with a Harrell’s C increment of 0.06. Harrell’s C indexes the probability that for any given case–control pair, the model will generate a higher predicted risk score for the case. Our refinement has effected an improvement from adequate to good, a level of prognostic accuracy exceeding that of the original CHR-P instruments (C-statistic 0.79).³⁹ Compared with CHR-P instruments, the NLP-based risk calculator produces reasonably well-calibrated and individualized estimates of risk (as opposed to group-level estimates), is automatable in EHRs and can be applied in large datasets. Furthermore, our risk calculator detects psychosis risk transdiagnostically outside the CHR-P designation.⁴⁰ This is crucial given recent evidence that a third of first episode psychosis cases do not evolve through a previous CHR-P stage.¹³ Indeed, the original risk calculator has already been externally validated, and shown to perform well, in other NHS Trusts that do not have CHR-P services.¹⁷ A recent review of psychosis risk prediction models found that clinical variables such as paranoia and unusual thought content consistently appear as significant predictors of psychosis.⁴¹ NLP techniques can extract symptom data at a fraction of the cost of individual patient recruitment. Prognostic performance of our NLP-based refinement of the transdiagnostic risk calculator also exceeds that achieved using harder-to-obtain neuroanatomical predictors (eg, grey matter volume), with accuracies ranging from 0.50 to 0.63.⁴² In a previous study, we found that the use of machine learning per se is not associated with improved prognostic accuracy.⁴³

The first step toward translating NLP tools into clinical benefits for patients is to apply these methods in larger risk cohorts to test their reproducibility.⁴⁴ This study represents the largest application of NLP in the area of predicting conversion to psychosis to date (the second largest study includes only 59 patients).⁴⁴ Our promising findings align with the existing NLP literature. For example, NLP-based automated speech analysis has been used to measure subtle, clinically relevant mental state changes in emerging psychosis.⁴⁵ Other studies have found that NLP-derived tools can identify symptom distributions in clinician notes beyond those captured by ICD codes, and that these domains usefully map onto Research Domain Criteria.⁴⁶ Our group has also confirmed that CRIS-NLP algorithms can reliably extract data on typically complex symptomatic domains such as negative psychotic symptoms or insight,^26,27 which has been replicated by an independent team.²⁸ These algorithms can also extract substance use data that predict longitudinal outcomes.⁴⁷ The accuracy of NLP-based estimates compared with gold-standard domains is further corroborated by their robust prognostic (ie, predicting the course of a condition) and predictive (ie, predicting the response to treatment) value.^28,46,48 NLP-derived data can also be incorporated into dynamic risk prediction models such as those recently used to predict psychosis onset risk.⁴⁹ For example, one could use NLP to extract dynamic treatment data relevant to CHR-P populations, such as exposure to cognitive behavioral therapy, which is not routinely recorded in structured fields.⁵⁰ This information could dynamically flow into risk predictions that are updated every time a patient’s record is updated with new information.

The NLP-refined transdiagnostic risk calculator is well suited for implementation in routine clinical care. First, our calculator represents the only available, pragmatic option for improving detection of individuals at risk of psychosis in secondary mental health care. The only existing alternative is to conduct extensive outreach campaigns that promote referrals based on clinicians’ suspicion of psychosis risk. This approach is inefficient because it dilutes risk enrichment (ie, refers more patients with only low risk for psychosis).⁵¹ Second, NLP-derived data were available for most at-risk individuals. Third, the NLP-refined model performed well in external validation efforts both temporally and geographically. The NLP algorithms used can be transferred to other sites with electronic health registers interrogable via CRIS for further external validation (eg, the Oxford Health NHS Foundation Trust). Fourth, this study refines an already well-performing model. Enhancing the benefits of clinical implementation through continual refinement of a given prognostic model is preferable to repeatedly developing new models from scratch that may never enter clinical routine.²¹ As the original risk calculator has already been piloted for implementation,¹⁸ the new NLP-refined model can be easily absorbed into clinical practice. Finally, this refined risk prediction model is designed to work around the way clinicians and mental health professionals enter text into EHRs. Future developments could implement an automated algorithm to trigger a prompt when all five baseline predictors are entered, taking into account any NLP symptom or substance use data entered prior to this point. For those (41%) without symptom or substance use information recorded prior to formal diagnosis, the original risk calculator can still be used. We have recently integrated the original version of this risk calculator in EHRs and prospectively piloted a real-time and real-world psychosis risk detection and alerting system.^18,52 This method leverages the CogStack platform, which is an open-source text extraction system.⁵² The CogStack platform offers full-text search of clinical data, real-time calculation of psychosis risk, early risk alerts to clinicians, and visual monitoring of patients over time.⁵² This method is highly transportable and can be easily deployed in NHS Trusts with a CRIS or CogStack platform. So far, the CRIS platform—including consenting procedures—is under expansion across 12 NHS Trusts in the UK, harnessing over 2 million deidentified patient records (https://crisnetwork.co/). This study also provides further empirical evidence supporting the expansion of EHRs in clinical psychiatry to facilitate precision and stratified medicine approaches on a global scale. Study limitations are appended in the eLimitations section.

Conclusions

Applying NLP techniques to EHR data recorded during routine clinical practice can facilitate robust research in large, representative samples of patients. NLP can add value to precision psychiatry by enhancing the prognostic accuracy of risk prediction models. Automatic text extraction from EHRs through NLP enhanced the transdiagnostic prognostic power achieved by a previously developed psychosis risk calculator. This can help to facilitate earlier detection of patients at risk of developing psychosis who require an assessment and specialized care, potentially improving outcomes of psychosis.

Author Contribution

The study was conceived by P.F.P. Data extraction and statistical analysis was performed by J.I. with support from C.C., R.P., M.P., and D.S. Reporting of findings were carried out by J.I. and P.F.P. All authors (J.I., C.C., D.S., M.P., R.S., P.F.P., H.B., and R.P.) contributed to study design, manuscript preparation, and approved the final version.

Ethics Approval

The CRIS data resource received ethical approval as an anonymized dataset for secondary analyses from Oxfordshire REC C (Ref: 18/SC/0372).

Funding

This study was supported by the King’s College London Confidence in Concept award from the Medical Research Council (MRC) (MC_PC_16048) to P.F.P. H.S., M.P., R.S., D.S. and C.C. received funding from the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London, which also supports the development and maintenance of the BRC Case Register. R.P. has received support from a Medical Research Council (MRC) Health Data Research UK Fellowship (MR/S003118/1) and a Starter Grant for Clinical Lecturers (SGL015/1020) supported by the Academy of Medical Sciences, The Wellcome Trust, MRC, British Heart Foundation, Arthritis Research UK, the Royal College of Physicians and Diabetes UK.

Data Availability

The data accessed by CRIS remain within an NHS firewall and governance is provided by a patient-led oversight committee. Subject to these conditions, data access is encouraged and those interested should contact RS (robert.stewart@kcl.ac.uk), CRIS academic lead.