Skip to main content
NCBI home page
Neurology logoLink to Neurology
. 2021 Sep 28;97(13):e1313–e1321. doi: 10.1212/WNL.0000000000012602

Association of Silent Cerebrovascular Disease Identified Using Natural Language Processing and Future Ischemic Stroke

David M Kent 1,✉, Lester Y Leung 1, Yichen Zhou 1, Patrick H Luetmer 1, David F Kallmes 1, Jason Nelson 1, Sunyang Fu 1, Chengyi Zheng 1, Hongfang Liu 1, Wansu Chen 1
PMCID: PMC8480402  PMID: 34376505

Abstract

Background and Objectives

Silent cerebrovascular disease (SCD), comprising silent brain infarction (SBI) and white matter disease (WMD), is commonly found incidentally on neuroimaging scans obtained in routine clinical care. Their prognostic significance is not known. We aimed to estimate the incidence of and risk increase in future stroke in patients with incidentally discovered SCD.

Methods

Patients in the Kaiser Permanente Southern California (KPSC) health system aged ≥50 years, without prior ischemic stroke, transient ischemic attack (TIA), or dementia/Alzheimer disease receiving a head CT or MRI between 2009 and 2019 were included. SBI and WMD were identified by natural language processing (NLP) from the neuroimage report.

Results

Among 262,875 individuals receiving neuroimaging, NLP identified 13,154 (5.0%) with SBI and 78,330 (29.8%) with WMD. The incidence of future stroke was 35.5 (95% confidence interval [CI] 34.0, 37.1) per 1,000 patient-years for patients with SBI: 21.1 (95% CI 20.6, 21.6) for patients with WMD and 7.6 (95% CI 7.4, 7.8) for patients without SCD. The crude hazard ratio (HR) associated with SBI was 3.34 (95% CI 3.19 to 3.50) and for WMD 2.57 (95% CI 2.49 to 2.66). With MRI-discovered SBI, the adjusted HR was 3.02 (95% CI 2.58 to 3.54) for those <65 years of age and 2.15 (95% CI 1.91 to 2.42) for those ≥65. With CT scan, the adjusted HR was 2.55 (95% CI 2.24 to 2.91) for those <65 and 1.81 (95% CI 1.71 to 1.92) for those ≥65. The adjusted HR associated with a finding of WMD was 1.76 (95% CI 1.70 to 1.83) and was not modified by age or imaging modality.

Discussion

Incidentally discovered SBI and WMD are common and associated with increased risk of subsequent symptomatic stroke, representing an important opportunity for stroke prevention.

Silent cerebrovascular disease (SCD), comprising silent brain infarction (SBI) and white matter disease (WMD), is commonly found incidentally on neuroimaging scans obtained in routine clinical care. SBI is believed to be pathophysiologically similar or identical to clinically evident stroke, but occurs with infarct locations in clinically less articulate parts of the brain. Epidemiologic studies based on screened cohorts indicate that SBI is far more common than clinically evident stroke.1-4 Among screened cohorts, these imaging findings are strong, independent risk factors for future stroke and dementia.1-3,5-7 However, their significance when discovered in routine care is unclear.

For both forms of SCD, there are no proven preventive treatments or guidelines regarding the initiation of risk factor–modifying therapies. While the American Heart Association/American Stroke Association has identified SBI as a major priority for new studies on stroke prevention,8,9 there are serious challenges for the study of SBI and WMD. Screening for SCD is not performed in routine care. When these lesions are identified on neuroimaging incidentally, patients may not be informed that they have SBI.10,11 Indeed, there are no ICD-9 codes for SBI or WMD, and it is generally not included in a patient's problem list or in structured fields of electronic health records (EHRs). This substantially impedes the study of these conditions.

In previous work, we developed a natural language processing (NLP) algorithm to identify individuals with incidentally discovered SBI and WMD through the automated review of neuroradiology reports obtained in clinical practice and demonstrated a high degree of accuracy with respect both to the findings on neuroimaging reports12 and also to the findings on the actual neuroimages when these are re-read.13 Herein, we port the algorithms into a large integrated health care system in southern California to examine the prognostic significance of NLP-identified, incidentally discovered SCD with respect to the future risk of stroke.

Methods

Environment

In this retrospective cohort study, we used health plan enrollees of Kaiser Permanente Southern California (KPSC), an integrated health care organization that serves 4.7 million individuals (approximately 19% of the region's population) at its 15 hospitals and 230+ medical offices. Enrollees' demographics and socioeconomic status are representative of the residents residing in the region.14 Implemented between 2004 and 2008 at KPSC, HealthConnect, Kaiser Permanente's comprehensive electronic medical record, is one of the largest private EHR systems in the world. This system integrates all aspects of care, including inpatient, emergency department, outpatient, pharmacy, and laboratory services, as well as billing and claims. Each Kaiser Permanente health plan enrollee is assigned a unique medical record number that allows for linkage to all the information relevant to the patient. All the structured data used in the current study were captured from the KPSC Research Data Warehouse, which incorporates information from HealthConnect and 20+ legacy systems with commonly used variables for research purposes. Clinical notes were extracted from Clarity, a data repository of HealthConnect. The study protocol was approved by Tufts Medical Center's and KPSC's institutional review board.

Population

We included individuals age ≥50 years enrolled in the KPSC health system who received head neuroimaging (CT, MRI) for a nonstroke indication between 2009 and 2019 and who had no history of ischemic stroke (ICD-9: 433.x, 434.x, 436.x, 437.1x, 437.9x, 438.x, 342.x) or dementia/AD (ICD-9 290.x, 291.2, 292.82, 294.1x, 294.2x, 331.0). Patients with transient ischemic attack (TIA) (ICD-9: 435.x) or hemiplegia and hemiparesis (ICD-9: 342.x) were also excluded to ensure a stroke-free cohort. The corresponding ICD-10 codes used to exclude patients with history of conditions mentioned above are available in eAppendix 1 (links.lww.com/WNL/B487). If there were multiple neuroimaging studies, the first study was considered the index scan. In order for neuroimaging evidence of cerebral infarction to be considered “silent,” individuals were only included in the study if they did not acquire a new ICD code for a diagnosis of cerebral infarction within 60 days after the index scan. Patients who were not actively enrolled in the KPSC health plan on the index date or not continuously enrolled in the prior 12 months were also excluded (a gap of 45 days or less was allowed).

Identification of Patients With SCD

An NLP algorithm designed at Mayo Clinic and Tufts Medical Center was applied to neuroimaging reports associated with these index scans to identify individuals with documented SBI or WMD.12 As described in prior work, these algorithms adopted the open source NLP pipeline MedTagger for generic NLP processing and task-specific knowledge engineering (coding of specific words and phrases referencing SBIs and WMD) to yield identification of SBI and WMD that was on par with human readers of the neuroimaging reports.12 This NLP algorithm achieved F scores of 0.86 and 0.89 in SBI and WMD on 490 reports from KPSC when it was initially implemented. After retraining on those reports, the F scores were enhanced to 0.93 and 0.92, respectively, on a separate test dataset (n = 490). The algorithm was found to perform consistently over time and across different demographic groups. Keyword-based features for the identification of SBI and WMD are shown in eAppendix 2 (links.lww.com/WNL/B488).

Follow-up

For each patient in the cohort, follow-up started on the date of index scan and ended with the earliest of the following events: disenrollment from the health plan, end of the study (December 31, 2019), death, or stroke (outcome).

Outcome Definitions

The primary outcome of this study was stroke. Based on a systematic review by Andrade et al.,15 we used a highly specific, previously validated algorithm that includes the following ICD-9 codes: 433.x1, 434.x1, 436.x, 437.1x, 437.9x, 438.x. The positive predictive value (PPV) of this definition has been reported at 97%.16 This code specifies TIA with a PPV of 70% or higher.17 Equivalent ICD-10 codes used are shown in eAppendix 1 (links.lww.com/WNL/B487).

Statistical Analysis

Kaplan-Meier plots were used to present stroke-free survival in patients without SCD, with SBI, with WMD, and with both SBI and WMD. The differences in the distributions between patients with SBI (WMD) and without SBI (WMD) were assessed by the log-rank test. The overall and risk factor stratified crude incidence rates and the 95% confidence intervals (CIs) were calculated using Poisson regression models and reported as per 1,000 person-years of follow-up time. We examined the crude and adjusted association of SBI and of WMD with stroke using Cox proportional hazards regression models (function coxph() in R survival package).18,19 For adjusted effects, we included known cardiovascular risk factors for stroke based on prediction models in the literature,20,21 including the following covariates: age, sex, race (non-Hispanic White, Asian/Pacific Islander, African American, Hispanic, multiple/other/unknown), diabetes, hypercholesterolemia, history of smoking, mean systolic blood pressure (averaged over prior year, excluding extreme values less than 70 or greater than 200 to avoid including measurements from periods of critical illness), atrial fibrillation, carotid disease, congestive heart failure, peripheral arterial disease, use of antiplatelet therapy, and use of statin therapy. For each risk factor, the proportional hazards assumption was examined by the Schoenfeld residuals test (function cox.zph() in R survival package)18,19 and when it was violated, the effect was assessed during 4 time windows (61 days–≤1 year, 1–≤3 years, 3–≤5 years, and ≥5+ years) to allow changes over time. Interaction terms were selected on the basis of clinical judgement. In particular, we hypothesized that the effects of WMD and of SBI would vary on the basis of whether the lesions were discovered by MRI or by CT scan. We also anticipated that these lesions might have greater prognostic importance when discovered in younger vs older patients and so included interactions with age (< age 65 vs ≥ age 65). Because 96.5% (252,810/261,960) of patients had complete data for all variables included in our multivariable model, we used a complete case analysis and did not impute missing variables.

Sensitivity Analysis

Three sensitivity analyses examining stroke incidence rates were performed: (1) defining the outcome as either stroke (ICD-9 codes: 433.x1, 434.x1, 436.x, 437.1x, 437.9x, 438.x) or TIA (ICD-9 code: 435.x)—that is, the same definition that was used to exclude patients from the cohort; (2) using a more sensitive, but less specific stroke definition for the ischemic stroke outcome (ICD-9: 433.x, 434.x, 436.x, 437.1x, 437.9x, 438.x, 342.x15; (3) excluding patients who were on antithrombotic therapy or who had an indication for antithrombotic therapy. This analysis was performed so as to emulate the potential target population for a future study testing antithrombotic therapy. In this analysis, we excluded patients on oral anticoagulation, on antiplatelet agents, with atrial fibrillation, with moderate to severe valvulopathy, with a history of myocardial infarction or coronary artery disease, with congestive heart failure, or with a history of deep venous thrombosis or pulmonary embolism in the last year.

In addition, we performed a sensitivity analysis to ensure that our hazard ratios (HRs) were not unduly influenced by the inclusion of persons with outcomes that were related to the indications for the index scan. In this analysis, we excluded patients whose indication for the visit or scan was a “stroke risk factor” (such as diabetes or hypertension) and we extended the required follow-up from at least 61 days to at least 91 days to exclude very early stroke outcomes. This analysis specifically examined the influence of these exclusions on the HRs in the first year.

All the analyses were performed using SAS (version 9.4 for Unix; SAS Institute) except for the R packages mentioned previously.18,19 All computations and analyses carried out in R were based on R Version 3.6.0 (R Foundation).

Data Availability

The datasets generated or analyzed during the current study are not publicly available due to ethical standards. The authors do not have permission to share data.

Standard Protocol Approvals, Registrations, and Patient Consents

This study was approved by institutional review boards at KPSC, Mayo Clinic, and Tufts Health Sciences. Informed consent requirements were waived by the institutional review boards.

Results

Figure 1 presents a flow diagram that details the inclusion and exclusion of patients in the analysis cohort. A total of 261,960 individuals receiving brain neuroimaging, with a total 1,173,955 person-years of follow-up time, were included in our analysis cohort. The median follow-up time was 3.87 (with a range of 61 days to 11.00 years; 25th percentile: 1.76 years; 75th percentile: 6.94 years). A total of 66,187 (25.3%) patients received MRI and 195,773 (74.7%) received CT scan. SCD was identified in 83,712 (32.0%) including 13,054 (5.0%) with SBI and 77,919 (29.7%) with WMD. There were 13,907 strokes identified in follow-up, with a median time to event among those with stroke of 2.93 years (interquartile range 1.28 to 5.21). Table 1 describes patient characteristics in the total cohort and in those with SBI (regardless of WMD), with WMD (regardless of SBI), and with both SBI and WMD.

Figure 1. Flow Diagram Illustrating Patient Selection Into the Study.

Figure 1

Open in a new tab

KPSC = Kaiser Permanente Southern California.

Table 1.

Patient Demographic and Clinical Characteristics at Baseline (n = 261,960)

graphic file with name NEUROLOGY2021171700t1.jpg

Open in a new tab

As shown in Table 2, the crude stroke incidence rate (per 1,000 person-years) was 35.5 (95% 34.0 to 37.1) in patients with incidentally discovered SBI, 21.1 (95% CI 20.6 to 21.6) for patients with WMD, and 43.8 (95% 41.4, 46.2) for patients with both WMD and SBI. This compares to an incidence rate of only 7.6 (95% CI 7.4 to 7.8) in patients free of SCD. Stroke-free survival in those without SCD, with WMD only, with SBI only, and with both SBI and WMD are shown in Figure 2.

Table 2.

Incidence Rate per 1,000 Person-Years of Ischemic Stroke in All and in Subgroups (n = 261,960)

graphic file with name NEUROLOGY2021171700t2.jpg

Open in a new tab

Figure 2. Stroke-Free Survival in Patients Without Silent Cerebrovascular Disease (SCD), With Silent Brain Infarction (SBI), With White Matter Disease (WMD), and With Both SBI and WMD.

Figure 2

Open in a new tab

This Kaplan-Meier plot depicts stroke-free survival in patients without SCD (n = 178,248), with SBI (n = 13,054), with WMD, and with both SBI and WMD over the study period.

The crude HR was 3.34 (95% CI 3.19–3.50) for SBI and 2.57 (95% CI 2.49–2.66) for WMD. After adjustment, the predictive effect of SBI was found to be stronger in younger vs older patients and for MRI- vs CT-discovered lesions. With MRI, the average (global) adjusted HR over time was 3.02 (95% CI 2.58–3.54) for those < age 65 and 2.15 (95% CI 1.91–2.42) for those ≥ age 65. With CT scan, the average (global) adjusted HR over time was 2.55 (95% CI 2.24–2.91) for those < age 65 and 1.81 (95% CI 1.71–1.92) for those ≥ age 65. The adjusted HR associated with a finding of WMD was 1.76 (95% CI 1.70–1.83) and was not modified by age or imaging modality. The effect of SBI attenuated gradually over time, while the effect of WMD appeared constant. Table 3 shows the time-specific HRs using thresholds of 1 year, 3 years, and 5 years.

Table 3.

Crude and Adjusted Hazard Ratios (HRs) of Silent Brain Infarction (SBI) and White Matter Disease (WMD) for Ischemic Stroke, Overall and Stratified by Imaging Modality and Age (<65, 65+ Years of Age) (n = 261,960)

graphic file with name NEUROLOGY2021171700t3.jpg

Open in a new tab

eTable 1 (links.lww.com/WNL/B489) shows the crude and adjusted effects of other risk factors included in our model. In general, the predictive effects of all stroke risk factors were in the expected direction in the multivariable model. Patients on statins or antiplatelet agents at the time of the index scan had crudely higher risks of stroke, presumably due to confounding by indication, but after adjustment, the antiplatelet effect was null and the statin appeared protective for stroke.

Sensitivity Analysis

Stroke incidence patterns were largely the same across all sensitivity analyses with a roughly uniform relative increase in outcome rates when TIAs were included in the outcome (eTable 2A, links.lww.com/WNL/B490) or when a somewhat more sensitive definition for stroke was used (eTable 2B, links.lww.com/WNL/B490). Patients without any indications for antithrombotic therapy had overall lower stroke incidence: 28.9 (27.2, 30.7) per 1,000 person-years in those with SBI; 16.7 (16.1, 17.2) for those with WMD; 36.3 (33.6, 39.2) for those with both SBI and WMD; and 6.1 (5.9, 6.2) for those without SCD (eTable 2C, links.lww.com/WNL/B490). Excluding patients with stroke risk factors as indications for the scans or with less than 90 days follow-up (eTable 3, links.lww.com/WNL/B491) did not substantially alter the results compared to the base case (eTable 3, links.lww.com/WNL/B491).

Discussion

SCD is often incidentally detected in patients during neuroimaging evaluation. In this observational cohort involving over 250,000 participants and over 1 million person-years of follow-up, we found that the presence of incidentally discovered NLP-identified SBI conferred an approximately 3-fold increased risk of subsequent stroke. After adjustment for potentially confounding vascular risk factors, the presence of SBI was associated with an approximately 2-fold higher risk of future stroke. The adjusted HR associated with WMD was slightly less than that associated with SBI, while defining a much larger cohort of patients.

Both the crude and the adjusted relative risk conferred by SBI was almost exactly the same as that found in a meta-analysis of screened cohorts22 including the Cardiovascular Health Study,23 the Atherosclerosis Risk in Communities Study,24 Northern Manhattan Study,25 Rotterdam Scan Study,26 and the Framingham Offspring Study.26 The current analysis is the only study we know to have examined patients with SBI detected incidentally in routine care. Because neuroimaging screening is not currently performed, our findings may be more relevant to clinical practice than prior studies. The study is also far larger than any prior study, well more than an order of magnitude larger than the 13 studies combined in a prior meta-analysis (n < 15,000 total, with all included studies having <3,000 individuals). Our study also confirmed prior studies that patients with WMD have an increased risk of future stroke.

The degree of risk conferred by incidentally discovered SBI seems to be of a similar magnitude to that found in patients with a prior history of clinically evident stroke. For example, in patients with atrial fibrillation, a prior history of stroke is associated with an HR of ∼2 for recurrent stroke,27 comparable to the magnitude of the relative risk we observed with SBI. The effect of SBI and of WMD was far stronger than the adjusted effects of other stroke risk factors in our study (eTable 1, links.lww.com/WNL/B489); stroke incidence rates in patients with SBI were similar to patients 2 decades older without SCD (Table 2).

The overall stroke incidence in the population of patients with SBI of 3.6% per year is similar in magnitude to that seen in some clinical trials testing interventions found to be beneficial for secondary stroke prevention.28-30 This suggests that SBI might be a “stroke equivalent”; that such patients may be potential candidates for “secondary” stroke prevention; and that a clinical trial for stroke prevention should be feasible. That we were able to identify more than 13,000 patients with SBI and approximately 80,000 with WMD who are free of stroke and dementia in one health system suggests that neuroimaging reports might be a repository of a large reservoir of high-risk patients suitable for trial enrollment. The ability to identify these patients with SCD at scale creates a new opportunity to test stroke (and possibly dementia) prevention strategies.

Both the strengths and limitations of our study derive from its unique design, in contrast to the previously described screening studies.2,3 These are analogous to the strengths and limitations of “pragmatic” vs “explanatory” designs described for randomized clinical trials,31 with our study design falling on the extreme pragmatic end of this design continuum. That the study did not have precise and standardized definitions of SBI and WMD as in the prior studies22 may be considered a limitation but might also (from the pragmatic perspective) be considered a strength, since we defined incidentally discovered SBI and WMD based on neuroimages obtained in routine clinical care, relying on the implicit definitions used by practicing radiologists. This is arguably the population of interest for interventions that might be used in incidentally discovered SBI and WMD. While definitions between radiologists may vary and standardization may be poor, we showed that lesions identified using this method confer a similar increased risk compared to “research grade” reading. Our prior work also showed that findings of SCD reflected in clinical neuroimaging reports was a reasonable proxy for the presence vs the absence of SBI and WMD,13 compared to research-grade readings of the same neuroimages. Additionally, the prior screening studies examining the effects of SCD relied exclusively on MRI, whereas patients were included in our study if they received either CT or MRI. We found that MRI-detected SBI conferred a slightly stronger risk than CT-detected SBI, presumably due to improved diagnostic performance with MRI.

In contrast to SBI discovered in population-based screening, selection of patients for clinically indicated neuroimaging (e.g., because of falls, headaches, change in mental status) might also “bias” our study findings through collider or index event bias.32,33 Collider bias might be expected to attenuate causal effects, if the risk factors for obtaining neuroimages are congruent with the risk factors for stroke. Nevertheless, since the population is more closely related to the clinically relevant target of inference than screened populations (since screening is not generally done in clinical care), our study design is again more relevant in quantifying the predictive effects of SBI or WMD discovered in routine care.

While the NLP algorithm was developed on data from Mayo and from Tufts, and performed well on KPSC data, the algorithm was subsequently slightly refined on KPSC data to accommodate heterogeneity in reporting across settings. Deployment of the NLP in other health systems should ideally be preceded by similar testing and potentially refinement. The value of the NLP algorithm might be further enhanced by including information about WMD grade or SBI location, and we plan to investigate this in future work. Finally, we cannot completely rule out some misclassification of silent brain infarct, through the inadvertent inclusion of patients with clinical stroke. However, these errors should be limited through the use of a sensitive stroke definition as an exclusion criteria. The sensitivity analyses using more rigorous exclusion criteria (i.e., requiring at least 91 days of stroke-free follow-up postscan and excluding patients with any potentially stroke-related indications) did not change the results.

Incidentally discovered SBI and WMD are common in patients ≥ age 50 and are associated with substantial increases in the risk of subsequent symptomatic stroke. The incidence of subsequent stroke in this population appears to be comparable to the incidence of recurrent stroke in some clinical trial populations examining secondary prevention strategies. The ability to identify these patients at scale represents a large opportunity for stroke prevention.

Glossary

CI

confidence interval

HR

hazard ratio

EHR

electronic health record

ICD-9

International Classification of Diseases–9

KPSC

Kaiser Permanente Southern California

NLP

natural language processing

PPV

positive predictive value

SBI

silent brain infarction

SCD

silent cerebrovascular disease

TIA

transient ischemic attack

WMD

white matter disease

Appendix. Authors

Appendix.

Open in a new tab

Study Funding

This work was funded by an NIH grant (R01-NS102233).

Disclosure

The authors report no disclosures. Go to Neurology.org/N for full disclosures.

References

  • 1.Fanning JP, Wesley AJ, Wong AA, Fraser JF. Emerging spectra of silent brain infarction. Stroke. 2014;45(11):3461-3471. [DOI] [PubMed] [Google Scholar]
  • 2.Fanning JP, Wong AA, Fraser JF. The epidemiology of silent brain infarction: a systematic review of population-based cohorts. BMC Med. 2014;12:119. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Vermeer SE, Longstreth WT Jr, Koudstaal PJ. Silent brain infarcts: a systematic review. Lancet Neurol. 2007;6(7):611-619. [DOI] [PubMed] [Google Scholar]
  • 4.Gouw AA, van der Flier WM, Fazekas F, et al. Progression of white matter hyperintensities and incidence of new lacunes over a 3-year period: the Leukoaraiosis and Disability Study. Stroke. 2008;39(5):1414-1420. [DOI] [PubMed] [Google Scholar]
  • 5.Hijdra A, Verbeeten B, Verhulst JA. Relation of leukoaraiosis to lesion type in stroke patients. Stroke. 1990;21(6):890-894. [DOI] [PubMed] [Google Scholar]
  • 6.Miyao S, Takano A, Teramoto J, Takahashi A. Leukoaraiosis in relation to prognosis for patients with lacunar infarction. Stroke. 1992;23(10):1434-1438. [DOI] [PubMed] [Google Scholar]
  • 7.Fu JH, Lu CZ, Hong Z, Dong Q, Luo Y, Wong KS. Extent of white matter lesions is related to acute subcortical infarcts and predicts further stroke risk in patients with first ever ischaemic stroke. J Neurol Neurosurg Psychiatry. 2005;76(6):793-796. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Kernan WN, Ovbiagele B, Black HR, et al. Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2014;45(7):2160-2236. [DOI] [PubMed] [Google Scholar]
  • 9.Smith EE, Saposnik G, Biessels GJ, et al. Prevention of stroke in patients with silent cerebrovascular disease: a scientific statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2017;48(2):e44-e71. [DOI] [PubMed] [Google Scholar]
  • 10.Leung LY, Han PKJ, Lundquist C, Weinstein G, Thaler DE, Kent DM. Clinicians' perspectives on incidentally discovered silent brain infarcts: a qualitative study. PLoS One. 2018;13(3):e0194971. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Leung LY, Han PKJ, Lundquist C, Weinstein G, Thaler DE, Kent DM. Patients' responses to incidentally discovered silent brain infarcts: a qualitative study. J Patient Rep Outcomes. 2019;3(1):23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Fu S, Leung LY, Wang Y, et al. Natural language processing for the identification of silent brain infarcts from neuroimaging reports. JMIR Med Inform. 2019;7(2):e12109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Leung LY, Zhou Y, Fu S, et al. Risk factors for incidentally discovered silent brain infarcts and white matter disease in a real world cohort identified by artificial intelligence. BMC Neurol. In press 2021. [Google Scholar]
  • 14.Koebnick C, Langer-Gould AM, Gould MK, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Perm J. 2012;16(3):37-41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Andrade SE, Harrold LR, Tjia J, et al. A systematic review of validated methods for identifying cerebrovascular accident or transient ischemic attack using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(suppl 1):100-128. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Birman-Deych E, Radford MJ, Nilasena DS, Gage BF. Use and effectiveness of warfarin in Medicare beneficiaries with atrial fibrillation. Stroke. 2006;37(4):1070-1074. [DOI] [PubMed] [Google Scholar]
  • 17.Kokotailo RA, Hill MD. Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke. 2005;36(8):1776-1781. [DOI] [PubMed] [Google Scholar]
  • 18.A Package for Survival Analysis in R [computer program]. Version R Package Version 3.2-112021.
  • 19.Therneau TM, Grambsch PM. Modeling Survival Data: Extending the Cox Model . Springer; 2000. [Google Scholar]
  • 20.Flueckiger P, Longstreth W, Herrington D, Yeboah J. Revised Framingham stroke risk score, nontraditional risk markers, and incident stroke in a multiethnic cohort. Stroke. 2018;49(2):363-369. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Wolf PA, D'Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke. 1991;22(3):312-318. [DOI] [PubMed] [Google Scholar]
  • 22.Gupta A, Giambrone AE, Gialdini G, et al. Silent brain infarction and risk of future stroke: a systematic review and meta-analysis. Stroke. 2016;47(3):719-725. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Bernick C, Kuller L, Dulberg C, et al. Silent MRI infarcts and the risk of future stroke: the cardiovascular health study. Neurology. 2001;57(7):1222-1229. [DOI] [PubMed] [Google Scholar]
  • 24.Windham BG, Deere B, Griswold ME, et al. Small brain lesions and incident stroke and mortality: a cohort study. Ann Intern Med. 2015;163(1):22-31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Di Tullio MR, Jin Z, Russo C, et al. Patent foramen ovale, subclinical cerebrovascular disease, and ischemic stroke in a population-based cohort. J Am Coll Cardiol. 2013;62(1):35-41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Poels MM, Steyerberg EW, Wieberdink RG, et al. Assessment of cerebral small vessel disease predicts individual stroke risk. J Neurol Neurosurg Psychiatry. 2012;83(12):1174-1179. [DOI] [PubMed] [Google Scholar]
  • 27.Diener HC, Eikelboom J, Connolly SJ, et al. Apixaban versus aspirin in patients with atrial fibrillation and previous stroke or transient ischaemic attack: a predefined subgroup analysis from AVERROES, a randomised trial. Lancet Neurol. 2012;11(3):225-231. [DOI] [PubMed] [Google Scholar]
  • 28.Kent DM, Dahabreh IJ, Ruthazer R, et al. Device closure of patent foramen ovale after stroke: pooled analysis of completed randomized trials. J Am Coll Cardiol. 2016;67(8):907-917. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Kernan WN, Viscoli CM, Furie KL, et al. Pioglitazone after ischemic stroke or transient ischemic attack. N Engl J Med. 2016;374(14):1321-1331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Amarenco P, Bogousslavsky J, Callahan A III, et al. High-dose atorvastatin after stroke or transient ischemic attack. N Engl J Med. 2006;355(6):549-559. [DOI] [PubMed] [Google Scholar]
  • 31.Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ. 2009;180(5):e47-e57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Dahabreh IJ, Kent DM. Index event bias as an explanation for the paradoxes of recurrence risk research. JAMA. 2011;305(8):822-823. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Munafò MR, Tilling K, Taylor AE, Evans DM, Davey Smith G. Collider scope: when selection bias can substantially influence observed associations. Int J Epidemiol. 2018;47(1):226-235. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The datasets generated or analyzed during the current study are not publicly available due to ethical standards. The authors do not have permission to share data.

Articles from Neurology are provided here courtesy of American Academy of Neurology

RESOURCES