Natural Language Processing Applied to Clinical Documentation in Post-acute Care Settings: A Scoping Review

Abstract

Objectives:

To determine the scope of the application of natural language processing to free-text clinical notes in post-acute care and provide a foundation for future natural language processing–based research in these settings.

Design:

Scoping review; reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines.

Setting and Participants:

Post-acute care (ie, home health care, long-term care, skilled nursing facilities, and inpatient rehabilitation facilities).

Methods:

PubMed, Cumulative Index of Nursing and Allied Health Literature, and Embase were searched in February 2023. Eligible studies had quantitative designs that used natural language processing applied to clinical documentation in post-acute care settings. The quality of each study was appraised.

Results:

Twenty-one studies were included. Almost all studies were conducted in home health care settings. Most studies extracted data from electronic health records to examine the risk for negative outcomes, including acute care utilization, medication errors, and suicide mortality. About half of the studies did not report age, sex, race, or ethnicity data or use standardized terminologies. Only 8 studies included variables from socio-behavioral domains. Most studies fulfilled all quality appraisal indicators.

Conclusions and Implications:

The application of natural language processing is nascent in post-acute care settings. Future research should apply natural language processing using standardized terminologies to leverage free-text clinical notes in post-acute care to promote timely, comprehensive, and equitable care. Natural language processing could be integrated with predictive models to help identify patients who are at risk of negative outcomes. Future research should incorporate socio-behavioral determinants and diverse samples to improve health equity in informatics tools.

Post-acute care settings (ie, home health care, long-term care, skilled nursing facilities, and inpatient rehabilitation facilities)¹ provide opportunities for early hospital discharge by continuing health care services once patients are stabilized and discharged from the hospital.² A critical goal of post-acute care is to avoid hospital readmissions. This can be achieved through implementing care guidelines, identifying risk factors early, and collaborating with the interdisciplinary team.³ Post-acute care patients have conditions and functional limitations that may put them at risk for hospitalization if not managed effectively.⁴ Despite efforts to leverage post-acute care to prevent hospitalizations, hospitalization rates have not significantly improved.⁵ For example, 1 in 5 patients are hospitalized during their time in home health care.⁶ In addition, among nursing home residents with advanced illnesses (eg, dementia, congestive heart failure, and chronic obstructive pulmonary disease), more than half are hospitalized.⁴ There is a need in post-acute care settings to surface risk factors, implement timely interventions, and provide collaborative care to prevent hospitalizations and negative outcomes.

Post-acute care clinicians typically record vital information and communicate with the interdisciplinary team through documentation in the electronic health record (EHR).⁷ EHRs contain clinical documentation in structured (eg, controlled vocabulary, assessment drop-down menus, flowsheets) and unstructured (eg, free-text clinical notes) formats. Structured data forms have been linked with documentation burden,^8,9 given the need to read these forms and click through several screens and response options.¹⁰ Existing evidence supports that clinicians document most information about patients’ clinical and social characteristics, risk factors, medications, follow-up recommendations, and care guidelines in unstructured free-text clinical notes.^11,12 Patients in post-acute care settings typically require several services (eg, nursing, medical, physical therapy, occupational therapy, social work, nutrition, and speech therapy⁶) and are seen by multiple clinicians per day; each writing a free-text clinical note about the patient encounter. This generates a large number of clinical notes for each patient during their time in post-acute care. Due to time constraints, clinicians may not be able to read all free-text clinical notes, leaving valuable information undiscovered.¹³ Further, free-text clinical notes include varied descriptions that may represent the same concept due to differences in clinician documentation styles, abbreviations, and misspellings.¹⁴ This poses challenges in clinical practice and research, limiting the ability to analyze unstructured data on a large scale.¹⁵ Addressing these challenges is essential to improve patient care and prevent negative outcomes.

Natural language processing (NLP) is an innovative informatics method that can be used to leverage information in free-text clinical notes to improve care and outcomes. NLP refers to computer-based algorithms used to identify and process language for further analysis¹⁶ and can be applied to free-text clinical notes to extract information on a large scale.^17,18 For example, NLP has been used to extract information about medications,^19–21 symptoms,^18,22 smoking history,²³ and cancer staging²⁴ from samples as large as approximately 2 million clinical notes.¹⁸ This information is needed to support care coordination,²⁵ understand risk factors for hospitalization,²⁶ and can be used to promote timely and comprehensive interventions. Further, NLP has been used to inform the development of clinical decision support systems that identify patients at risk of declining health status and assist clinicians in preventing hospitalization and mortality.²⁷ The recent development of new informatics tools that use artificial intelligence and NLP techniques, like ChatGPT, make this study well-timed, especially as these tools are applied in health care settings, such as post-acute care.

Given the potential benefits of this novel method, NLP in post-acute care settings is emerging; however, the extent to which NLP is used in post-acute care remains unclear. Therefore, this review aimed to determine the scope of the application of NLP to free-text clinical notes in post-acute care and provide a foundation for future NLP-based research in these settings.

Methods

This scoping review was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines.²⁸ Study selection and data extraction were performed using Covidence (www.covidence.org), a web-based tool designed to facilitate these processes. As reporting standards differ among NLP studies, a formal quality appraisal was not conducted. Rather, we developed a data extraction template and evaluated the overall quality of each study based on NLP quality indicators cited in prior systematic^22,29,30 and integrative³¹ reviews that focused on NLP. A study protocol was developed a priori and listed in the Open Science Framework database (https://osf.io/pc3b5/).

Eligibility Criteria

The search included articles from database inception through February 2023. Inclusion criteria were (1) quantitative studies that used NLP in post-acute care settings (ie, home health care, skilled nursing facilities, long-term care, inpatient rehabilitation facilities), (2) studies that analyzed clinical documentation, (3) studies written in English or translated to the English language, and (4) studies that had the full text available. Studies were excluded if they were review articles, abstracts, qualitative studies, study protocols, dissertation studies, editorials, psychometric studies, or case reports.

Search Strategy

Three databases were searched (PubMed, Cumulative Index of Nursing and Allied Health Literature (CINAHL), and Embase) in February 2023 to identify potentially relevant studies related to NLP applied in post-acute care settings. Reference lists of included studies were manually searched for additional relevant articles. Search terms describing NLP and post-acute care settings included a combination of Medical Subject Heading (MeSH) vocabulary and keywords. Table 1 presents the full search strategies applied in PubMed, CINAHL, and Embase.

Table 1.

Search Strategies Used to Retrieve Articles

Database	Search Strategy	Publications Retrieved
PubMed	((“Natural language processing” [Mesh] OR “natural language processing” [tiab] OR “NLP” [tiab] OR “text mining” [tiab]) AND (“Subacute care” [Mesh] OR “post acute care” [tiab]))	51 2/21/2023
	OR ((“Natural language processing” [Mesh] OR “natural language processing” [tiab] OR “NLP” [tiab] OR “text mining” [tiab]) AND (“Home Health Nursing” [Mesh] OR “Home Care Agencies” [Mesh] OR “Home Care Services” [Mesh] OR “home health nursing” [tiab] OR “home care agenc*” [tiab] OR “homecare agenc*” [tiab] OR “HHA” [tiab] OR “home care services” [tiab] OR “homecare services” [tiab] OR “home health care” [tiab] OR “home healthcare” [tiab] OR “home health” [tiab] OR “home care” [tiab] OR “homecare” [tiab] OR “HHC” [tiab] OR “home care services” [tiab] OR “homecare services” [tiab])) OR ((“Natural language processing” [Mesh] OR “natural language processing” [tiab] OR “NLP” [tiab] OR “text mining” [tiab]) AND (“Skilled Nursing Facilities” [Mesh] OR “skilled nursing facilit*” [tiab] OR “SNF” [tiab] OR “extended care facilit*” [tiab])) OR ((“Natural language processing” [Mesh] OR “natural language processing” [tiab] OR “NLP” [tiab] OR “text mining” [tiab]) AND (“long term care facilit*” [tiab] OR “long-term care facilit*” [tiab] OR “long-term care” [tiab] OR “long term care” [tiab])) OR ((“Natural language processing” [Mesh] OR “natural language processing” [tiab] OR “NLP” [tiab] OR “text mining” [tiab]) AND (“inpatient rehabilitation facility*” [tiab] OR “IRF” [tiab]))
CINAHL	((MH “Natural language processing”) OR TI(“natural language processing” OR “NLP” OR “text mining”) OR AB(“natural language processing” OR “NLP” OR “text mining”)) AND ((MH “Subacute care”) OR TI(“subacute care” OR “post acute care”)) OR ((MH “Natural language processing”) OR TI(“natural language processing” OR “NLP” OR “text mining”) OR AB(“natural language processing” OR “NLP” OR “text mining”))	28 2/21/2023
	AND ((MH “Home Health Care”) OR TI(“home health nursing” OR “home care agenc*” OR “homecare agenc*” OR “HHA” OR “home care services” OR “homecare services” OR “home health care” OR “home healthcare” OR “home health” OR “home care” OR “homecare” OR “HHC” OR “home care services” OR homecare services”) OR AB(“home health nursing” OR “home care agenc*” OR “homecare agenc*” OR “HHA” OR “home care services” OR “homecare services” OR “home health care” OR “home healthcare” OR “home health” OR “home care” OR “homecare” OR “HHC” OR “home care services” OR homecare services”)) OR ((MH “Natural language processing”) OR TI(“natural language processing” OR “NLP” OR “text mining”) OR AB(“natural language processing” OR “NLP” OR “text mining”)) AND ((MH “Skilled Nursing Facilities”) OR TI(“skilled nursing facilit*” OR “SNF” OR “extended care facilit*”) OR AB(“skilled nursing facilit*” OR “SNF” OR “extended care facilit*”)) OR ((MH “Natural language processing”) OR TI(“natural language processing” OR “NLP” OR “text mining”) OR AB(“natural language processing” OR “NLP” OR “text mining”)) AND ((MH “Long Term Care”) OR TI(“Long term care facilit*” OR “long-term care facility” OR “long- term care” OR “long term care”) OR AB(“Long term care facilit*” OR “long-term care facility” OR “long-term care” OR “long term care”)) OR ((MH “Natural language processing”) OR TI(“natural language processing” OR “NLP” OR “text mining”) OR AB(“natural language processing” OR “NLP” OR “text mining”)) AND (TI(“Inpatient rehabilitation facilit*” OR “IRF”) OR AB(“Inpatient rehabilitation facilit*” OR “IRF”))
Embase	((‘natural language processing’:ti,ab OR ‘NLP’:ti,ab OR ‘text mining’:ti,ab) AND (‘home health nursing’:ti,ab OR ‘home care agencies’:ti,ab OR ‘home care services’:ti,ab OR ‘home health nursing’:ti,ab OR ‘home care agenc*’:ti,ab OR ‘homecare agenc*’:ti,ab OR ‘HHA’:ti,ab OR ‘home care services’:ti,ab OR ‘homecare services’:ti,ab OR ‘home health care’:ti,ab OR ‘home health’:ti,ab OR ‘home care’:ti,ab OR ‘homecare’:ti,ab OR ‘HHC’:ti,ab OR ‘home care services’:ti,ab OR ‘homecare services’:ti,ab)) OR ((‘natural language processing’:ti,ab OR ‘NLP’:ti,ab OR ‘text mining’:ti,ab) AND (‘skilled nursing facilit*’:ti,ab OR ‘SNF’:ti,ab OR ‘extended care facilit*’:ti,ab)) OR ((‘natural language processing’:ti,ab OR ‘NLP’:ti,ab OR ‘text mining’:ti,ab) AND (‘long term care facilit*’:ti,ab OR ‘long-term care’:ti,ab OR ‘long term care’:ti,ab)) OR ((‘natural language processing’:ti,ab OR ‘NLP’:ti,ab OR ‘text mining’:ti,ab) AND (‘inpatient rehabilitation facilit*’:ti,ab OR ‘IRF’:ti,ab))	56 2/21/2023

Selection of Sources of Evidence

All potentially relevant studies were imported to EndNote X9 software and deduplicated using the Bramer method.³² To determine eligibility, 2 reviewers (D.S., M.H.) independently screened all studies by title and abstract using Covidence (www.covidence.org). A third reviewer was consulted to resolve discrepancies (M.T.). The same 2 reviewers independently screened the full-text studies and documented reasons for exclusion. Discrepancies were resolved in consultation with the third reviewer (M.T.).

Data Extraction and Synthesis

The data extraction template was created by the authors (D.S., M.H., M.T.) who determined the appropriate variables to extract. The variables of interest for data extraction included authors, year of publication, country, study design, purpose, setting, study sample and characteristics (ie, age, sex, race, ethnicity), data source, total number of clinical notes analyzed, total number of distinct patients, note type and author, NLP methods, NLP preprocessing, standardized terminologies used, variables extracted by NLP, NLP performance metrics, and NLP stage (ie, development or application). Quality indicators included clarity of the study purpose, adequacy of the description of the NLP methods, information related to the number of clinical notes analyzed, and performance metrics reported.^22,29–31 In each quality indicator domain, studies could receive a “yes” response, meaning that they provided sufficient detail about the quality indicator, or a “no” response, meaning that they did not report adequate or any information. Findings were summarized in tables to identify patterns and facilitate data synthesis.

Results

Selection of Sources of Evidence

A summary of the study selection process is provided in the PRISMA flow diagram (Figure 1). The initial search yielded 135 potentially eligible studies. After deduplication, 79 studies were included for the title and abstract screening. Twenty-nine studies were included in the full-text review. After 8 studies were excluded, 21 studies met eligibility criteria and were included in this review.

Fig. 1.

PRISMA flow diagram. Summary of article selection process for the current scoping review.

Characteristics of Sources of Evidence

Characteristics of the 21 included studies^{18,26,33–51} are presented in Table 2. The 21 studies were published between 2015 and 2023, representing data from 12,847 to 2,610,767 clinical notes for 908 to 89,459 distinct patients. Most studies (n = 19; 90.5%)^{18,26,33–36,39–51} took place in home health care settings and examined symptoms and risk factors for negative outcomes. The remaining 2 studies^37,38 (9.5%) took place in long-term care settings (eg, assisted living and nursing homes). These studies investigated suicide mortality in long-term care by using NLP to extract characteristics about suicide deaths from free-text clinical documentation that were not captured in structured data forms.^37,38 No studies were conducted in skilled nursing facilities or inpatient rehabilitation facilities. All studies were done in the United States except for 1 study conducted in Finland.³⁶

Table 2.

Characteristics of Sources of Evidence

Authors, year	Country	Study Purpose	Study Setting	Study Population; Characteristics (Age, Gender, Race, and Ethnicity) if Specified	Study Design	Data Source and Type	Total Number of Clinical Notes Analyzed	Total Number of Distinct Patients	Note Type and Author
Bjarnadottir et al.33	United States	To examine documentation of sexual orientation and gender identity in HHC nurses’ narrative notes.	HHC	Patients who received HHC services in Manhattan in 2015	Retrospective observational	EHR, unstructured data	862,715	20,447	Narrative, referral, and coordination of care notes; nurses
Chae et al.34	United States	To identify patients with heart failure in HHC with poor self-management using NLP.	HHC	Patients with heart failure who received HHC services from one of the largest HHC agencies in the Northeastern United States between 2015 and 2017	Retrospective observational	EHR, unstructured data	~2.3 million	Not specified	Visit and care coordination notes; clinicians
Chae et al.35	United States	To identify HHC patients with heart failure who have poor self-management using NLP.	HHC	Patients admitted to HHC after being discharged from the hospital with a diagnosis of heart failure between 2015 and 2017; older adults (mean age 81.56 years), 60.9% male, 13.52% non-Hispanic Black, 37.30% non-Hispanic White, 11.56% Hispanic, 37.62% Asian/Other/Unknown/Native Hawaiian or Pacific Islander	Retrospective observational	EHR, structured and unstructured data	353,718	9710	Visit and care coordination notes; nurses, social workers, physical and occupational therapists
Harkanen et al.36	Finland	To identify terms that are related to the most common contributing factors to medication administration incidents in free-text narratives.	HHC	Patients with chronic illnesses, disabilities, or recovering from acute illnesses, receiving HHC services from the Helsinki Health Center	Retrospective cohort	Medication administration incident reports from the HaiPro incident reporting system; structured and unstructured data	19,725	Not specified	Incident reports; nurses, supervisors, students, social workers, physicians
Lohman et al.37	United States	To evaluate the association between state policies about availability, regulation, and cost of LTC and suicide mortality over a 5-year time period.	LTC	Text narratives describing suicide deaths of older adults aged 55 years and older in LTC facilities reported between 2010 and 2015	Longitudinal ecological	National Violent Death Reporting System (2010–2015); unstructured data	Not specified	25,040	Narratives providing detailed circumstances associated with a suicide that may not be accounted for in categorical data, such as the content of a suicide note or contributing circumstances preceding death; medical examiners
Mezuk et al.38	United States	To estimate the number of suicides associated with residential LTC and identify whether machine learning tools can improve suicide surveillance data.	LTC	Adults aged 55 years and older in LTC facilities in 27 states who died by suicide or undetermined cause; 77.6% male, 90% non-Hispanic White	Cross-sectional	National Violent Death Reporting System (2003–2015); unstructured data	47,759	47,759	Narrative reports; coroners, medical examiners
Pesko et al.39	United States	To evaluate whether communication failures between HHC nurses and physicians during an episode of home care after hospital discharge are associated with hospital readmission, stratified by patients at high and low risk of readmission.	HHC	Patients with congestive heart failure who received HHC services from VNS Health from 2008 to 2009; mean age 80.46 years, 64% female, 54% White, 22% Black, 19% Hispanic, 5% Other/Unknown	Retrospective	EHR, CMS claims; structured and unstructured data	Not specified	Not specified	Free-text responses; nurses
Popejoy et al.40	United States	To quantify care coordination by identifying care coordination activities used by Aging in Place nurse care coordinators and HHC nurses.	HHC	Two groups of patients: (1) patients admitted to a HHC agency between 1999 and 2002 for enhanced Aging in Place Care, (2) patients who received traditional HHC without enhanced care coordination between 2003 and 2005	Cross-sectional	EHR; structured and unstructured data	128,135	908	Care coordination; nurses
Press et al.41	United States	To identify failed communication attempts between HHC nurses and physicians using NLP to identify predictors of communication failure and assess the association with hospital readmission.	HHC	Medicare beneficiaries with congestive heart failure who received HHC services from VNS Health in 2008 and 2009; mean age 81 years, 62% female, 58% White, 19% African American, 18% Hispanic	Retrospective cohort	EHR, CMS claims; structured and unstructured data	12,847	5698	Free-text comments when attempting to contact physicians; nurses
Song et al.42	United States	To extract factors from clinical notes to more accurately describe a patient’s condition regarding wounds to build predictive models to identify risk of wound infection—related hospitalization.	HHC	Patients who received HHC services from VNS Health in 2014; mean age 67.6, 56.7% female, 7% Asian or Pacific Islander, 23.5% Black, 20.6% Hispanic, 48.9% White	Retrospective secondary data analysis	EHR; structured and unstructured data	2,610,757	112,789	Visit and care coordination notes; not specified
Song et al.43	United States	To compare the predictive performance of 4 risk models for hospitalization and emergency department visits in HHC.	HHC	Patients who received HHC services from VNS Health, largest not-for-profit HHC organization in the northeastern United States, mean age 78.8 years, 64% female, 56.7% non-Hispanic White, 21.8% non-Hispanic Black, 16% Hispanic, 5.5% Other	Retrospective cohort	EHR, structured and unstructured data	2,321,977	66,317	Visit notes and care coordination; nurses, social workers, physical therapists, occupational therapists
Song et al.44	United States	To develop an NLP algorithm to identify concerning language for HHC patients’ risk of hospitalization or emergency department visit.	HHC	Patients admitted to HHC at VNS Health, a large urban HHC organization in the northeastern United States between 2015 and 2017	Retrospective cohort	EHR, structured and unstructured data	2,321,977	66,317	Visit and care coordination notes; note author not specified
Song et al.45	United States	To identify risk factor clusters in HHC and determine if clusters are associated with emergency department visits or hospitalizations.	HHC	Patients who received HHC services between 2015 and 2017; mean age 78.8 years, 64% female, 5.62% Asian, 16.8% Black, 12.9% Hispanic, 64.3% White	Retrospective observational cohort study	EHR, structured and unstructured data	2,321,977	61,454	Visit and care coordination notes; nurses, physical therapists, occupational therapists social workers
Topaz et al.46	United States	To develop and evaluate an open-source software (called NimbleMiner) that allows clinicians to interact with word embedding models with a goal of creating lexicons of similar terms.	HHC	Patients treated by clinicians at one of the largest homecare agencies in the United States (located in New York, NY) during 2015	Case study	EHR, unstructured data	1,149,586	89,459	Clinical notes ranged from lengthy admission notes (often written by an RN) to shorter progress notes (eg, physical therapy progress notes); nurses, physical therapists, occupational therapists, social work
Topaz et al.47	United States	To describe the general system architecture and user interface and classify fall-related information (including fall history, fall prevention interventions, and fall risk) in homecare visit notes.	HHC	Patients treated by clinicians of the largest homecare agency in the United States (located in New York, NY) during 2015	Cross-sectional	EHR, unstructured data	1,149,586	89,459	Visit notes; nurses, physical therapy, occupational therapy, social workers
Topaz et al.48	United States	To develop an NLP algorithm to identify common neuropsychiatric symptoms of Alzheimer’s disease and related dementia in free-text clinical notes and describe symptom clusters and emergency department visits and hospitalization rates.	HHC	Patients in New York City with any diagnoses admitted for post-acute HHC services at the study organization during 2014; mean age 70.8 years, 60.8% female, 43% White, 27% Black or African American, 24% Hispanic/Latino, 6% Asian	Retrospective cohort	EHR, structured and unstructured data	2,610,757	89,459	Visit and care coordination notes; nurses, physical therapists, occupational therapists, social workers
Topaz et al.26	United States	To identify patients at high risk for emergency department visits or hospitalizations using HHC clinical notes.	HHC	Patients who received HHC services from the largest nonprofit HHC agency in New York, NY in 2014; mean age 70.8 years, 60.8% female, 43% White, 27% Black or African American, 24% Hispanic/Latino, and 6% Asian	Retrospective cohort	EHR, structured and unstructured data	727,676	89,459	Visit notes; nurses, physical therapists, occupational therapists, social workers
Topaz et al.18	United States	To identify documentation of 7 common symptoms (anxiety, cognitive disturbance, depressed mood, fatigue, sleep disturbance, pain, well-being) in HHC narrative nursing notes using NLP and examine the association between symptoms and ED visits or hospitalizations.	HHC	Patients admitted to the largest not-for-profit HHC provider in the United States, VNS Health, in 2014; mean age 70.8 years, 60.8% female, 43% White, 27% Black or African American, 24% Hispanic/Latino, and 6% Asian	Retrospective cohort	EHR; structured and unstructured data	2,610,767	89,825	Visit and care coordination; nurses
Woo et al.49	United States	To develop an NLP algorithm to identify urinary tract infection —related information in nursing notes.	HHC	Patients treated by clinicians of the largest nonprofit home care agency in the United States (located in New York, NY) in 2014	Cross-sectional	EHR; structured and unstructured data	2,610,757	89,459	Visit and care coordination; nurses
Woo et al.50	United States	To develop an NLP algorithm to identify wound infection—related information from nursing notes, estimate wound infection prevalence in the HHC setting, and describe related patient characteristics by linking NLP-identified wound infections to structured data in HHC.	HHC	Patients who received HHC services from the largest nonprofit homecare agency in the United States in 2014; 57% female, 22% Black, 22% Hispanic, 52% White, 7% Asian	Retrospective cohort	EHR; structured and unstructured data	2,610,757	89,459	Visit notes and care coordination notes; nurses
Zolnoori et al.51	United States	To develop and test an NLP algorithm that identifies reasons for delayed visits in HHC free-text clinical notes and describes reasons for delayed visits.	HHC	Patients admitted to the largest not-for-profit urban HHC agency in the United States during the calendar year of 2019 following a hospitalization; 61.8% older than 65 years, 57.9% female, 42.1% male, 43.5% non-Hispanic White, 23.9% non-Hispanic Black, 23.5% Hispanic, 9.2% Other	Retrospective cohort	EHR, structured and unstructured data	118,767	45,390	Nursing notes, intake clinical comment notes, insurance and other additional information notes, telephone communication notes, 17 other less frequent note categories; nurses, other HHC admission staff

CMS, Centers for Medicare and Medicaid Services; ED, emergency department; HHC, home health care; LTC, long-term care; RN, registered nurse; VNS, visiting nurse service.

Study purposes, samples, and data sources

Study purposes varied and used NLP to identify a range of concepts. Six studies^{18,34,42,48–50} (28.6%) used NLP to identify symptom information in free-text clinical notes. Among these studies, symptoms related to wound infections,^42,50 urinary tract infections,⁴⁹ dementias,⁴⁸ heart failure,³⁴ and general mood and well-being¹⁸ were identified. Six studies^{33,36–38,46,47} (28.6%) used NLP to extract language related to falls,^46,47 suicide deaths,^37,38 medication administration incidents,³⁶ and sexual orientation and gender identity.³³ Eighteen studies^{18,26,34–38,41–51} (85.7%) used NLP to examine risk factors for negative outcomes. Among these negative outcomes were acute care utilization (ie, emergency department visits and hospitalizations),^{18,26,42–48} medication errors,³⁶ and suicide mortality.^37,38 These studies examined risk factors across several domains, including physiological (eg, general signs and symptoms,^35,43,44 increased pain,⁴⁵ impaired skin integrity,⁴⁵ and multimorbidity),⁴⁵ psychosocial (eg, emotional and behavioral status,⁴³ impaired cognitive functioning),⁴⁵ environmental (eg, living arrangements,^26,43 and medical device use),²⁶ and health-related behaviors (eg, health care utilization,²⁶ medication regimen,⁴³ and self-management).³⁵ Among these studies, only one⁴⁵ used cluster analysis to evaluate the aggregate effect of multiple risk factors on negative outcomes rather than risk factors in isolation. Four studies^39–41,51 (19%) used NLP to evaluate care coordination activities, including communication between members of the health care team^39,41 and reasons for delayed home health care visits.⁵¹

Most studies (n = 17; 81%)^{18,26,33,36–38,40,42–51} focused on general patient populations. The remaining 4 studies^34,35,39,41 (19%) focused on patients with heart failure. Twelve studies^{18,26,35,38,39,41–43,45,48,50,51} (57%) reported age, sex, race, and ethnicity data for the study samples. Among these studies, most (n = 10)^{18,26,35,39,41–43,45,50,51} involved adults aged 65 years and older. Most studies (n = 10)^{18,26,39,41–43,45,48,50,51} included samples with more than 50% of participants who reported as female. Regarding race and ethnicity information, 6 studies involved samples composed of mostly (ie, greater than 50%) white participants,^{38,39,41,43,45,50} with 1 sample composed of as many as 90% white participants.³⁸ The remaining 6 studies^{18,26,35,42,48,51} included samples of roughly 40%−48% white participants, with the remainder of these samples consisting of Asian, Black, and Hispanic participants.

Regarding data sources, most studies (n = 18; 85.7%)^{18,26,33–35,39–51} used EHR data. Two studies used data from the Medicare Provider Analysis and Review file from Centers for Medicare and Medicaid Services claims data.^39,41 The remaining 3 studies (14.3%) used data from national repositories, including a National Medication Incident Reporting System³⁶ and the National Violent Death Reporting System.^37,38 Seven studies^{18,33,39–41,49,50} (33.3%) analyzed notes written only by nurses, and 8 studies (38%) examined notes written by nurses and other disciplines, including social workers, physical therapists, occupational therapists,^{26,35,43,45–48} physicians, supervisors, students,³⁶ and admission staff.⁵¹ Two studies^37,38 (9.5%) evaluated notes written by medical examiners. The remaining 3 studies did not specify the discipline of the note authors.^34,42,44

NLP methods, performance evaluation, and stage

Details about NLP methods, evaluation, and stage are provided in Table 3. Most studies (n 19; 90.5%)^{18,26,33–44,47–51} used keywords to develop a vocabulary, trained the NLP algorithm on a manually annotated set of clinical notes, expanded the vocabulary through interaction between humans and the NLP software, and classified cases based on the outcome of interest. Of these studies, most (n = 10)^{18,34,35,42,44,46–50} used NimbleMiner, an open-source software that uses a word embedding approach for comparable term detection in clinical notes.⁴⁶ Other software packages used included Statistical Analysis System (SAS) Text Miner tool within SAS software³⁶ and Python Scikit-Learn^37,38 and Python Natural Language Toolkit³³ within Python. For studies that constructed a vocabulary using NLP, most studies (n = 16; 76%) referred to literature^{18,34–36,39,42,44,45,47–50} or team domain expertise.^{26,35,38,40,42,44,46,49,50} Other methods used to build the preliminary vocabulary included conceptual frame-works,^35,41 qualitative interviews,³³ and statistical evaluation of variables from structured data and clinical notes.^26,43 Most studies^{18,33,35,40,42–50} (n = 13; 62%) used standardized terminologies to expand and map their list of terms. Of these studies, the most common standardized terminology used was the Unified Medical Language System (UMLS).^{18,42,44,47–50} Other standardized terminologies used included the Omaha System,^40,43–45 Synthesized Nomenclature of Medical Terms (SNOMED),^33,35,46 International Classification of Diseases (ICD),³³ Logical Observation Identifiers Names and Codes (LOINC),³³ and International Classification for Nursing Practice (ICNP).³⁵ Eight studies did not report using any standardized terminologies.^{26,34,36–39,41,51}

Table 3.

NLP Methods in Studies Using NLP in Post-acute Care

Authors, year	Vocabulary Development	Gold Standard Development	NLP Methods	NLP Preprocessing	Standard Terminologies Used	Variables Extracted by NLP	NLP Performance Metrics	NLP Stage
Bjarnadottir et al.33	Keywords were identified based on qualitative interviews and examination of how sexual orientation and gender identity were encoded in medical terminologies	Manual review of notes to assess context of keywords	Bag-of-words method with n-gram—based text retrieval	Fix common typographical errors, all numbers, symbols, stop words, and noise words removed (pronouns were retained), all text was converted to upper case, applied Krovetz stemmer, all texts converted to n-grams	ICD, SNOMED, LOINC	Terms related to sexual orientation, gender identity or expression, relationships and family, sexual behaviors, and supportive services	Not specified	Development
Chae et al.34	Symptom domains and self-management were identified based on a previous study. Vocabulary expansion was conducted by 3 researchers with PhDs in nursing with expertise in symptoms and health informatics	Manual review of 20 notes per each symptom category (n = 240) by 3 nurses with PhDs and expertise in symptoms and informatics	NimbleMiner (word2vec word embedding method)	Not specified	Not specified	Anorexia, chest pain, confusion, cough, dizziness, weight loss, dyspnea, fatigue, nausea, palpitations, peripheral edema, weight gain, unspecified nonadherence, poor diet adherence, poor medication adherence, poor exercise physical activity adherence, issues with other self-care activities, missed medical encounters	Precision = 0.86	Development
Chae et al.35	Relevant literature, Self and Family Management Framework, clinical expertise in nursing, medicine, and pharmacy	Full-text review of 200 narrative notes performed by 2 health informaticians with HHC clinical expertise	NimbleMiner (word2vec word embedding method)	Not specified	SNOMED, ICNP	Poor diet adherence, poor medication adherence, poor exercise/physical activity tolerance, issues with other self-care activities/selfmonitoring, missed health care encounters, and unspecified nonadherence	Precision = 0.94	Development
Harkanen et al.36	Identified common contributing factors based on literature	One team member read a sample of incident reports to manually identify keywords; research team agreed on terms in consensus meetings	Concept linking	Spell-check, tokenization, stemming, and part-of-text tagging, stop words removal (such as auxiliary verbs, conjunctions, possessive pronoun, interjections, numbers, participles, and prepositions), unwanted terms (such as most abbreviations) excluded, terms occurring in fewer than in 10 reports excluded	Not specified	Type of incident: communication and flow of information, patient and relatives, practices, education and training, work environment, and resources	Not specified	Development
Lohman et al.37	Review of National Violent Death Reporting System text narratives	Manual review of medical examiner and law enforcement text narratives; team consensus meetings	Text classification	Converted text narratives into a matrix of brief sequences of words (eg, “felt ill”) that capture subject/verb or noun/adjective, etc. elements of language	Not specified	Status related to LTC: residing in a LTC facility, moving into or anticipating moving into a LTC facility, or expressed anxiety about a loved one living in or transitioning into LTC	Precision and recall; values not specified	Application
Mezuk et al.38	Keywords selected based on prior knowledge	Two reviewers annotated a random sample of text narratives and labeled true positives (deaths associated with LTC) and true negatives	Keyword searching, text classification	Tokenization, n-grams (unigram, bigram), removing stop words, removing words appearing in only 1 note	Not specified	Death characteristics: cause of death, means of injury, death location	Sensitivity = 67.7%, specificity = 99.7%	Development and application
Pesko et al.39	Categories identified based on prior work41	Manual review of select records	Not specified	Not specified	Not specified	Category A: conversation between the nurse and the physician’s office staff, but not involving the physician Category B: one-way communication in which the nurse left a message, paged the physician, or sent a fax. Category C: communication failure in which the nurse was unable to reach the physician, physician office staff, or to leave a message Category D: communication failures compared with the other categories	Not specified	Development
Popejoy et al.40	Terms developed by clinical experts (3 registered nurses and licensed clinical social worker with care coordination and research and practice experience)	Manual annotation of 20 medical records from care coordination narrative notes	Concept extraction, activity-focus recognition	Sentence boundary disambiguation, tokenization, and stemming	Omaha System	Care coordination activities contained action verbs used by nurses when coordinating care; care coordination foci represented the objects the activities acted upon; actors contained people who interacted with care coordinators; problems described specific patient problems identified by the care coordinator; places included locations where patients resided when they received care	Not specified	Development
Press et al.41	Conceptual framework was developed; set of keywords and phrases indicative of each category was developed	Random samples of 20–50 communication attempts through 17 iteration cycles; 2 team members manually coded 363 communication attempts	Keyword matching, regular expression	Not specified	Not specified	Category A: spoke to MD Category B: left message with secretary Category C: sent fax to office Category D; patient unknown to MD	Not specified	Development
Song et al.42	Literature review, team domain experts (HHC, certified wound, ostomy and continence nurse, nursing PhD student)	Random sample of 200 notes was annotated by 2 human experts with more than 5 years of experience in HHC; interrater reliability was relatively high (k = 0.72)	Keyword searching, word embedding (word2ved applied by NimbleMiner software), negation detection	Not specified	UMLS	Wound type, general wound infection, exudate, foul odor, periwound skin, wound bed tissue, spreading systemic signs, possible wound infection name, possible wound infection treatment	Positive predictive value = 0.87, sensitivity = 0.91, F-score = 0.88	Development and application
Song et al.43	Statistical evaluation of variables from structured data, clinical notes, and Omaha System problems	Manual annotation of 4000 clinical notes by HHC clinical experts who labeled each note as “concerning” or “not concerning”	Not specified	Not specified	Omaha System	(1) Concern (2) 31 Omaha System problems (eg, circulation, bowel function, abuse) that were identified as risk factors for hospitalizations or ED visits in HHC	F-score = 0.6, F- score = 0.8	Application
Song et al.44	Concerning concepts were identified by 5 experts in HHC based on the literature, expert opinion, and/or their clinical experience; all had extensive clinical or research experience in HHC, 1 had a master’s degree, and the other 4 had PhDs in nursing	Manual annotation of 1000 clinical notes by 3 HHC clinical experts; interrater agreement was high (k = 0.89)	Word embedding (word2vec applied by NimbleMiner software)	Not specified	Omaha System, UMLS	Neuromusculoskeletal function, pain, circulation, mental health, skin, health care supervision, cognition, respiration, communicable infectious condition, social contact, digestion hydration, medication regimen, bowel function, genitourinary function, nutrition, neglect, communication with community resources, speech and language, hearing, income, residence, consciousness, abuse, interpersonal relationships, personal care	Precision = 0.95, Recall = 0.78, F-score = 0.84	Development
Song et al.45	Used terms developed for an NLP algorithm from previous study44	N/A; used a previously developed and validated NLP algorithm44	Not specified	Not specified	Omaha System	Abuse, bowel function, circulation, cognition, infectious condition, consciousness, digestion/hydration, genitourinary function, health care supervision, medication regimen, mental health, neglect, nutrition, neuro musculoskeletal function, pain, respiration, skin, social contact, speech and language, substance use	Precision = 0.95, recall = 0.78, F-score = 0.84	Application
Topaz et al.46	Clinical knowledge or an existing biomedical thesaurus	N/A	Word embedding (word2ved applied by NimbleMiner software)	Remove punctuation and lowercase all letters, convert frequently co-occurring words in the clinical notes into phrases with lengths of up to 4 words (4-grams)	SNOMED	Terms referring to falls	Precision	Development
Topaz et al.47	Literature review	Manual annotation of 1704 clinical notes by2 members of the study team; interrater agreement was high (k = 0.84)	Word embedding (word2ved applied by NimbleMiner software)	Remove punctuation and lowercase all letters, convert frequently co-occurring words in the clinical notes into phrases with lengths of up to 4 words (4-grams)	UMLS	General fall history, falls within 2 weeks; and falls within 2 days of the note date	Precision, recall, F-score = 0.86	Development and application
Topaz et al.48	Literature review	Manual annotation of 400 clinical notes by 2 expert reviewers; interrater agreement was high (k = 0.84)	Terminology identification, word embedding (word2ved applied by NimbleMiner software)	Not specified	UMLS	Depressed mood or apathy, agitation, aggression, anxiety, impaired memory, delusions or hallucinations	Precision = 0.87, recall = 0.91, F-score = 0.88	Development
Topaz et al.26	Gain ratio attribute evaluation technique to measure the predictive power of each word or expression for acute care utilization	Two PhD prepared nurses with experience in HHC reviewed list of terms identified as having high predictive value for acute care utilization; interrater agreement was high (k= 0.94)	Not specified	Omitting punctuation, omitting common stop words, omitting terms with high frequency, converting all words to lowercase, convert to n-gram	Not specified	Clinical factors, coordination/communication, service use, social/environmental factors, temporal, device/equipment	Precision, recall, F-score = 0.83	Application
Topaz et al.18	Seven symptoms selected as National Institute of Nursing Research common data elements; nurse clinician scientists reviewed and revised synonymous words and expressions for each symptom	Manual annotation of 500 clinical notes by 2 reviewers; interrater agreement was high (k = 0.91)	Terminology identification, word embedding (word2ved applied by NimbleMiner software)	Not specified	UMLS	Anxiety, cognitive disturbance, depressed mood, fatigue, sleep disturbance, pain, well-being	Precision, recall, F-score = 0.87	Development
Woo et al.49	Literature review, team domain expertise	Manual annotation of 300 clinical notes by 2 expert reviewers (master’s level registered nurses with more than 10 years of clinical experience in HHC); interrater agreement was relatively high (k = 0.84)	Terminology identification, word embedding (word2ved applied by NimbleMiner software)	Not specified	UMLS	Urinary tract infection terms: specific names, specific symptoms, nonspecific symptoms, nausea and vomiting, fever, confusion	F-score = 0.9	Development
Woo et al.50	Literature review, team domain expertise; 2 team members (certified wound, ostomy and continence nurse and a nursing PhD student) reviewed the preliminary list independently and validated the terms	Manual annotation of 200 clinical notes by 2 expert reviewers; interrater agreement was relatively high (k = 0.72)	Terminology identification, word embedding (word2ved applied by NimbleMiner software)	Not specified	UMLS	Wound type, wound infection, exudate, foul odor, periwound skin, wound bed tissue, spreading systemic signs, possible wound infection name, possible wound infection treatment	Precision = 0.87, recall = 0.91, F-score = 0.88	Development
Zolnoori et al.51	Standardized categories from the EHR that allow HHC staff to document reasons for delayed visits	Manual review of 2200 clinical notes by 3 HHC clinical experts to label each note with delayed start-of-care categories; interrater agreement was high (k > 0.80), discrepancies resolved via expert team discussion	Regular expressions	Lowercasing, stripping (removing extra spaces), removing special characters	Not specified	Delayed visit reason categories —no answer at the door or phone, administrative or scheduling issues, patient or family request to postpone or refuse some HHC services	Precision = 0.8, recall = 0.75, F-score = 0.77	Development

ED, emergency department; HHC, home health care; LTC, long-term care; N/A, not applicable; UMLS, Unified Medical Language System.

Nineteen studies^{18,26,33–44,47–51} (90.5%) created a gold standard data set (ie, human expert annotated data set used to train the NLP algorithm) by having members of the research team annotate a random sample of clinical notes. Five studies^33,36–39 (23.8%) did not provide details about number of notes or qualifications of the annotators. Six studies^{18,40,41,47,48,50} (28.6%) specified number of notes, but did not provide any detail about qualifications of the annotators. One study²⁶ (4.8%) described qualifications of the annotators, but did not identify number of notes. Seven studies^{34,35,42–44,49,51} (33.3%) specified that at least 200 clinical notes were manually annotated by home health care clinicians^{26,35,42–44,51} and nurses.^26,49 Nine studies^{18,26,42,44,47–51} (42.9%) reported interrater agreement using the kappa statistic, which ranged from 0.72^42,50 to 0.94²⁶; all indicating relatively high interrater agreement.⁵² Gold standard development was not applicable to the remaining 2 studies^45,46 (9.5%), as one study used an NLP algorithm that was developed in a previous study⁴⁵ and the other study focused on the development of an NLP software.⁴⁶

NLP concept identification performance was evaluated in 16 studies^{18,26,34,35,37,38,42–51} (76%) through several performance metrics: precision, recall, sensitivity, specificity, and F-score. Precision is the ratio of true positives to the total number of predicted positives, which is the same as positive predictive value. Recall is the ratio of true positives out of the actual number of positives, which is the same as sensitivity. F-score is the weighted harmonic average of precision and recall. F-scores range from 0 to 1, with scores closer to 1 indicating stronger performance. Among these studies, 2 studies^37,46 reported that precision and recall were calculated, but no values were provided. The remaining 5 studies^{33,36,39–41} (23.8%) did not report any performance metrics. Of the 16 studies that reported calculating performance metrics, most studies (n 11)^{18,26,34,35,42–45,47–51} used F-score. Among these studies, F-scores ranged from 0.77⁵¹ to 0.9,⁴⁹ all of which indicate good performance of the NLP algorithm.

Among the included studies, 4^38,42,45,47 (19%) reported a multi-method approach describing both the development of the NLP algorithm and the application of NLP in further analyses to predict clinical outcomes.^53,54 The remaining studies (n = 17; 81%)^{4,5,25–30,32–34,36,40,42–45} reported the development of the NLP algorithm or applying NLP in further analyses. Most studies (n 14; 66.7%)^{18,33–36,39–41,44,46,48–51} solely described the development of the NLP algorithms, meaning they reported on their technique to convert narrative text into interpretable, structured data that could be used in further analyses.⁵³ Conversely, 3^26,37,43 (14.3%) were application studies, meaning they reported on the NLP techniques and how they integrated this information into further analyses.⁵³

Quality indicators

A summary of the quality indicators across the 21 included studies is provided in Table 4. Most studies^{18,34,35,38,42,44,47,49–51} (n 11; 52.4%) received “yes” responses in all quality indicator domains. Eight studies^{26,33,36,40,41,43,45,46} received “yes” responses in 75% of quality indicator domains. The remaining 2 studies received yes responses in 50%³⁷ and 25%³⁹ of quality indicator domains. All studies^{18,26,33–51} (n 21) had a clearly defined purpose. All but 4 studies^26,39,43,45 adequately described the NLP methods that were used. All but 2 studies^37,39 specified the number of clinical notes analyzed. Most studies (n = 14; 66.7%)^{18,26,34,35,38,42–45,47–51} reported that performance metrics were calculated and provided corresponding values.

Table 4.

Quality Indicators

Authors, year	Clearly Defined Purpose	NLP Methods Adequately Described	Number of Documents Analyzed Specified	Performance Metrics Reported
Bjarnadottir et al.33	Yes	Yes	Yes	No
Chae et al.34	Yes	Yes	Yes	Yes
Chae et al.35	Yes	Yes	Yes	Yes
Harkanen et al.36	Yes	Yes	Yes	No
Lohman et al.37	Yes	Yes	No	No
Mezuk et al.38	Yes	Yes	Yes	Yes
Pesko et al.39	Yes	No	No	No
Popejoy et al.40	Yes	Yes	Yes	No
Press et al.41	Yes	Yes	Yes	No
Song et al.42	Yes	Yes	Yes	Yes
Song et al.43	Yes	No	Yes	Yes
Song et al.44	Yes	Yes	Yes	Yes
Song et al.45	Yes	No	Yes	Yes
Topaz et al.46	Yes	Yes	Yes	No
Topaz et al.47	Yes	Yes	Yes	Yes
Topaz et al.48	Yes	Yes	Yes	Yes
Topaz et al.26	Yes	No	Yes	Yes
Topaz et al.18	Yes	Yes	Yes	Yes
Woo et al.49	Yes	Yes	Yes	Yes
Woo et al.50	Yes	Yes	Yes	Yes
Zolnoori et al.51	Yes	Yes	Yes	Yes

Discussion

This scoping review included 21 studies that used NLP in post-acute care settings. The relatively small number of studies represents opportunities to use NLP in post-acute care settings to address a variety of concepts and examine associations with negative outcomes. The included studies highlight that NLP can be used to investigate symptom descriptions, patient characteristics, and risk factors. Further research should broaden the use of NLP applied to free-text clinical notes in post-acute care settings to identify other concepts and evaluate associations with important quality outcomes, including length of stay, symptom management, and health care costs.

Results of this review provide data indicating that NLP can be used to accurately identify information in free-text clinical notes. Clinicians rely on free-text clinical notes to efficiently document patients’ clinical characteristics, including social risk factors (eg, environmental, psychological, and health-related behaviors).^12,55 Current EHR systems are faulted for not being optimized for clinicians’ workflows and having poor user interface designs.¹⁰ This is largely due to structured data forms, which do not capture all relevant domains, are time-consuming, and may result in duplicative documentation.⁸ This can lead to increased manual work (eg, switching between screens, clicking through structured data forms), clinicians having less face-to-face time with patients, documentation burden, and clinician burnout.⁸ Optimizing existing EHR systems or a total redesign of EHR structure and function are immense tasks that may take years to complete. Therefore, using data that are readily available in free-text clinical notes is warranted to address the time-sensitive needs of patients in post-acute care settings. Approximately half of the studies included in this review showed that NLP systems have good performance^{18,26,34,35,42–45,47–51} and can be implemented into clinical practice to identify patient characteristics, symptoms, and risk factors not captured in structured data forms. Building on these findings to create NLP algorithms that are generalizable in post-acute care is essential to leveraging the vast amount of free-text clinical documentation generated in these settings.

Although the use of structured data forms can support interoperability (ie, the exchange of health information) as documentation is standardized,⁵⁶ adding more structured data forms to current EHRs will likely contribute to documentation burden and negatively affect quality of care. Free-text clinical notes are not standardized and reflect different descriptions that may represent the same concept.¹⁴ Standardizing terms used in free-text clinical notes is essential to improve clinical relevancy and generalizability of NLP algorithms among post-acute care settings. This can be achieved by mapping concepts to standardized terminologies (eg, UMLS, the Omaha System, SNOMED, ICD, LOINC, ICNP) when developing preliminary vocabularies for NLP algorithms. Surprisingly, more than one-third of the studies included in this review did not use standardized terminologies.^{26,34,36–39,41,51} This not only impedes interoperability, but also limits generalizability to other post-acute care settings. The studies that mapped concepts to standardized terminologies^{18,33,35,40,42–50} developed NLP algorithms that can be used in other similar clinical settings. For these reasons, future studies that use NLP in post-acute care settings should use standardized terminologies to expand and define vocabularies for NLP systems.

NLP is beneficial in settings where a large number of free-text clinical notes are generated for patients with multiple comorbidities, symptoms, and risk factors. For example, home health care clinicians see patients independently in the community and communicate with the interdisciplinary team remotely through EHR documentation.⁷ In addition, in nursing home settings, nurses see patients more frequently and are responsible for communicating details about patients’ clinical characteristics and health status changes to physicians who may be off site.⁵⁷ Similar to home health care, this communication often takes place through the EHR. On average, patients remain in home health care for 30–60 days,⁶ and in nursing homes for short (ie, 90 days) or long (ie, 365 days) stays.⁵⁸ For these reasons, numerous free-text clinical notes are written about these patients and stored in the EHR.¹³ Further, in contrast to patients in long-term assisted living facilities who need basic assistance with activities of daily living (eg, bathing, dressing, and grooming) and medication management, most home health care and nursing home patients are medically complex,⁵⁹ requiring skilled nursing and other clinical services. Whether these patients continue on a trajectory of stabilization and recovery or not largely depends on quality of care during this time. NLP can be used to uncover valuable information about patients’ physical, psychological, and socio-behavioral characteristics, which can be used as a basis for research involving risk prediction and clinical decision support.

Using NLP for risk detection was the primary focus of more than two-thirds of the studies included in this review.^{18,26,34–38,41–51} Early identification of risk for negative outcomes in post-acute care is an essential tenet of clinical practice in this setting. A fundamental goal of post-acute care is to avoid hospital readmissions.³ More than one-third of included studies examined the risk for acute care utilization (ie, emergency department visits and hospitalizations).^{18,26,42–48} One-third of studies evaluated the risk for other negative outcomes (ie, medication errors,³⁶ suicide mortality,^37,38 poor chronic disease self-management,^34,35 urinary tract infections,⁴⁹ skin breakdown,⁵⁰ and failed communication among the health care team).⁴¹ The feasibility of using NLP to examine risk of these negative outcomes on a large scale supports the potential to use NLP to identify concepts linked to other important quality outcomes, including worsening status of specific chronic conditions, symptom management, length of stay, and health care–related expenditures. On the policy level, these outcomes are key to determining the allocation of resources, reimbursement, and guidelines for systems and models of care. NLP can integrate patient and clinician insight into these analyses by examining what information clinicians document about what patients report. For example, NLP algorithms that extract symptom information can provide evidence that can be used in future symptom-based research geared toward building evidence-based holistic care guidelines to change the way care is provided to patients with multiple comorbidities in post-acute care settings. In addition, NLP algorithms that extract factors associated with medication errors can provide evidence to support the need for better staffing ratios and technologies aimed at reducing these errors in settings with limited information technology maturity (eg, nursing homes).⁶⁰ Further, NLP can be used to identify clinical details that may be missed by diagnosis codes,⁶¹ which can support changes in the way health care visits are billed. For these reasons, policymakers may be interested in incentivizing NLP research in post-acute care settings to foster improvements in these quality outcomes.

Although most studies focused on risk detection, all relevant risk factors were not always considered. Approximately half of the included studies used NLP to extract variables related to physical signs or symptoms.^{18,26,34,35,42–45,49,50} Existing evidence supports that risk factors from psychosocial or socio-behavioral domains, including social determinants of health, influence health outcomes.⁶² Although this is known, only approximately one-third of studies included variables from these domains.^{18,26,34,35,43–45,48} Further, prior studies suggest that including variables from these domains can improve risk detection for negative outcomes among racial and ethnic minority populations.⁶² However, approximately half of the studies in this review did not report race and ethnicity data and only approximately one-third included racially and ethnically diverse samples.^{18,26,35,42,48,51} Emerging evidence suggests that clinical documentation contains language reflecting implicit biases perpetuating health care inequities.^63,64 Prior studies in other clinical settings have shown that NLP can detect this language and examine associations between language that conveys bias or stigma and negative outcomes.^65–67 Given that these biases can be communicated to patients, there is a need for research using NLP to examine language in clinical notes to reduce care disparities. The inclusion of racially and ethnically diverse samples in studies that use NLP can illuminate in-equities in post-acute care and highlight opportunities to improve equitable care delivery.

In addition to providing descriptive data relating to frequencies of word expressions in clinical notes, NLP can be incorporated into prediction algorithms that analyze patterns in data to predict risk for clinical deterioration and prevent negative outcomes.^42,68 For example, NLP has been used to identify cases of infection that may have been missed by relying only on diagnosis codes.⁶¹ Existing evidence supports that prediction models incorporating both structured and unstructured data elements have better prediction performance than models using only structured data.⁴² Therefore, including variables derived from concepts extracted by NLP can improve the predictive ability of models by integrating multidimensional concepts (eg, physical, psychological, and socio-behavioral characteristics) that are not well-captured in structured data.

NLP algorithms can only extract concepts identified by researchers as important to investigate. The gold standard development process is essential to using NLP to answer research questions and developing well-performing algorithms that are clinically relevant. For example, if researchers were interested in examining risk factors for rehospitalization in post-acute care settings, but only included cardiovascular and respiratory symptoms as concepts for the NLP algorithm to extract, many clinically relevant domains (ie, symptoms affecting other physiological systems, psychological, and socio-behavioral characteristics) would be missed. Further, if the vocabulary was not developed rigorously, the NLP algorithm may perform well, but may not be clinically meaningful. More than two-thirds of studies included in this review relied on literature review or domain expertise to identify concepts for the NLP algorithm to extract.^{18,26,38,40,42,44–50} Only 1 study derived concepts from qualitative interviews with key stakeholders.³³ Qualitative methods can be a critical first step to developing an NLP algorithm underscored by clinician and patient insight. Data obtained through qualitative interviews can be triangulated with findings from literature review and validated with domain expertise. Only approximately one-third of included studies reported interrater agreement was calculated after annotators reviewed a subsample of clinical notes to create the gold standard.^{18,26,42,44,47–51} In addition, only approximately one-third of included studies provided sufficient details about the qualifications and expertise of the annotators.^{26,35,42–44,49,51} These are important initial steps in NLP research that can impact NLP performance and results. Future research should consider ways to enhance rigor of the gold standard development process and integrate insight from key stakeholders into NLP algorithm development to strengthen the reliability of performance results and applicability to clinical practice.

There is a dearth of research using NLP in post-acute settings worldwide, as all but 1 study was conducted in the United States. Given that post-acute care systems exist in most developed and developing countries,^1,69 it was surprising to find a lack of studies using NLP in post-acute settings on a global scale. Reasons for this may include limited information technology maturity, resources, and data available to conduct this research worldwide. Post-acute care, which includes medical, nursing, rehabilitation, and residential services following acute hospitalization, varies in definition across countries due to differences in health care models, insurance coverage, and cultural care practices.¹ Future studies should account for these differences when using NLP to extract variables and assess patient outcomes.

More than two-thirds of studies analyzed data from one home health care agency (ie, VNS Health).^{18,26,34,35,39,41–51} VNS Health is one of the largest home health care agencies in the United States, serving more than 75,000 patients per year in New York City, Westchester, and Long Island.⁷⁰ In addition to clinical care services, VNS Health provides opportunities for research through the Center for Home Care Policy and Research.⁷⁰ This allows for access to data and study personnel needed to conduct NLP analyses on a large scale. Other smaller home health care agencies and post-acute care settings may not have this research infrastructure, which may contribute to the small body of research using NLP in post-acute care. Policymakers should consider how they can provide resources to support research and incentivize post-acute care settings to use informatics tools such as NLP to improve care and outcomes.

All studies analyzed data that were already collected and used a cross-sectional or retrospective cohort design. Most included studies used EHR data^{18,26,33–35,39–51} and only a few studies used other clinical documentation data sources (ie, national reporting systems).^36–38 Further, studies analyzed data collected at least 4 years before publication, with the longest delay between collection and publication being 13 years.⁴⁰ Regarding EHR data in the United States, this may be because of a lack of standardization of EHR systems, leading to in-efficiencies in obtaining and preparing data for analysis.^71,72 This may lead to analyses not reflecting current EHR systems or differences in note format over time. However, the included studies used datasets that spanned several years to reflect differences in note format over time and account for potential changes in EHR systems. In addition, NLP algorithms used in studies included in this review achieved good overall performance, indicating readiness to be integrated into clinical practice. Despite data collection and analysis lags, study findings are still relevant, clinically meaningful, and can be applied to current clinical contexts to reduce negative outcomes.

Clinical Implications

NLP can be used to support clinical decisions in post-acute care settings in several ways. NLP can alleviate challenges faced by clinicians related to manual review of unstructured free-text clinical notes and improve clinical decision making in terms of quality and efficiency. Manual review of unstructured free-text clinical notes is time-consuming, costly, and can cause pertinent clinical information to be missed. NLP can analyze these data in real-time, extract relevant information, and present it to clinicians in a meaningful way.⁷³ At the start of care, post-acute care clinicians often receive discharge instructions, care guidelines, and medication information in unstructured free-text formats.⁷⁴ NLP can be used to extract important information about risk factors and follow-up care that can be presented to clinicians concisely. In addition, this information can be integrated into predictive tools that can alert clinicians of patients’ health status changes²⁷ and prompt increased monitoring, changes to medication regimens, or more tailored care. Further, NLP can be integrated into clinical decision support systems to improve diagnostic accuracy,⁷⁵ patient safety, and quality of care.⁷³ NLP is essential in using unstructured free-text data to underpin clinical decision support and incorporate patient and clinician insight into these systems. Results of this review support that well-performing NLP algorithms have been developed in post-acute care settings and are ready to be implemented in clinical practice.

Limitations

This scoping review is not without limitations. First, it is possible that some relevant studies may have been missed, as the search was limited to 3 databases and excluded articles not published in English. In addition, the definition of post-acute care can vary globally, and the search strategy that was used may not have represented all uses of this term. However, the search included a range of concepts to capture variations in this term to include studies that represent the use of NLP in different post-acute care models. Most included studies were performed in the home health care setting and used data from 1 home health care agency (ie, VNS Health). Although this impedes generalizability to other home health care agencies and post-acute care settings, the use of standardized terminologies by most included studies promotes interoperability. Only 2 studies were conducted in long-term care facilities (eg, assisted living and nursing homes), which illuminates the limited information technology maturity in these settings.⁶⁰ This review recognizes that a limitation to NLP being applied in post-acute care settings is related to the information technology maturity of the clinical practice facility. Some of the included studies did not report NLP performance metrics. This causes the utility of the NLP algorithm to remain ambiguous and limits comparability of NLP algorithms across studies. Last, a formal quality appraisal was not conducted, and a quality appraisal template was developed based on prior reviews that focused on NLP.^15,24–26 Although this is appropriate given that reporting standards differ among NLP studies, appraising each study with a formal quality appraisal tool may have yielded different results.

Conclusions and Implications

This study provides an overview of how NLP has been used in post-acute care settings and suggests implications for clinical practice and policy. NLP can be used to leverage unstructured data in post-acute care settings to promote timely, comprehensive, and equitable care. Given the potential to improve clinical practice and reduce negative outcomes in post-acute care, policymakers are encouraged to champion the use of NLP in these settings with a similar degree of urgency as initiatives in acute care settings. Increasing research on free-text clinical notes in post-acute care settings using standardized terminologies could help analyze variations in clinician documentation over time and aid in developing prediction models that identify patients at risk to prevent negative outcomes. This study also highlights a need for future researchers to explore socio-behavioral determinants in clinical notes and more diverse sample sets to improve health equity in informatics tools.