From evidence-based guidelines to practice: Using natural language processing to identify impactful interventions in falls prevention

Abstract

Objectives

This study aimed to identify and characterize implementation strategies for falls prevention in long-term care (LTC) homes using natural language processing (NLP) with a focus on the Registered Nurses’ Association of Ontario’s (RNAO) Preventing Falls and Reducing Injury from Falls (2017) Best Practice Guideline (BPG).

Methods

A retrospective quality improvement study was conducted using narrative reports submitted to RNAO MyBPSO reporting system from 63 Ontario LTC homes (2017 to 2022). Free-text data describing implementation activities were analyzed using an NLP pipeline that included text preprocessing, tokenization, and keyword extraction using Rapid Automatic Keyword Extraction (RAKE). Extracted terms were aggregated into candidate themes and further analyzed using a lexicon-informed sentiment analysis approach adapted for clinical language. Themes were iteratively refined through a structured consensus process involving multidisciplinary experts. Inter-rater agreement was assessed using Cohen’s kappa.

Results

A total of 153 reports generated 235 preliminary themes, which were refined through sentiment analysis and expert consensus into 10 themes. Commonly identified strategies included falls risk assessment, fall prevention strategies, risk factors of falls, care plan development, post-fall assessments, falls tools, fall documentation, education and training, huddles, organizational policies, rounding, reflecting substantial uptake of key BPG recommendations. Sentiment analysis demonstrated predominantly neutral or positive sentiments across implementation narratives, with variability indicating differences in implementation experience. Gaps were most evident in documentation, i.e., post-fall follow-up actions and policy-level strategies, suggesting opportunities to strengthen implementation fidelity.

Conclusions

The study identified a core set of frequently implemented falls prevention strategies in LTC settings. These findings provide actionable guidance for clinical practice by emphasizing the need to standardize risk screening and reassessment processes, strengthen post-fall evaluation and follow-up, improve documentation systems to support continuity of care, and implement ongoing, competency-based education for staff. Together, these targeted strategies can support more consistent translation of evidence-based guidelines into routine practice and enhance quality-improvement efforts in falls prevention in LTC settings.

What is known?

• •Falls are a leading cause of injury and death among older adults; preventing falls is a core focus of evidence-based nursing practice in long-term care settings.
• •Traditional evaluations of guideline implementation (e.g., audits, surveys) are labor-intensive and difficult to scale.
• •Natural language processing-based thematic analysis is recognized as a powerful approach for extracting meaningful insights from large volumes of unstructured texts to identify patterns and themes.

What is new?

• •This study identified the most frequently implemented falls prevention strategies, including updated falls risk assessment, individualized care planning, purposeful hourly rounding, formalized post-fall procedures, and mandatory education for staff and families.
• •Sentiment analysis showed the strongest positive associations for purposeful hourly rounding and guideline-aligned rounding policies, while negative sentiment consistently clustered around documentation gaps, incomplete execution, and follow-through challenges.
• •There is a need to standardize falls risk screening and reassessment, embed individualized prevention strategies into care plans, prioritize purposeful hourly rounding, strengthen post-fall follow-up processes, address persistent documentation and environmental strategy gaps, and deliver targeted, competency-based education to support sustained implementation and quality improvement.

1. Introduction

Falls are a leading cause of injury and mortality among older adults and individuals with neurological or mobility impairments [1]. Patient falls represent a major safety concern with high incidence rates across acute care, long-term care (LTC), and community settings [2,3]. Consequently, improving fall prevention interventions is essential to ensuring patient safety and delivering high-quality nursing care [4]. Substantial studies [[5], [6], [7], [8], [9]] have examined fall prevention strategies, demonstrating that nursing interventions, such as screening, risk assessment, mobility support, and patient- and family-oriented care, improve health outcomes. However, studies [[10], [11], [12]] also highlight variability in how these interventions are implemented and documented in clinical practice, revealing persistent gaps between evidence-based strategies and real-world practice.

The Registered Nurses’ Association of Ontario (RNAO) developed and published Preventing Falls and Reducing Injuries from Falls (2017) Best Practice Guideline (BPG) [13] to support consistent evidence-informed falls prevention across health care sectors. The RNAO Falls BPG is widely implemented through the Best Practice Spotlight Organizations® (BPSO®) program, which supports organizations with structured implementation strategies, coaching, monitoring, and evaluation. However, limited empirical evidence exists on how specific falls prevention recommendations are operationalized in routine LTC practice.

Implementation strategies for the Falls BPG include embedding evidence-based recommendations into everyday practice through leadership engagement, practice changes, education, policy revisions, and organizational-level supports. Implementation supports provided by RNAO, including the RNAO Leading Change Toolkit® [14] and coaching support, enable organizations to drive, embed, and sustain evidence-informed practice change. Sustained uptake of BPGs requires local leadership engagement, adequate resources, and ongoing evaluation to embed best practices into routine workflows and to support changes during implementation. The structured approach to implementing and evaluating the Falls BPG strengthens understanding of how to operationalize it in real-world settings. Limited insight exists into how specific falls prevention recommendations are enacted and fully integrated into routine clinical practice. This gap highlights the need for a systematic examination of real-world data to generate actionable insights into falls-specific interventions and to operationalize processes across diverse care settings in support of evidence-based practice guidelines.

Natural Language Processing (NLP) is a subfield of computer science and artificial intelligence (AI) that has been widely used to extract insights from clinical narratives. A systematic review [15] highlights NLP as one of the most widely used techniques for analyzing nursing-related text data. NLP algorithms apply language rules and relevant domain knowledge to process syntax (tokenization, sentence detection), extract structured information (e.g., convert unstructured text into a structured form), assign concepts (e.g., assign a concept to a word or group of words), and identify relationships (assign relationships between concepts) in free-text [16]. NLP supports clinical decision-making, advancing research on patient outcomes and quality of care [[17], [18], [19]].

Building on these advances, health-care organizations worldwide are increasingly leveraging both structured and unstructured data from health-care records and nursing documentation to generate nursing information and knowledge [20]. Understanding nursing interventions is critical for public health, as effective nursing care improves patient safety, continuity of care, patient satisfaction, and interprofessional communication while reducing healthcare costs [21]. In addition to electronic health records, data generated during the evaluation of implemented clinical guidelines represents a particularly important yet underutilized source of nursing knowledge. These evaluations frequently include unstructured textual data such as narrative feedback, expert recommendations, implementation notes, and free-text comments. Manual analysis of these data is time-consuming and resource-intensive, limiting scalability and efficiency [19]. NLP techniques enable the systematic analysis of this unstructured guideline-related text, facilitating the efficient extraction of nursing interventions, contextual factors, and implementation insights. Leveraging NLP to analyze guideline evaluation data can therefore enhance understanding of nursing practice, support adherence to BPG recommendations, and inform quality improvement initiatives.

As the RNAO BPSO® program, health service, and academic organizations participate, it continues to evolve and expand globally, and there is a growing need to systematically organize and analyze large volumes of narrative data related to Falls BPG practices and policies. In response, RNAO began using NLP in 2022 to uncover the underlying meaning of interventions and their impacts on health outcomes reported by BPSO®s [22]. Accordingly, the purpose of this study was to leverage NLP techniques to routinely collect data from LTC homes to identify patterns in risk factors, interventions, and outcomes, thereby informing implementation strategies associated with RNAO Falls BPG. By applying NLP to qualitative data, it is expected to deepen understanding of implementation strategies, support the uptake and sustainability of BPG-recommended nursing practices, and provide invaluable insights into effective nursing practices that support patient health outcomes.

2. Methods

2.1. Study design and data sources

This study employed a retrospective quality improvement study design using routinely collected guideline implementation reports based on falls prevention strategies. Data were extracted from the RNAO MyBPSO progress reporting system, a structured platform used by BPSO®s to document and report implementation activities over time. A convenience sample of reports from LTC homes was chosen for the analyses. Specifically, these Canadian-based LTC settings were selected due to data availability and accessibility through the MyBPSO reporting system.

2.2. Inclusion/exclusion criteria

Although reports from multiple sectors were available, LTC was the primary focus of this study due to the higher fall risk associated with residents’ advanced age, frailty, multimorbidity, cognitive impairment, and mobility limitations. Any intervention reported by fewer than five unique organizations was excluded from the analysis. A subset of BPSO®s was excluded from the analysis due to governance structures or the specificity of context, such as indigenous BPSO®s, organizations that had withdrawn from the program, and cases where data use required additional privacy safeguards and consensus processes.

Eligible documents in this analysis contained free-text descriptions of falls, fall risk assessment, prevention strategies, or related interventions. Text entries with fewer than 10 tokens or containing only administrative placeholders were excluded from the analysis. Documents related to complex interventions but defined under different guidelines were also excluded from the analysis, i.e., falls strategies that involved organizational processes on pain management.

2.3. Data extraction

Retrospective data on the implementation of the Falls BPG were obtained from BPSO® reports housed within the RNAO MyBPSO qualitative data repository, spanning the period from 2017 to 2022. From 2017 to date, RNAO has received reports from BPSO®s on a semi-annual and annual basis. During the pre-implementation period, BPSO®s submit semi-annual reports; after 3 years of implementation, LTC homes submit annual reports. Reports from 63 Ontario-based LTC homes were considered for the analysis, within the first two to three years of their formal Falls BPG implementation. A total of 153 reports were analyzed; some sites submitted multiple reports over a three-year period, which were converted into a structured format, enabling effective information mining across diverse nursing domains.

These reports were submitted by nursing leads from the LTC homes using a structured web form in MyBPSO with an open-text field to provide information on infrastructure, organizational governance, and BPG context, including practice change, organizational policy, education provided, and impacts. BPSO®s adhere to standardized documentation procedures for implementation and evaluation activities, with MyBPSO reports capturing relevant contextual information. Unstructured free-text data describing implementation activities and program impact were extracted using established procedures that ensured both organizational and individual data privacy.

As RNAO does not collect any socio-demographic information (e.g., age, sex) from BPSO®s, the data extraction and cleaning process was streamlined to focus on specific variables in the system related to practice change, education, and policy. Semantic heterogeneity in the data (i.e., differences in the meaning and interpretation of terminology) was minimized by consistently reporting key variables: practice change, education, and organizational policies aligned with BPG implementation and evaluation.

2.4. Data analyses

The data analysis was conducted in three distinct phases: 1) text pre-processing & keyword extraction, to identify useful information and patterns from the data; 2) sentiment analysis, to assess emotional tone within the textual data; and 3) consensus analysis, conducted by an internal-external working group and a core working group to identify overlapping themes, refine content, and validate keyword and stems derived from the unstructured free texts.

2.4.1. Phase 1- text processing & keyword extraction

The initial phase focused on preparing unstructured text data for analysis through a series of NLP techniques. These techniques involved data cleaning, language normalization, thematic grouping, keyword extraction, and text summarization to identify core concepts and condense content effectively. Text preprocessing included: breaking sentences into individual text units (tokenization); assigning grammatical roles to each word [part-of-speech (POS) tagging]; stop word and punctuation removal of non-letter characters; and converting written personal quotes into frequency-based arrays for structured analysis.

To manage competing matches, Rapid Automatic Keyword Extraction (RAKE) scoring was used, while Backtracking Non-deterministic Finite Automation (NFA) (Perl-style/PCRE-style), NFA algorithm [23] was used to explore patterns sequentially. When the branch of themes failed, Backtracking NFA supported nested quantifiers, thereby improving the model’s performance. The mechanism helped develop a heuristic approach to generate descriptors and related themes, extending beyond the capabilities of classical automated NLP models and yielding more quantifiable results. The RAKE scoring further helped with data-driven keyword relevance estimation to support branching and sub-nodes, and it explored greater clustering for validation (see Appendix A for illustration). The rationale guiding the combination of NFA and RAKE are: 1) linguistic precision due to NFA; 2) computational balance; and 3) robust feature relevance obtained by combining rule-driven pattern extraction with rank ordering.

Keyword extraction was essential for identifying recurring patterns in BPG implementation activities. However, the initial analysis faced challenges due to the large data volume, multilingual content (e.g., Spanish and French), and the global scope of the BPSO® program. To address these, manual translation and extraction were performed first, followed by stop-word removal and punctuation handling using the Python natural Language toolkit (NLTK) library [24]. Finally, the RAKE-NLTK [23] was used to extract and score keywords.

2.4.2. Phase 2- sentiment analysis

Sentiment analysis was subsequently applied to the RAKE-identified phrases using a lexicon-based scoring system tailored to clinical language to interpret the emotional tone conveyed in the extracted phrases. In the context of BPG implementation, the textual data often includes specialized nursing terminology and specific jargon (e.g., “falls,” “pain,” and “pressure injuries”). These terms, due to their connotative nature, are frequently misclassified as negative by conventional sentiment analysis models. To improve classification accuracy, contextual sentiment analysis was applied, assigning positive or negative sentiment based on the intended meaning in the nursing context.

Analyzing the sentiment of narratives within the documents helped identify underlying concerns, barriers, and positive interventions. However, traditional sentiment analysis has a limited scope in this context, as not all texts contribute equally: for example, terms associated with fall risk assessment carry negative connotations, while neutral or procedural language can dilute the emotional or evaluative content relevant to falls-related implementation strategies. To address this, high-relevance keywords (with positive and negative tones weighted equally) were combined with neutral-tone text, and the resulting text was assigned weighted scores using RAKE.

Sentiment polarity was numerically reclassified (+1 for positive, −1 for negative, and 0 for neutral) to improve data representation with a defined sentiment score. Overall sentiment variation with sentiment rates (weighted values∗100) vs sub-branch nodes of implementation strategies was plotted to identify patterns across the dataset, yielding major themes localized within a range of 0 to 500.

The resulting sentiment rate enhanced the interpretability of the analysis. This integrated approach enabled assessment of sentiments within broader categories of practice changes, organizational education processes, and policy development/revisions. An illustration of sentiment score calculations and decisions for a selected single theme, associated with the rounding policy and its branches (sub-themes), is shown in Appendix B. Sentiment scoring quantified the emotional tone of responses and categorized them, enabling assessment of perceptions and attitudes toward implementation strategies. Together, RAKE and sentiment analysis built a structured interpretation of narrative data that helped identify core topics shaping organizations’ experience in implementing BPGs.

2.4.3. Phase 3- expert validation

To support accurate interpretation and iterative refinement of the analysis, a multidisciplinary core working group was established. The group comprised RNAO staff, primarily registered nurses, along with data scientists and evaluation specialists, representing expertise in clinical practice, research, education, administration, and policy across public health, primary care, acute care, home health care, and LTC care. The group met regularly to review NLP outputs, apply clinical interpretation, confirm emerging themes, and identify analytic gaps, enabling ongoing refinement of the NLP process.

Generated themes underwent internal and external validation by RNAO professionals, including nurse informaticians, guideline developers, implementation coaches, quality improvement specialists, and evaluators, as well as BPSO® representatives involved in guideline implementation. Inter-rater agreement was assessed using Cohen’s kappa, with values above 70 percent considered acceptable. Validation results were reviewed independently, with discrepancies resolved through discussion or, when necessary, adjudication by a senior lead. This structured process supported consistency and minimized bias. Theme refinement involved three rounds of expert review and two additional rounds with LTC nursing teams, which informed the branching and confirmation of main themes and subthemes and strengthened their relevance to real-world practice.

2.5. Ethical approval

As this was an aggregated thematic analysis based on implementation approaches—no personal identifiers were included in the analysis-- ethical approval was not required.

3. Results

3.1. Text pre-processing and keyword identification

A total of 153 reports were analyzed. From a total of 580K tokens, more than 1,919 keywords (main branch) were extracted from unstructured free-text; these were subsequently reduced to more than 900-word stems through semantic structuring and cutoff thresholds within broader categories: practice changes, educational support, and organizational policy. Stop words were removed during the stemming phase, and the remaining 241 structured phrases were further analyzed to define themes aligned with clinical classifiers. RAKE scores ranging from 0.5 to 50.0 were identified. Fig. 1 illustrates the first 300 recurrent sub-nodes (indexed 1–300) used for subsequent thematic analysis.

Fig. 1.

Illustration of general themes within the classification of practice changes.

3.2. Theme identification

The final set of themes underwent additional analysis to eliminate duplication and ensure clinical relevance. Several initially derived themes were identified as general practices rather than specific clinical interventions. Fig. 2 presents a snapshot of the preliminary results, ranked by RAKE scores and positive sentiment values. In Fig. 2, the x-axis shows sentiment scores, and the y-axis shows RAKE scores for different implementation strategies, ranging from 0 to 500. While aggregated clinical records related to falls revealed several recurring patterns aligned with nursing practice, certain themes, such as “execute physiotherapy,” “practice therapy,” “meal planning,” and “hydration program,” were classified as less common within nursing workflows. Most frequently repeated themes include: “updated falls risk screening tool,” “fall risk assessment including updated care plan focus,” “annual mandatory falls prevention training,” “formalizing post-fall procedures,” “purposeful hourly rounding piloted,” and “person/resident/family education.” These repeated occurrences were based on the highest RAKE scoring within practice changes, education, and policy categories.

Fig. 2.

Overall sentiment rates of themes.

Note: LTC = long-term care.

Within the documentation, activities supporting falls tracking and monitoring were frequently reported, including progress notes, shift and incident reports, updates to falls documentation, and trend reporting. Under the theme of “plans,” high-frequency subthemes included individualized care plans, visual care plans, program evaluation plans, organizational action plans, and universal falls precaution plans. The most frequently reported subtheme related to rounding and policy was “mandate rounding within the organization as a policy update”. These themes offer nuanced insights not typically captured in structured falls-prevention programs but also represent impactful interventions that support organizational workflows.

3.3. Sentiment analysis

An illustrative example is presented to demonstrate the holistic sentiment analysis approach. The sentiment analysis demonstrated that purposeful and structured rounding practices, as recommended by a BPG policy, were generally associated with positive sentiment, with variation across documentation and implementation-related topics (see Appendix B). Strong positive sentiment was most evident for purposeful hourly rounding, with the highest RAKE score at 24.55 and a positive sentiment score (SC) of 0.62 (weighted SC of 15.22). Comfort-focused rounding approaches, characterized by intentional and patient-centered care, also elicited positive sentiment, with an SC of 0.33 (weighted score of 7.19). Rounding practices aligned with evidence-based guidelines, including enhanced rounding policies that supported falls monitoring, showed similarly high positive sentiment with an SC of 0.75 (weighted score of 14.55).

Moderate positive sentiment was observed regarding rounding documentation for intentional and hourly practices. Although sentiment remained positive with a weighted score of 5.01, these references were often accompanied by notes indicating a need for organizational-level improvements. In contrast, rounding initiation displayed minimal emotional association, with a near-neutral sentiment score of 0.01 (weighted score of 0.14).

Negative sentiment was most associated with challenges related to documentation and implementation. References to incomplete rounding documentation, task-focused implementation, and incomplete rounding sheets demonstrated negative sentiment scores ranging from −0.25 to −0.40 (weighted scores between −1.84 and −2.90). These scores serve as an additional analytical filter to support theme consolidation and refinement. Themes with similar semantic content and comparable sentiments were grouped, allowing overlapping or redundant implementation concepts to be merged into broader clinically meaningful categories.

Conversely, themes with persistently negative or near-neutral sentiment, i.e., documentation burden or incomplete executions of certain interventions, were interpreted as implementation challenges rather than distinct strategies and were therefore collapsed into higher-level themes. This additional step in the process helped reduce fragmentation in the themes and contributed to consolidating a smaller, coherent core set of falls prevention implementation strategies.

3.4. Theme validation

During the validation phase, a panel of internal and external clinical experts reviewed 235 themes related to fall prevention strategies. These themes, with total sentiment scores, were presented to clinical expert leads for focused discussions and decisions. Criteria were developed to validate themes based on their total sentiment score and relevance. Sentiment analysis supported theme selection by highlighting prevailing patterns within the themes. The team evaluated each theme in relation to nursing interventions and activities, patient-centered outcomes, and documentation practices. Special attention was given to clinical relevance, provider experience, and standardized practices within LTC settings. This expert validation process ensured that the final themes were both meaningful and applicable to real-world practice. It also supported refining machine-generated outputs, enhancing the accuracy and utility of the NLP-based methodology. Based on expert consensus, themes were defined as main themes and sub-themes (branching). The final set of main themes: falls risk assessment; falls prevention strategies; risk factors for falls; care plan development; post-fall assessments; falls tools; fall documentation, education, and training; huddles; organizational policies and rounding. These themes provide insight into the clinical interventions and practices related to fall strategies and help evaluate the implementation of the evidence-based guideline-tracking outlined in RNAO’s Falls BPG. Table 1 describes the themes and their subthemes generated by NLP and validated by experts and end users.

Table 1.

Extract of themes evolved from unstructured text analytics using natural language processing.

Main themes	Sub-themes
Fall risk assessment	Assessment upon admission, status change, improved/updated assessment, documenting assessment, and formalized/standardized assessments.
Fall prevention strategies	Common mitigation strategies include: high-risk of falls reviews, fact sheets, and documentation (e.g., board, pamphlet, brochures, poster equipment), precautions (e.g., falls mats), alarms (e.g., bed alarm, chair alarm), restraints, risk management strategies (e.g., screening, falls tracking).
Risk factors of falls	Materials (e.g., slippery floors, cluttered rooms, or falls occurring during toileting or bed transfers).
Care plan development	Care plan development (e.g., care plan focuses statements, universal fall precautions), care plan status, review, update, and family/caregiver involvement.
	Within the theme “plans”: individualized plan, picture care plan, program evaluation plan, organizational action plan, universal falls precaution plan.
Post-fall assessments	Universal falls precautions, incident reporting, removal of bed rails, high-low bed rails compliance, fall mats, assistive devices (walker, canes, wheelchairs), shoe size assessment, and education of family and person.
Falls tools	Picture care plan tool, brainstorming tools, reference tools, falls tracking tool, high risk monitoring tools, risk management tool, screening tool: Scotts, 4Ps (Pain, Position, Placement, and Personal Needs), Situation-Background-Assessment-Recommendation (SBAR) communication tool, Morse tool, post admission screening tool, post fall assessment tool, fall huddle tool, and audit tools.
Fall documentation	Progress notes, reports (shifts, incidents), new/updated/improved falls documentation, falls trend reporting, and frequency of reporting.
Education and training	The most frequently repeated themes with the highest scores: mandatory education for staff/family/residents; e-learning sessions; training on falls drills; modules; quality programs; resources and materials; and education on the model of delivery.
	Other themes included: education roll out (mandatory education, falls order set, falls prevention, Scott assessment, continence care, pressure ulcer prevention, restraints, rounding (comfort, mandatory, purposeful, intentional), Best Practice Guideline training/champions, knowledge exchange boards and bulletin boards, facility transfer records and development, new hire training, peer to peer training, post fall huddle training guide, quality group huddle, and interdisciplinary huddle).
Huddles	Monthly, weekly, and daily.
Organizational policies	Corporate pain and falls prevention policies: review/revise; neurological signs and policy revisions; medication-related policies, safety measures, and policies; mechanical lift and Safe Ambulation Lift and Transfer Team established documentation.
Rounding	Purposeful rounding, comfort care, intentional, frequent, hourly rounding, rounding documentation, and rounding initiation.

4. Discussion and implications

This study provides insight into the real-world implementation of falls prevention BPG recommendations in LTC settings through analysis of narrative clinical reports submitted to RNAO MyBPSO qualitative data repository. Evaluating changes in BPG implementation is essential for ongoing quality improvement, staff education, and identifying supports required for sustained implementation, as emphasized in prior implementation science literature [25].

Clear and recurring patterns were identified using NLP in how LTC homes operationalize falls prevention. Frequently documented and highly salient practices clustered around risk assessment, education and training, structured monitoring, and organizational policy support. The most consistently reported strategies included updated falls risk screening tools, comprehensive risk assessments linked to care plan updates, annual mandatory falls prevention training, formalized post-fall procedures, purposeful hourly rounding, and person-centered and family-focused education. These findings are consistent with established evidence demonstrating that multifactorial interventions – including risk assessment, individualized care planning, and education – are central to effective falls prevention in older adults [[5], [6], [7], [8], [9]].

Consistent with prior evidence linking hourly rounding to improved patient outcomes and reduced fall rates [26,27], this study extends existing research by identifying purposeful hourly rounding as a key supporting practice and policy within LTC settings. Purposeful rounding emerged as one of the most frequently documented and positively perceived implementation strategies, particularly when framed as intentional, structured, and aligned with organizational policy. The prominence of rounding as both a practice and policy intervention reinforces its role as a cornerstone strategy for translating evidence-based BPG recommendations into routine care, supporting both patient safety and staff responsiveness.

Despite strong evidence highlighting the importance of staff education and communication in falls prevention [28,29], persistent knowledge and awareness gaps among LTC staff were evident. While education was commonly reported as an implementation strategy, training efforts were often focused on initial uptake rather than on reinforcing documentation standards, follow-through actions, and environmental interventions. Despite strong evidence highlighting the importance of staff education and communication in falls prevention [28,29], persistent knowledge and awareness gaps among LTC staff were evident. While education was commonly reported as an implementation strategy, training efforts were often concentrated on initial uptake rather than ongoing reinforcement of documentation standards, follow-through actions, and environmental interventions. This finding aligns with prior research demonstrating that sustained implementation of evidence-based interventions requires continuous education, reinforcement, and competency-based approaches rather than one-time training initiatives [[10], [11], [12],30,31].

These findings highlight the need for targeted, annual, mandatory, competency-based falls prevention education to support sustained implementation of BPG recommendations. Regular mandatory education may also mitigate barriers to timely uptake, particularly those related to delays in integrating updated clinical guidelines into routine practice [30,31].

Importantly, high reported adoption of guideline-aligned interventions did not consistently translate into full implementation fidelity. Persistent gaps in documentation quality, consistency, and policy-level integration indicate that operational processes remain central determinants of sustained BPG implementation. Documentation related to falls tracking and monitoring, such as progress notes, incident and shift reports, and trend analyses, was frequently observed; however, longer-term follow-up actions, including care plan revisions, medication adjustments, and referrals to allied health services, were less consistently documented. Strengthening policy integration, documentation quality, and tracking mechanisms, therefore, represents a critical target for long-term sustainability [32]. These findings align with prior evidence demonstrating that structured documentation, including care plans and incident reports, supports falls prevention by improving awareness of risk factors and enabling timely risk reduction actions, particularly through comprehensive post-fall documentation [30].

The results further underscore the central role of nursing and staff engagement in supporting resident-centered care. Tailored interventions, individualized care planning, resident and family education, and risk-informed prevention strategies emerged as important contributors to sustained BPG-aligned practice [31]. At the same time, variability in terminology, practice descriptions, and references to organizational policy highlights ongoing challenges in standardization across LTC settings. Such variability has been widely reported in implementation research and reflects differences in local context, resources, and organizational readiness [10,11].

Sentiment analysis further enriches these findings by identifying how implementation strategies are experienced in practice. Structured and intentional rounding practices were associated with positive sentiment, whereas documentation burden and incomplete implementation were associated with neutral or negative sentiment. These findings align with broader evidence suggesting that implementation challenges are often linked to workflow burden, resource constraints, and system-level barriers rather than resistance to evidence-based practices themselves [11,12]. This distinction is important, as it suggests that targeted system-level improvements – rather than changes to the interventions themselves – may be most effective in strengthening implementation.

Beyond generating actionable insights, this study demonstrates the value of large-scale analysis of clinical documentation for evaluating implementation fidelity. Traditional approaches such as manual chart audits and self-reported measures are resource-intensive, subjective, and difficult to scale [33,34]. In contrast, NLP-enabled analysis of narrative documentation allows for systematic, reproducible, and scalable evaluation of practice patterns across organizations. This aligns with emerging evidence on the application of NLP in healthcare, which highlights its potential to extract meaningful insights from unstructured clinical data and support quality improvement initiatives [19,32]. This is relevant to all health settings, particularly in LTC environments, where resource constraints and variability in documentation practices are common [35].

Collectively, these findings contribute to the growing body of literature on implementation science in LTC by demonstrating how NLP can help bridge the gap between guideline recommendations and real-world practice. By identifying both high-frequency interventions and persistent gaps, this study provides a data-driven foundation for targeted quality improvement efforts, supporting the sustained implementation of evidence-based falls prevention strategies.

5. Limitations and challenges

This study has several limitations that warrant consideration. First, the analysis relied primarily on self-reported data from the MyBPSO reporting system, which may not fully capture all clinical actions taken in LTC homes. Documentation may reflect what is required or expected rather than what was performed, introducing potential bias. Second, variability in documentation quality and style across providers and LTC homes may have influenced the accuracy of the NLP analysis. Inconsistent terminology, incomplete entries, or differing levels of detail could affect the tool’s ability to detect adherence to specific BPG components. While this approach enabled focused analysis within a specific jurisdiction, consideration will be given to the generalizability or adaptability of findings to other geographic regions and care settings. Additionally, the NLP model was developed and applied in an LTC setting, which has unique approaches to caring for the older adult population; this may limit the generalizability of the findings to other settings without further adaptation and validation. Despite these limitations, the study provides important insights into how AI-based tools can complement traditional implementation monitoring approaches.

6. Conclusions

The findings highlight important opportunities for LTC homes to strengthen systematic, proactive environmental safety practices, which are emphasized across falls-prevention frameworks but remain inconsistently implemented in practice. By leveraging the identified themes and subthemes, LTC homes can better understand key barriers and facilitators to BPG implementation and translate these insights into actionable quality improvement strategies. These insights can inform workflow redesign, staff education, and policy refinement, ultimately contributing to improved falls-prevention outcomes and resident safety in LTC homes. The study also demonstrates the value of NLP as a scalable, objective approach to evaluating implementation fidelity in real-world clinical settings. Unlike traditional manual audits, NLP enables automated, continuous analysis of large-scale unstructured narrative reports, making the evaluation of falls-prevention BPG implementation more efficient and feasible at scale.

Data availability statement

The datasets generated and/or analyzed during the current study are derived from the RNAO Nursing Quality Indicator for Reporting and Evaluation data system® (NQuIRE), with restricted access to researchers and evaluators of the Best Practice Guideline Program at RNAO. Due to data privacy, this data is not public; see Nursing Quality Indicators for Reporting and Evaluation® | RNAO.ca.

CRediT authorship contribution statement

Shanoja Naik: Conceptualization, Validation, Methodology, Project administration, Formal analysis, Data curation, Visualization, Writing – original draft, Writing – review & editing, Investigation. Hasitha Wimalarathna: Writing – review & editing. Alexa Parousis: Writing – review & editing. Doris Grinspun: Conceptualization, Writing – review & editing, Methodology, Funding acquisition.

Funding

This work was funded by the Government of Ontario. All work produced by the Registered Nurses’Association of Ontario is editorially independent from its funding source.

Declaration of competing interest

The authors have declared no conflict of interest.

Appendices. Supplementary data

The following are the Supplementary data to this article.