Skip to main content
NCBI home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2025 Mar 1.
Published in final edited form as: J Patient Saf. 2023 Dec 26;20(2):119–124. doi: 10.1097/PTS.0000000000001193

Enhancing pressure injury surveillance using Natural Language Processing (NLP)

Carly E Milliren 1, Al Ozonoff 2,3, Kerri A Fournier 1, Jennifer Welcher 4, Assaf Landschaft 5, Amir A Kimia 3,5
PMCID: PMC10922576  NIHMSID: NIHMS1941850  PMID: 38147064

Abstract

Objective

This study assessed the feasibility of nursing handoff notes to identify under-reported hospital acquired pressure injury (HAPI) events.

Methods

We have established a Natural Language Processing (NLP) assisted manual review process and workflow for data extraction from a corpus of nursing notes across all medical inpatient and ICUs in a tertiary care pediatric center. This system is trained by two domain experts. Our workflow started with keywords around HAPI and treatments, then regular expressions, distributive semantics, and finally a document classifier. We generated three models: a Tri-gram classifier, binary logistic regression model using the regular expressions as predictors, and a random forest model utilizing both models together. Our final output presented to the event screener was generated using a random forest model validated using derivation and validation sets.

Results

Our initial corpus involved 70,981 notes over a one-year period from 5,484 unique admissions for 4,220 patients. Our inter-rater human reviewer agreement on identifying HAPI was high (Ҡ=0.67; 95% CI 0.58-0.75). Our random forest model had 95% sensitivity (95% CI 90.6%-99.3%) 71.2% specificity (95% CI 65.1%-77.2%), and 78.7% accuracy (95% CI 74.1-83.2%). n=264 notes from 148 unique admissions (2.7% of all admissions) were identified describing likely HAPI. n=61 described new injuries, n=64 describe known yet possibly evolving injuries. Relative to the total patient population during our study period, HAPI incidence was 11.9 per 1,000 discharges and incidence rate 1.2 per 1,000 bed-days.

Conclusions

NLP based surveillance is proven to be feasible and high yield using nursing handoff notes.

INTRODUCTION

Hospital-acquired pressure-induced skin and soft tissue injuries are among the most common yet burdensome iatrogenic injuries encountered in both acute and long-term hospitalizations.1-3 Nearly 2.5 million pressure injuries are treated in the US annually at a cost of nearly $27 billion.2-5 Pressure-induced injuries are associated with higher rates of hospital-acquired infections and prolonged hospital stays.6-11 The number of such injuries continues to grow in part due to an aging population, as well as longer survival of patients with multiple comorbidities and poor mobility.1 Hospital-acquired pressure injuries (HAPIs) are no longer limited to the geriatric population; prevalence of HAPIs in pediatric patients range from 2.4 to 7.7 per 1000 discharges, with rates from 7% to 27% reported in patients admitted to intensive and critical care units.8-11

HAPIs occur due to skin and soft tissue ischemia secondary to prolonged pressure, most commonly over a bony prominence, resulting in localized damage and ulceration at later stages.1 Sustained external pressure may be applied to soft tissue in bedbound hospitalized patients due to hard surfaces associated with hospital beds as well as secondary to medical equipment and devices, and can begin to injure tissue in as little as four hours.12-14 Non-blanchable erythema with intact skin is the earliest sign of HAPI development, so-termed a Stage I pressure injury per the National Pressure Injury Advisory Panel (NPIAP).14 Multiple systems, such as the Norton and Braden scales,15 describe HAPIs and identify patients at high risk, yet the study of HAPIs continues to be a challenging task due to inconsistent reporting and billing practices as well as barriers to consistent surveillance.

Healthcare reporting of HAPIs via the electronic health record (EHR) is increasingly utilized as a source of data for the study of pressure injuries.16,17 There is data collection and clinical protocols to detect HAPIs.6,18,19 A further complication to the study of HAPIs arises from discrepancies between International Classification of Diseases-9th and 10th Edition (ICD-9/10) codes and submitted federal and private insurer claim codes.6,20 Early-stage pressure injuries are often not coded, and treatment of Stage III/IV pressure injuries are not reimbursed by Medicaid, Medicare, and some insurers if the injury was not present at admission. Thus HAPIs are not always captured by code-based review of the EHR as they may have not been submitted depending on the institution.4,21 While national quality improvement (QI) efforts have sought to address some of these inconsistencies, deviation from protocol, low inter-rater reliability, and inclusion of only high-risk patients limit the generalizability of studies they produce.22-24

Any surveillance system limited to a single source is flawed. This is particularly true of self-report systems involving HAPIs, which are partially preventable. Our main hypothesis is that while these events are under-reported to systems external to the Electronic Health Record (EHR), healthcare providers will provide an accurate summary in their handoff notes to assure continuity of care. We further hypothesize that the textual data from these notes can be mined efficiently from the EHR, using Natural Language Processing (NLP) to screen large numbers of handoff notes. We therefore sought to assess the feasibility of applying NLP methodology to handoff notes within the EHR, as an HAPI surveillance method complimentary to self-reporting system.

METHODS

Study Design

We performed a retrospective cross-sectional study of consecutive patients admitted at a single urban tertiary care pediatric hospital. Electronic medical records of patients admitted from January 2019 to December 2019 were reviewed electronically (i.e., all were screened through an NLP screening model, with only a small subset being screened manually (see elaborated methods below). The study protocol was determined to be non-human subjects research and exempt from further ethical review by our institution’s Institutional Review Board.

Study Population

We included all patients ≤ 21 years of age who were admitted to inpatient medical units, both regular floor and intensive care unit (ICU) level. Patients in the neonatal intensive care unit (NICU) were excluded, as neonates represent a unique patient population that is likely not generalizable to other care settings. We obtained nursing handoff documentation and accompanying supplements, including pain treatment, event reports, and ICU event reports. We also extracted aggregate total bed-days and number of inpatient discharges for the same clinical units and patient populations during our study time period.

Case Identification

Case identification was conducted in three phases: 1) leverage domain expertise to establish a training set for document classifier; 2) create and apply an tri-gram NLP model and manually review the NLP model output; and 3) combine the previous two methods using a random forest model. All cases were identified and verified by an experienced RN (KF). In addition, 7% of the data was reviewed by an experienced Pediatric and Pediatric Emergency Medicine specialist (AK), to assure adequate inter-rater agreement.

Creating and training the NLP model

In this section we describe the process to train a medical expert without formal training in NLP methods to use NLP methodology effectively and efficiently. The approach allows a new user to progress from the familiar (key word search) to the sophisticated (applying Tri-gram document classifiers), through the process of labeling and identifying documents addressing pressure injuries within the corpus. The overall workflow is illustrated in Figure 1.

Figure 1.

Figure 1.

Open in a new tab

Workflow of training and validating NLP models by a lay NLP user.

1. Establishing the very first dataset, from keywords to regular expressions.

In order to train an NLP model, we must establish a training set of documents, consisting of both nursing handoff notes describing pressure injuries and notes that do not describe such injuries. For that purpose, we created a computer-assisted screening tool25 similar to keyword search tools, using regular expressions (RegEx).26-28 The technique of RegEx matching29 provides a more comprehensive and inclusive search than simple keyword searching, because it includes possible misspelled and mistyped variations of the keyword(s) of interest. We use a Graphic User Interface (GUI) which allows the lay user to begin with keywords and progressively develop RegEx related to the keywords chosen. As one example, RegEx can expand the key word ‘ulcer’ to capture ‘ulceration’ while omitting negative expressions such as ‘no ulcer’, and unrelated expressions such as ‘peptic ulcer’ or ‘mouth ulcer’. Our home-grown GUI is called DrT (Document Review Tool) which has been applied in numerous prior studies including patient safety surveillance as well as cohort identification for clinical and health services research.30-33 RegEx can be used for combinations of words as well as combining expressions; for example if the user inputs ‘pressure ulcer’ and ‘decubitus sores’ to describe pressure injuries, the RegEx might evolve to ‘(pressure∣decub\w*)\s*(\bsore(?)\b∣ulcer(s)?)’ which would capture combinations including ‘pressure sore’, ‘pressure sores’, ‘pressure ulcer’, ‘pressure ulcers’, ‘decub ulcer’, ‘decubitus sores’, etc. while omitting expressions such as ‘resolved pressure ulcer’, ‘unlikely to be a pressure sore’, etc. Figure 2 presents a screenshot of our GUI, and Table 1 presents a list of RegEx developed during the process of identifying pressure injuries.

Figure 2.

Figure 2.

Open in a new tab

RegEx assisted document labeling.

A. List of RegEx created by the user; B. Note per patient sorted by score; C. Note narrative with highlighted RegEx; D. Categorization of notes.

Table 1.

Final list of regular expressions (RegEx) used to flag potential pressure injury notes.

List of Regular Expressions
\bdecub\w*\b \bdebr[ie]+d\w*\b \bstage\b
\bulcer[s]?\b breakdown topotecan
\bsore[s]?\b pressure\s*area(s)? \b(no[tn]e?)?[-\s]?blanch\w*\b
air mattress (pressure∣decub\w*)\s*(ulcer(s)?∣\bsore(s)?\b∣injur\w*) Wound∣ abrasion
Sheep[`s]?\s+skin erythema\w* leak
\bsacr\w*\b∣\bcocc\w*\b diaper\s*rash \d(\.\d+)?\s*(cm)?\s*x\s*\d(\.\d+)?\s*(cm)?
xeroform \btear\b z\s*[-]?flow
duoderm∣Mepilex∣allevym \bbridge\b \bg[-]?\s*t(ube)?\b
Open in a new tab

2. The use of distributional semantics

Once the user has exhausted expressions to be used for RegEx, we use distributional semantics to offer the user expressions that are likely to be associated with some index terms. Distributional semantics is a research area that develops and studies theories and methods for quantifying and categorizing semantic similarities between linguistic items based on their distributional properties in large samples of language data.34 For example, the words ‘ulcer’ and ‘sore’ are quite similar, although, not exact synonyms. The word ‘debridement’ appears in many sentences where ‘pressure ulcer’ may appear, etc.

Figure 3 shows a list of distributional semantics terms offered to the user based on the term. For each new term that is added from distributional semantics list, the user then writes a RegEx to expand the capture of the new term added.

Figure 3.

Figure 3.

Open in a new tab

Distributional semantics assisted document labeling, building new RegEx

3. Training a Tri-gram classifier

Next, the output of the computer-assisted tool is used as input for an automated document classifier. Once trained on a certain dataset, the automated classifier screens unlabeled documents and assigns a continuous score to each document based on its similarity to positive documents in the training set. A document classifier, unlike word/text matching, will also provide statistical metrics to the search (e.g. accuracy, sensitivity, specificity). We typically use a portion of the manually labeled documents to train a support vector machine (SVM) model and a separate human labeled subset to validate the model. Figure 4 presents the output of the document classifier using the validation set.

Figure 4.

Figure 4

Open in a new tab

Receiver Operating Characteristic (ROC) curve for each of the three models evaluated.

4. Putting the final RegEx to another use

The domain expertise of the user is encoded in RegEx, including the type of expression and the additional look-around terms (words that precede or follow the expression of interest). Each RegEx can appear zero or more times in any given document, making that RegEx a potential predictor of a document of interest. Once enough documents are labeled as positive or negative, the RegEx can be used in a multivariable logistic regression model. RegEx may be a standalone predictor or one of several predictors within an SVM model.

5. Identifying multiple documents describing the same event

A nursing handoff note describing a pressure injury does not necessarily identify a singular and unique pressure injury event. Multiple notes during handoff may describe the same event. As pressure injuries are associated with slow recovery, we automatically assigned a category of ‘note in temporal proximity to an identified event’ to all notes that either preceded or followed the positively labeled note within an 18h window. After reviewing our initial 150 notes we felt that at that point, repetitive description of the known ulcer may be diminishing, making ulcer-describing notes slightly more likely to describe a new event. This reduced the number of notes that required human review to ascertain an event, although it is possible that we failed to identify a second pressure injury described within that timeframe. We opted to accept this limitation as only a portion of pressure injuries will be followed immediately by a new one, and following day documentation describing an ulcer will most likely be a description of the known95% injury.

6. Combining models using a random forest

We use a random forest model to combine the document classifier SVM model and the RegEx logistic regression model. The output of the final model is used to assess the overall performance of the project.

Our overall workflow and number of documents reviewed are presented in figure 5.

Figure 5.

Figure 5.

Open in a new tab

Overall data screening workflow.

Outcome Measures

Our primary outcome was classification of documents as positive or negative for describing a pressure ulcer. After manual review of a subsample (n=150 documents), we observed high variability within nursing documentation. Stage of ulcers was not uniformly documented, nor was the size or perceived depth. In some instances, the management of a wound was described without mention of the wound type or nature (e.g. ulcer, surgical wound, rash). We therefore coined three levels of ‘positive’ documentation: a) new pressure injury/ulcer; b) known injury that has a worsening pattern; and c) possible early pressure injury, unclassified. Within ‘negative’ documents the categories included but were not limited to: notes discuss past ulcers or ulcer tendency, notes describing preventive measures for pressure injuries but no active insult, previously known pressure injuries, and other false positives. For cases where it was unclear whether an ulcer was involved or not, the note was classified as ‘unclear’.

Statistical Analysis

For the note classifier, we used a Tri-gram SVM to generate a continuous score for likelihood of any given document describing a pressure injury. For use of RegEx as predictors, we used a binary logistic regression model. We combined the SVM and binary logistic model using a random forest model.

We used Bayesian credible intervals to calculate percentages and confidence intervals for the prevalence of the pressure ulcer events. We use the chance-corrected kappa statistic to describe inter-rater agreement.

Using aggregate data on the number of discharges and bedded days during our study time period we calculated the incidence of pressure injury events per 1,000 inpatient discharges and per 1,000 bed-days.

Data were analyzed using IBM SPSS for Windows V27 (Chicago, 2019).

RESULTS

During the study period there were 70,981 notes available for review, these notes represent 5,484 unique admissions involving 4,220 patients. The patents median age was 5.5 years (IQR 1.3-14.2), and 45% were female. The length of stay ranged from 1 day to 1-year and11 months, the median was 4.5 days (IQR 1-13). The number of notes per patient ranged from 1-531 with a median number of notes per patient of 5 (IQR 3-10). Of these, 69,436 notes were nursing handoff notes, 1,091 were pain assessment notes, 300 were addendums for event nursing, and 64 were ICU event nursing notes. After assessment and retraining, the RegEx chosen by the domain expert are presented in Table 1 (excluding look-around lists which are too long to present).

Manual Review of flagged notes and retraining of models

A total of 944 notes were flagged for manual review (944/70981, 1.3% of all notes). An additional 2654 notes were identified as describing known events (2654/70981, 3.7% of all notes). The automated labeled notes were not used to train further models. The process of classifying a note based on highlighted RegEx is presented in Figure 2.

Prior to using a training set for a document classifier, we assessed the inter-rater agreement between the RN and the MD labeling the data to assure an objective review is used for trainingan AI model. Inter-rater agreement on identifying notes presented to the reviewer (i.e. ‘positive’ notes) versus notes unrelated to active pressure injuries was Ҡ=0.67 (95% confidence interval 0.58-0.75), indicating substantial agreement.

We present Figure 3 to demonstrate distributional semantics. For example, for the word ‘ulcer’ the expressions ‘debridement’ and ‘sacral’ appears in many sentences where ‘pressure ulcer’ also appears.

Model Performance

For each one of our models, in order to achieve the goal of identifying unreported events, we aimed at a desired sensitivity of 95%. A lower target for sensitivity would leave many events unreported, while a higher target would result in more false positives. While doing so we attempted to keep a reasonable specificity to assure that if the output will be manually reviewed the PPV is also reasonable. Our model sensitivities presented in Table 2 ranged from 93.7-95%.

Table 2.

Model performance

Model Sensitivity
(95% CI)		 Specificity
(95% CI)		 Accuracy
(95% CI)
N-Gram Classifier 93.7% (91.0% - 96.1%) 51.6% (48.2% - 54.9%) 65.9% (62.3% - 67.5%)
Binary Logistic Regressions using RegEx 94.5% (92.3% - 96.7%) 36.4% (33.2% - 39.6%) 65.9% (62.3% - 67.5%)
Random Forest 95% (90.6% - 99.3%) 71.2% (65.1% - 77.2%) 78.7% (74.1% - 83.2%)
Open in a new tab

Final cohort of pressure injuries

In total, 264 notes were identified describing likely pressure injuries, of which 61 described new injuries and 64 describe known yet possibly evolving pressure injuries. These notes represent 148 unique admissions (148/5484, 2.7% of all admissions). Relative to the total patient population during our study time period, the observed incidence of HAPI was 11.9 per 1,000 discharges and the incidence rate accounting for patient days was 1.2 per 1,000 bed-days. Sixteen admissions involved both a new and a worsening pressure injury.

DISCUSSION

We present a workflow and methods demonstrating how NLP methodology can be efficiently applied for HAPI surveillance in nursing handoff notes with high sensitivity and specificity. This methodology can be used by domain experts with little computer science background. These results demonstrate the ability to perform an NLP-assisted chart review on large scale of data for daily of weekly assessment of HAPI within an institution.

Our findings and process add to the body of literature describing the applicability of NLP methods for extracting information from clinical notes for use in patient safety surveillance.30 The final random forest model had very high sensitivity at 95% with nearly 80% accuracy. These findings along with those from a recent study using similar tools to identify peripheral intravenous infiltrates demonstrate the feasibility of applying these tools for safety surveillance in a hospital setting.30 Robust electronic patient safety surveillance systems are needed to accurately capture events and facilitate meaningful improvements for patients across the spectrum of, unfortunately common, medical errors and hospital acquired conditions.

Our findings indicate an incidence of HAPI events of 11.9 per 1,000 discharges which is higher than previously reported estimates ranging from 2.4 to 7.7 per 1,000 discharges.8-11 This discrepancy may be partly explained by nursing notes capturing early and evolving pressure injuries that are not captured in other reporting systems i.e. voluntary safety reporting. Nursing notes may provide a comprehensive and complementary data source that could be leveraged to augment voluntary safety reports and other reporting systems. Event identification using notes may be more likely to capture low severity or ‘near-miss’ events which would offer a more complete picture of patient safety thereby informing intervention or prevention efforts, which is undoubtedly a net benefit even if the initial burden of increased reporting is high. Using narrative data for patient safety related events poses a few challenges, particularly in interpretation of the data. With the competing data source being a self-reported log, possibly by the same provider, additional steps may be required to assess the newly identified under-reported cases. Patient safety events are often under-reported for a variety of reasons including providers perception of reporting systems 35, concerns about implications to the reporter, time consuming36 and more. Surveys, demonstrate that providers report near-miss events more readily than harm-associated events37, which is the case at hand with pressure injuries. However, there may also be actual clinical differences between cases reported and those failing to meet certain provider perceived standards for reporting. The authors of this manuscript who are about to implement an active surveillance moving forward believe that the best way to overcome some of these factors is to use the identified cases as basis for generating a discussion about a possible self-reported log rather than accepting the event based of clinical documentation alone.

Another challenge will be to see if clinical documentation will change over time once it is used as a source for surveillance. One might think that providers worried about punitive actions may change the reports placed in the EHR. Conversely, we may see that surveillance systems like ours generating a conversation about an event thus mitigating the ‘snitching’ on a newly discovered event.

This study has a number of limitations. First, this methodology to identify HAPI from nursing notes was evaluated at a single pediatric tertiary care hospital and have not yet generalized to other institutions including adult tertiary care or community hospitals. Although the approach is generalizable across care settings, future work is needed to validate these tools and approaches in a variety of settings and patient populations. Second, we utilized nursing notes as an independent source of data to capture HAPI events and were not able to cross-reference to events captured by self-reported safety event reporting or other hospital systems. Third, our trade off of not reviewing next day documentation is likely to be associated with some injuries that will be missed. This will be a challenge to address in the next iteration of our system. Fourth, we only reviewed 7% of the data for inter-rater agreement, yet with this large number of notes reviewed there is still a narrow confidence interval around these results. Finally, we were unable to characterize severity of pressure injuries. It is possible that events captured by our approach that were not previously reported in other hospital systems are more likely to be of low severity.

In summary, NLP methodology is becoming more and more accessible to lay providers using systems such as ours. The largest component of the EHR, the narrative data, is proven to be a very useful source for patient safety in general, and health acquired events specifically.30

Conflicts of Interest and Source of Funding:

This study was funded by the Agency for Healthcare Research and Quality (grant number: R01 HS026246 01A1). This work was conducted with support from Harvard Catalyst ∣ The Harvard Clinical and Translational Science Center (National Center for Advancing Translational Sciences, National Institutes of Health Award, Award UL1 TR002541) and financial contributions from Harvard University and its affiliated academic healthcare centers. The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic healthcare centers, or the National Institutes of Health. There were no other conflicts of interest declared.

References

  • 1.Li Z, Lin F, Thalib L, Chaboyer W. Global prevalence and incidence of pressure injuries in hospitalised adult patients: A systematic review and meta-analysis. Int J Nurs Stud. May 2020;105:103546. doi: 10.1016/j.ijnurstu.2020.103546 [DOI] [PubMed] [Google Scholar]
  • 2.VanGilder C, Amlung S, Harrison P, Meyer S. Results of the 2008-2009 International Pressure Ulcer Prevalence Survey and a 3-year, acute care, unit-specific analysis. Ostomy Wound Manage. Nov 1 2009;55(11):39–45. [PubMed] [Google Scholar]
  • 3.Reddy M, Gill SS, Rochon PA. Preventing pressure ulcers: a systematic review. JAMA. Aug 23 2006;296(8):974–84. doi: 10.1001/jama.296.8.974 [DOI] [PubMed] [Google Scholar]
  • 4.Padula WV, Delarmente BA. The national cost of hospital-acquired pressure injuries in the United States. Int Wound J. Jun 2019;16(3):634–640. doi: 10.1111/iwj.13071 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Lyder CH. Pressure ulcer prevention and management. JAMA. Jan 8 2003;289(2):223–6. doi: 10.1001/jama.289.2.223 [DOI] [PubMed] [Google Scholar]
  • 6.Zhang W, Sotoodeh M, Ho JC, Simpson RL, Hertzberg VS. Examining the Concordance in the Documented Pressure Injury Site, Stage, and Count in Medical Information Mart for Intensive Care-III. Appl Clin Inform. Aug 2021;12(4):897–909. doi: 10.1055/s-0041-1735179 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Scheckler WE, Peterson PJ. Infections and infection control among residents of eight rural Wisconsin nursing homes. Arch Intern Med. Oct 1986;146(10):1981–4. [PubMed] [Google Scholar]
  • 8.Schluer AB, Schols JM, Halfens RJ. Risk and associated factors of pressure ulcers in hospitalized children over 1 year of age. J Spec Pediatr Nurs. Jan 2014;19(1):80–9. doi: 10.1111/jspn.12055 [DOI] [PubMed] [Google Scholar]
  • 9.Razmus I, Bergquist-Beringer S. Pressure Injury Prevalence and the Rate of Hospital-Acquired Pressure Injury Among Pediatric Patients in Acute Care. J Wound Ostomy Continence Nurs. Mar/Apr 2017;44(2):110–117. doi: 10.1097/WON.0000000000000306 [DOI] [PubMed] [Google Scholar]
  • 10.Miller MR, Zhan C. Pediatric patient safety in hospitals: a national picture in 2000. Pediatrics. Jun 2004;113(6):1741–6. doi: 10.1542/peds.113.6.1741 [DOI] [PubMed] [Google Scholar]
  • 11.Delmore B, Deppisch M, Sylvia C, Luna-Anderson C, Nie AM. Pressure Injuries in the Pediatric Population: A National Pressure Ulcer Advisory Panel White Paper. Adv Skin Wound Care. Sep 2019;32(9):394–408. doi: 10.1097/01.ASW.0000577124.58253.66 [DOI] [PubMed] [Google Scholar]
  • 12.Thomas DR. Does pressure cause pressure ulcers? An inquiry into the etiology of pressure ulcers. J Am Med Dir Assoc. Jul 2010;11(6):397–405. doi: 10.1016/j.jamda.2010.03.007 [DOI] [PubMed] [Google Scholar]
  • 13.Gefen A. How much time does it take to get a pressure ulcer? Integrated evidence from human, animal, and in vitro studies. Ostomy Wound Manage. Oct 2008;54(10):26–8, 30-5. [PubMed] [Google Scholar]
  • 14.Coleman S, Nixon J, Keen J, et al. A new pressure ulcer conceptual framework. J Adv Nurs. Oct 2014;70(10):2222–34. doi: 10.1111/jan.12405 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Bergstrom N, Braden BJ, Laguzza A, Holman V. The Braden Scale for Predicting Pressure Sore Risk. Nurs Res. Jul-Aug 1987;36(4):205–10. [PubMed] [Google Scholar]
  • 16.Jin Y, Jin T, Lee S-M. Automated Pressure Injury Risk Assessment System Incorporated Into an Electronic Health Record System. Nursing Research. 2017;66(6):462–472. doi: 10.1097/nnr.0000000000000245 [DOI] [PubMed] [Google Scholar]
  • 17.Kaewprag P, Newton C, Vermillion B, Hyun S, Huang K, Machiraju R. Predictive models for pressure ulcers from intensive care unit electronic health records using Bayesian networks. BMC Med Inform Decis Mak. Jul 5 2017;17(Suppl 2):65. doi: 10.1186/s12911-017-0471-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Weller CD, Gershenzon ER, Evans SM, Team V, McNeil JJ. Pressure injury identification, measurement, coding, and reporting: Key challenges and opportunities. Int Wound J. Jun 2018;15(3):417–423. doi: 10.1111/iwj.12879 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Squitieri L, Ganz DA, Mangione CM, et al. Consistency of pressure injury documentation across interfacility transfers. BMJ Qual Saf. Mar 2018;27(3):182–189. doi: 10.1136/bmjqs-2017-006726 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Coomer NM, McCall NT. Examination of the accuracy of coding hospital-acquired pressure ulcer stages. Medicare Medicaid Res Rev. 2013;3(4)doi: 10.5600/mmrr.003.04.b03 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Padula WV, Makic MB, Wald HL, et al. Hospital-Acquired Pressure Ulcers at Academic Medical Centers in the United States, 2008-2012: Tracking Changes Since the CMS Nonpayment Policy. Jt Comm J Qual Patient Saf. Jun 2015;41(6):257–63. doi: 10.1016/s1553-7250(15)41035-9 [DOI] [PubMed] [Google Scholar]
  • 22.Waugh SM, Bergquist-Beringer S. Inter-Rater Agreement of Pressure Ulcer Risk and Prevention Measures in the National Database of Nursing Quality Indicators((R)) (NDNQI). Res Nurs Health. Jun 2016;39(3):164–74. doi: 10.1002/nur.21717 [DOI] [PubMed] [Google Scholar]
  • 23.Hart S, Bergquist S, Gajewski B, Dunton N. Reliability testing of the National Database of Nursing Quality Indicators pressure ulcer indicator. J Nurs Adm. Oct 2010;40(10 Suppl):S16–25. doi: 10.1097/NNA.0b013e3181f37d15 [DOI] [PubMed] [Google Scholar]
  • 24.Hart S, Bergquist S, Gajewski B, Dunton N. Reliability testing of the National Database of Nursing Quality Indicators pressure ulcer indicator. J Nurs Care Qual. Jul-Sep 2006;21(3):256–65. doi: 10.1097/00001786-200607000-00011 [DOI] [PubMed] [Google Scholar]
  • 25.AA K. DocumentReviewTools. Accessed 01.02.2021, 2021. https://documentreviewtools.com/ [Google Scholar]
  • 26.Kimia AA, Savova G, Landschaft A, Harper MB. An Introduction to Natural Language Processing: How You Can Get More From Those Electronic Notes You Are Generating. Pediatr Emerg Care. Jul 2015;31(7):536–41. doi: 10.1097/PEC.0000000000000484 [DOI] [PubMed] [Google Scholar]
  • 27.Friedl J, Safari Books Online (Firm). Mastering Regular Expressions, 3rd Edition. 3rd edition ed.:1 online resource (542 pages). https://go.oreilly.com/pennsylvania-state-university/library/view/-/0596528124/?ar [Google Scholar]
  • 28.Nield T. An introduction to regular expressions : decoding simple regex features to match complex text patterns. First edition. ed. 2019:1 online resource (1 volume). https://go.oreilly.com/massachusetts-institute-of-technology-mit/library/view/-/9781492082569/?ar MIT Access; Only [Google Scholar]
  • 29.Friedl JEF. Mastering regular expressions. 3rd ed. O'Reilly; 2006:xxiv, 515 p. [Google Scholar]
  • 30.Ozonoff A, Milliren CE, Fournier K, et al. Electronic surveillance of patient safety events using natural language processing. Health Informatics J. Oct-Dec 2022;28(4):14604582221132429. doi: 10.1177/14604582221132429 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Paydar-Darian N, Kimia AA, Monuteaux MC, et al. C-reactive protein or erythrocyte sedimentation rate results reliably exclude invasive bacterial infections. The American Journal of Emergency Medicine. 2019/August/01/ 2019;37(8):1510–1515. doi: 10.1016/j.ajem.2018.11.011 [DOI] [PubMed] [Google Scholar]
  • 32.Johnson KB, Michelson KA, Lyons TW, et al. Pediatric status epilepticus: How common is cerebrospinal fluid pleocytosis in the absence of infection? Seizure. 2014/August/01/ 2014;23(7):573–575. doi: 10.1016/j.seizure.2014.03.015 [DOI] [PubMed] [Google Scholar]
  • 33.Stewart AM, Kanak MM, Gerald AM, et al. Pediatric Emergency Department Visits for Homelessness After Shelter Eligibility Policy Change. Pediatrics. Nov 2018;142(5)doi: 10.1542/peds.2018-1224 [DOI] [PubMed] [Google Scholar]
  • 34.Cohen T, Widdows D. Empirical distributional semantics: Methods and biomedical applications. Journal of Biomedical Informatics. 2009/April/01/ 2009;42(2):390–405. doi: 10.1016/j.jbi.2009.02.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Skutezky T, Small SS, Peddie D, Balka E, Hohl CM. Beliefs and perceptions of patient safety event reporting in a Canadian Emergency Department: a qualitative study. CJEM. Dec 2022;24(8):867–875. doi: 10.1007/s43678-022-00400-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Rutledge DN, Retrosi T, Ostrowski G. Barriers to medication error reporting among hospital nurses. J Clin Nurs. May 2018;27(9-10):1941–1949. doi: 10.1111/jocn.14335 [DOI] [PubMed] [Google Scholar]
  • 37.Burlison JD, Quillivan RR, Kath LM, et al. A Multilevel Analysis of U.S. Hospital Patient Safety Culture Relationships With Perceptions of Voluntary Event Reporting. J Patient Saf. Sep 2020;16(3):187–193. doi: 10.1097/PTS.0000000000000336 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES