Abstract
Background: Natural language processing (NLP), a form of computer-assisted data abstraction, rapidly identifies serious illness communication domains such as code-status confirmation and goals of care (GOC) discussions within free-text notes, using a codebook of phrases. Differences in the phrases associated with palliative care for patients with different types of illness are unknown.
Objective: To compare communication of code-status clarification and GOC discussions between patients with advanced pancreatic cancer undergoing palliative procedures and patients admitted with life-threatening trauma.
Design: Retrospective cohort study.
Setting/Subjects: Patients with in-hospital admissions within two academic medical centers.
Measurements: Sensitivity and specificity of NLP-identified communication domains compared with manual review.
Results: Among patients with advanced pancreatic cancer (n = 523), NLP identified code-status clarification in 54% of admissions and GOC discussions in 49% of admissions. The sensitivity and specificity for code-status clarification were 94% and 99% respectively, while the sensitivity and specificity for a GOC discussion were 93% and 100%, respectively. Using the same codebook in patients with life-threatening trauma (n = 2093), NLP identified code-status clarification in 25.9% of admissions and GOC discussions in 6.3% of admissions. While NLP identification had 100% specificity, the sensitivity for code-status clarification and GOC discussion was reduced to 86% and 50%, respectively. Adding dynamic phrases such as “ongoing discussions” and phrases related to “family meetings” increased the sensitivity of the NLP codebook for code status to 98% and for GOC discussions to 100%.
Conclusions: Communication of code status and GOC differ between patients with advanced cancer and those with life-threatening trauma. Recognition of these differences can aid in identification in patterns of palliative care delivery.
Keywords: natural language processing, palliative care communication, surgical palliative care
Introduction
Achallenge in studying serious illness communication is the identification of associated process measures, including code-status clarification and goals of care (GOC) discussions.1 While related administrative codes exist, they have a poor sensitivity for palliative care.2,3 Instead, documentation of code-status and GOC discussions are more likely to be found in free-text notes. Thus, identification of these process measures requires resource-intensive manual review.
Previously we reported on the use of natural language processing (NLP), a form of computer-assisted abstraction, to identify code-status clarification and GOC discussions within free-text notes.4 This methodology uses a computer algorithm to filter through electronic health records (EHRs) to identify selected words and phrases. These selected words and phrases constitute a “codebook,” which allows for rapid identification of palliative care process measures.
The sensitivity and specificity of NLP-identified process measures are reliant on the phrases included in the codebook. However, phrases associated with code-status clarification and GOC discussion may differ between patients with different types of serious illnesses. We undertook this study to compare the performance of NLP codebooks in patients with metastatic or unresectable pancreatic cancer undergoing palliative procedures and in patients admitted with life-threatening trauma. We chose these patient populations, because while both have serious illness and can benefit from palliative care, the chronicity and acuity differ between the two groups.5 Specifically, we hypothesized that patients with advanced pancreatic cancer would have more established care preferences, while patients with life-threatening trauma would have more dynamic goals reflecting rapid changes in their clinical state.
Materials and Methods
Data sources and patient populations
The Partners research patient data registry is a repository of clinical/administrative data that include international classification of disease, ninth revision and tenth revision diagnosis/procedure (ICD-9-CM/PCS, ICD-10-CM/PCS) and current procedure terminology (CPT) codes for all encounters at Massachusetts General Hospital and Brigham and Women's Hospital, two urban tertiary-care referral centers in the Partners HealthCare System. These data are linked to the EHR, providing admission notes, progress notes, and discharge summaries.
We included patients with advanced (metastatic or unresectable) pancreatic cancer (ICD-9-CM 157 or ICD-10-CM C25), who were ≥18 years of age, and who underwent a palliative procedure between January 1st, 2011 and April 30th, 2016. Palliative procedures were identified using CPT codes (Supplementary Table S1). The palliative indication for the procedure and presence of unresectable/metastatic disease were confirmed through review of admission, operative, and procedure notes.
Patients with life-threatening trauma were identified from the registries of the same two hospitals between July 1st, 2016 and June 30th, 2017. A severe injury was defined by (1) an Injury Severity Score ≥9 for patients ≥65 or (2) an Abbreviated Injury Scale ≥3 for a single region for patients ≥18. Scores in these ranges correspond to open fractures, perforated viscus, and intracranial hemorrhage.6 The Partners Institutional Review Board approved this study.
Identifying palliative care delivery
The primary outcome was the documentation of code-status clarification and GOC discussion. An NLP codebook containing keywords and phrases previously described by Lilley et al. was used to identify documentation code-status clarification and GOC.4 Code-status clarification was defined as a conversation with patients/surrogate decision-makers about preferences for cardiopulmonary resuscitation and intubation. GOC discussions were defined as a conversation between clinicians and the patient/surrogates regarding their values, goals, or priorities for treatment and outcomes. All clinical notes were assessed for the presence or absence of code-status clarification or GOC discussion by NLP.
Statistical analysis
The sensitivity and specificity of the NLP codebook in patients with advanced cancer was compared with a randomly selected set of 20 admissions through manual review.4 Sensitivity and specificity of NLP-identified code-status clarification and GOC discussion in trauma patients was compared with a randomly selected set of 40 admissions assessed through manual review. We chose to examine 40 trauma admissions given the reduced prevalence of palliative care process measures in this group.
Results
We identified 523 patients with advanced pancreatic cancer and 2093 trauma admissions. In patients with advanced pancreatic cancer, code-status clarification was identified in 54% of admissions and GOC discussions in 49%. Compared with manual review, the sensitivity and specificity for code-status clarification were 94% and 99%, respectively. The sensitivity and specificity for a GOC discussion were 93% and 100%, respectively.
In patients admitted with life-threatening trauma, the same NLP codebook identified code-status clarification in 26% of admissions, whereas GOC discussions were performed in 6% of admissions. Sensitivity and specificity, compared with manual review for code-status clarification, were 86% and 100%, respectively (Table 1). Likewise, the sensitivity and specificity for GOC discussion were 50% and 100%, respectively.
Table 1.
Sensitivity and Specificity of Natural Language Processing Compared with Gold Standard of Manual Review
| Admission for life-threatening trauma (n = 2093) |
Admission for a palliative procedure in patients with advanced pancreatic cancer (n = 523) |
|||
|---|---|---|---|---|
| Code-status clarification | GOC discussion | Code-status clarification | GOC discussion | |
| Sensitivity, % | 86 | 50 | 94 | 94 |
| Specificity, % | 100 | 100 | 99 | 100 |
GOC, goals of care.
We reviewed all trauma admission notes in which there was a discrepancy between NLP and manual review. For code-status clarification, the reduction in sensitivity was primarily attributed to language related to the continuation of treatment. Representative phrases, such as “wishes to remain full code,” “would like to remain full code,” and “remaining full code,” were more common in trauma patients compared with patients with advanced cancer. Additionally, there was language pertaining to more nuanced conversations about code-status limitations, such as “code reversal” or “ok to intubate.”
Similarly, differences in language surrounding GOC discussions helped explain the reduced sensitivity. GOC discussions in patients with acute traumatic injuries were more reliant on family members, and phrases related to family discussions were missed in the original codebook. GOC discussions were more frequently assessed and were more dynamic and “ongoing” in patients with traumatic injuries compared with patients with advanced pancreatic cancer. The full list of phrases and keywords added to the codebook are listed in Table 2.
Table 2.
Natural Language Processing Codebook
| Process measure and definition | Keywords |
|---|---|
| Clarifying code status: conversations with patients or family members about preferences for cardiopulmonary resuscitation and intubation. Includes limitations on life-sustaining treatment and clarification, by the patient or family, of full code status. Does not include presumed full code status or if obtained from other sources (i.e., review of records, according to team). | Included in original codebook for patients with advanced cancer: dnr, dnrdni, dni, do not resuscitate, do-not-resuscitate, do not intubate, do-not-intubate, chest compressions, no defibrillation, no endotracheal intubation, no mechanical intubation, shocks, cmo, comfort measures, life sustaining treatments, full code confirmed, would like to be full code, full code per patient, MOLST, DNR/DNI/LLST/Comfort Measures, dnr/dni |
| Added for patients with life-threatening trauma: code reversal, ok to intubate, no compressions, full code d/w, full code discussed, full code verified, would like to be full code, wishes to be full code, would like to remain full code, wishes to remain full code, wish to be full code, remaining full code | |
| GOC discussions: conversations with patients or family members about the patient's goals, values, or priorities for treatment and outcomes. Includes statements that conversation occurred as well as listing specific goals. | Included in original codebook for patients with advanced cancer: GOC, goals for care, goals of treatment, goals for treatment, treatment goals, family discussion, family discussions, patient goals, patient's goals |
| Added for patients with life-threatening trauma: ongoing discussions, ongoing discussion, wish, wished, met w/family, met with family, family meeting, discussion with family, per family, d/w family, family mtg, #GOCa |
Process measures and definition adapted from Lilley et al.4
Hashtags were frequently used to designate different sections of clinician notes.
After the addition of the key phrases described above, we again compared NLP identification with manual review of a separate randomly selected group of 40 patients (Table 3). In our reassessment, the sensitivity of NLP-identified code-status clarification rose to 98%, whereas the sensitivity of GOC discussions rose to 100%. The specificity remained 100% for both process measures. Using the revised codebook, we found that among all patients admitted with severe trauma, 26.3% had clarification of code status and 18% had a GOC discussion during their hospitalization.
Table 3.
Sensitivity and Specificity of Revised Natural Language Processing Codebook Customized For Trauma
| Admission for life-threatening trauma |
||
|---|---|---|
| Code-status clarification | GOC discussion | |
| Sensitivity, % | 98 | 100 |
| Specificity, % | 100 | 100 |
Discussion
In this brief report, we describe differences in the communication of code-status and GOC discussions in patients with advanced pancreatic cancer and patients with life-threatening trauma. Compared with patients with advanced cancer, we found that patients with severe traumatic injuries were involved in more dynamic discussions. Additionally, GOC discussions in patients with acute traumatic injuries tended to contain language-related surrogate decision makers. We employed a novel application of NLP to explore these differences in communication across thousands of in-patient admissions.
A major finding was that language varies depending on the underlying disease process and the clinicians treating the patient. These subtle differences are likely related to differences in the expected health trajectories and acuity within these two groups of patients, and the expectations and experiences of the clinicians treating them. Patients with advanced pancreatic cancer have a 2.9% five-year survival and the majority die within one year.7
In contrast, patients with life-threatening trauma have wide variation in long-term survival related to their trauma, as well as underlying conditions, such as advanced age and frailty.8,9 Correspondingly, the advanced training of trauma surgeons, surgical oncologists, and medical oncologists differs. Surgical and medical oncologists are also more likely to share a longitudinal relationship with a patient, whereas trauma surgeons rarely have this opportunity.10 Both differences in training and patient/clinician relationship may contribute to differences in communication of palliative care process measures.
Additionally, advanced pancreatic cancer represents a progressive disease. Although devastating, knowledge of terminal illness provides patients and their families with the opportunity to plan for end of life. Traumatic injuries are by their nature acute and unexpected. Due to critical illness, patients are often unable to participate in health care decisions and most rely on family and surrogate decision makers for decisions about treatment prefernces.11,12 Surrogates may be less prepared to make major decisions for the patient, which is reflected in “ongoing” discussions with clinicians.
As with all medical care, treatment should be personalized to the individual patient. Recognizing the differences in the language associated with palliative care in patients with different forms of serious illness is important in accomplishing this goal. Language can have a profound influence on thought and perception.13,14 Understanding how communication differs in patients with cancer and those with a traumatic injury can help tailor serious illness communication to the specific needs of the patient and help assess the quality of communication.
Limitations of this study include its setting of two urban tertiary hospitals within a single health care system, which may affect the generalizability. In addition, identification of code-status clarification and GOC discussions remain dependent on the accuracy of the NLP codebook. GOC discussion can be easily conflated with simple discussions of treatment goals and this may falsely inflate the number of patients identified in this study. As NLP becomes more widespread, care must be taken to ensure that the codebook remains consistent with accepted definitions and evolves to keep pace with evidence-based practice in serious illness communication. Lastly, while we focused on one aspect of palliative care delivery in two different disease processes, palliative care is a broad discipline that goes well beyond code-status clarification and GOC discussions.
Conclusions
The language used to confirm code status and document GOC discussion differs between patients with advanced cancer and those admitted with severe trauma. Recognizing and identifying the root of these differences may uncover patterns in palliative care delivery and help personalize treatment to the needs of the individual patient.
Supplementary Material
Funding Information
B.V.U. was supported by the Society of University Surgeons-KARL STORZ Resident Research Award (2017–2018). Z.C. is supported by the Paul B. Beeson Emerging Leaders Career Development Award in Aging (1K76AG054859-01), the American Federation for Aging Research, and the Cambia Health Foundation. C.L. is supported by the Cambia Health Foundation Sojourns Scholar Leadership Program.
Author Disclosure Statement
No competing financial interests exist.
Supplementary Material
References
- 1. Kelley AS, Bollens-Lund E: Identifying the population with serious illness: The “Denominator” challenge. J Palliat Med 2018;21:S7–S16 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Hua M, Li G, Clancy C, et al. : Validation of the V66. 7 code for palliative care consultation in a single academic medical center. J Palliat Med 2017;20:372–377 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Belanger E, Loomer L, Teno JM, et al. : Early utilization patterns of the new medicare procedure codes for advance care planning. JAMA Intern Med 2019;179:829–830 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Lilley EJ, Lindvall C, Lillemoe KD, et al. : Measuring processes of care in palliative surgery: A novel approach using natural language processing. Ann Surg 2018;267:823–825 [DOI] [PubMed] [Google Scholar]
- 5. American College of Surgeons Trauma Quality Improvement Program: ACS TQIP Palliative Care Best Practice Guidelines: https://www.facs.org/~/media/files/quality%20programs/trauma/tqip/palliative_care.ashx 2018. (Last accessed June1, 2018)
- 6. Greenspan L, Mclellan BA, Greig H: Abbreviated injury scale and injury severity score: A scoring chart. J Trauma 1985;25:60–64 [DOI] [PubMed] [Google Scholar]
- 7. Surveillance Epidemiology End Results Stat Fact Sheets: Pancreas: http://seer.cancer.gov/statfacts/html/pancreas.html 2018. (Last accessed October19, 2018)
- 8. Ritt M, Ritt JI, Sieber CC, et al. : Comparing the predictive accuracy of frailty, comorbidity, and disability for mortality: A 1-year follow-up in patients hospitalized in geriatric wards. Clin Interv Aging 2017;12:293. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Joseph B, Pandit V, Zangbar B, et al. : Superiority of frailty over age in predicting outcomes among geriatric trauma patients: A prospective analysis. JAMA Surg 2014;149:766–772 [DOI] [PubMed] [Google Scholar]
- 10. Udelsman BV, Lee KC, Traeger LN, et al. : Clinician-to-clinician communication of patient goals of care within a surgical intensive care unit. J Surg Res 2019;240:80–88 [DOI] [PubMed] [Google Scholar]
- 11. Sullivan DR, Liu X, Corwin DS, et al. : Learned helplessness among families and surrogate decision-makers of patients admitted to medical, surgical, and trauma ICUs. Chest 2012;142:1440–1446 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Long B, Clark L, Cook P: Surrogate decision making for patients with severe traumatic brain injury. J Trauma Nurs 2011;18:204–212 [DOI] [PubMed] [Google Scholar]
- 13. Imai M, Kanero J, Masuda T: The relation between language, culture, and thought. Curr Opin Psychol 2016;8:70–77 [DOI] [PubMed] [Google Scholar]
- 14. Gleitman L, and Papafragou A: Relations between language and thought. In: Resiberg D. (ed.), The Oxford Handbook of Cognitive Psychology. Oxford, Oxford University Press, 2014, pp. 504–523 [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.