Comparison of Methods To Identify Advance Care Planning in Patients with Severe Chronic Obstructive Pulmonary Disease Exacerbation

Abstract

Background: Advance care planning (ACP) is recommended for patients with chronic obstructive pulmonary disease (COPD). Yet, ACP documentation is often inaccessible at the time of impending respiratory failure, which may lead to unwanted and costly medical intensive care unit admissions. Electronic medical records (EMRs) contain directive fields and the ability to search for keywords and phrases, but these strategies to rapidly identify ACP have not been validated.

Objectives: The aim of this study is to identify the percentage of patients with severe COPD exacerbation who have outpatient ACP documentation and validate two EMR-based methods of rapidly identifying ACP documentation.

Design: Retrospective cohort analysis.

Setting/Subjects: Patients who required medical intensive care unit admission for exacerbation of COPD at an urban safety-net hospital between 2009 and 2014 were observed.

Measurements: We analyzed the sensitivity and specificity of two methods to rapidly identify outpatient ACP documentation: (1) documentation in the EMR directive field and (2) text string search of notes for key phrases, compared with a gold standard clinician review.

Results: Our cohort (n = 311) was racially diverse and severely ill with obstructive lung disease. One hundred thirty-two patients (43%) had ACP documentation by gold standard chart review. Compared with a gold standard chart review, a parsimonious text string search was both sensitive (95%) and specific (97%), while the directive box was specific (100%), but not sensitive (54%), for identifying outpatient ACP documentation.

Conclusions: EMR directive fields may substantially underestimate ACP when used alone. As full clinician chart reviews are impractical in the emergent setting, text string searches may be a useful strategy to rapidly identify ACP discussions for clinical care and research.

Introduction

Chronic Obstructive Pulmonary Disease (COPD) is the third leading cause of death in the United States,¹ and many patients requiring medical intensive care for COPD exacerbations die² despite invasive and costly critical care.³ Previous studies suggest that many patients with chronic disease, including COPD, do not want aggressive therapies at the end of life and prefer to die at home.⁴ Advance care planning (ACP), completed in the outpatient setting, increases the likelihood that patients receive hospital care that coincides with their previously stated wishes⁵ and is recommended for patients with end-stage pulmonary disease.⁶ Despite this, several studies based on patient self-report estimate that only 11%–19% of patients with severe COPD had outpatient ACP discussions.^7–9

The process of documenting ACP must begin early, but patient preferences for end-of-life care fluctuate over time,¹⁰ necessitating accurate documentation and retrieval of each discussion at the time of emergent clinical care. Although emergency room clinicians rely heavily on documentation regarding advance directives, as patients often cannot communicate their own wishes at the time of acute exacerbation, they report that this documentation is often elusive and difficult to retrieve rapidly in the clinical setting.¹¹ Morrison et al. reported that only 26% of patients who completed advanced directives had them recognized at the time of inpatient hospitalization.¹²

The presence of electronic medical records (EMRs) has facilitated the creation of dedicated fields to easily document and retrieve outpatient ACP information; however, low utilization of these fields may compromise their reliability. Another increasingly available function in EMRs allows text string searching of progress notes and orders for key words and phrases that are suggestive of ACP. While clinicians often rely on these shortcuts, their accuracy for identifying documented ACP discussions has not yet been assessed.

We undertook a retrospective chart review of patients who required intensive care for an acute exacerbation of COPD to determine both the frequency of documented outpatient ACP and the accuracy of two strategies to rapidly identify documented ACP discussions in the EMR.

Materials and Methods

Study population

We performed a retrospective cohort analysis of patients who were admitted to an urban, safety-net hospital MICU (Medical Intensive Care Unit) between May 2009 and May 2014 for a primary diagnosis of acute exacerbation of COPD (ICD9 codes 490, 491, 492, 493.22, and 496)¹³ or a primary diagnosis of acute respiratory failure (ICD9 codes 518.81, 518.84, 518.86, and 799.1) and secondary diagnosis of acute exacerbation of COPD. Patients who did not have an outpatient visit documented within one year of their MICU admission (first admission, if more than one) were excluded from analysis.

Primary outcome: documented outpatient ACP discussions

We used two methods to rapidly identify potential ACP documentation: (1) For each patient, we evaluated the documentation in the free-text, clinician-edited advance directive field located on the patient's home screen of the EMR. (2) We performed a text string search of the outpatient progress notes of the same outpatient record for several key words, phrases, or character strings associated with ACP discussions. The full-text string search, which was developed through discussion with 8 pulmonary and critical care physicians, included the following search terms: DNR (Do Not Resuscitate), Full Code, Goals of..,, Life Support, Comfort Measures, CPR (Cardiopulmonary Resuscitation), Resuscitation, CMO (Comfort Measures Only), DNI (Do Not Intubate), Intubat*, and aggressive (Table 1). To determine the true presence of ACP documentation, a pulmonary and critical care physician (A.R.S.) performed a full chart review on all patients in the cohort to identify documented ACP. This entailed reviewing all outpatient notes identified by the full-text string search (up to 250 notes per patient) in 304 patients and a complete outpatient chart review for seven patients not identified by the full-text string search. This chart review served as the gold standard for retrieving ACP documentation.

Table 1.

Analysis of Text String Search Algorithms

Search terms	Total patients identified	False positivea	Positive predictive value (%)a	Examples of documentation identified as false positive	Unique patients added with search termb
Parsimonious search algorithm
DNR (do not resuscitate)	72	1	99	“no tendnress”	71
Full code	80	2	98	“FULL CODE, presumed”, copied from inpatient notes	43
Goals of…	55	3	95	“goals of treatment with pain therapy”	11
Additional (excluded) terms
Life support	25	2	92	“support group”	0
Comfort measures	10	2	80	“agree with transition toward comfort measures,” “died on comfort measures”	0
CPR (cardiopulmonary resuscitation)	30	6	80	“mild CPR after walking,” “shock necessitating CPR,” “cprn nebs”	3
Resuscitation	4	1	75	“resuscitation with IV fluids,” “had cardiopulmonary resuscitation”	0
CMO (comfort measures only)	14	9	36	“completely dark room” “smoking cessation” “uncomfortable,” “ehcmo”	0
DNI (Do Not Intubate)	189	123	35	“midnight,” “prednisone,” “dnies”	1
Intubata (Intubated, intubating)	246	198	20	“intubated for respiratory failure,” “asthma with 3 previous intubations”	0
Aggressive	165	151	8	“aggressive chest PT,” “aggressive diuresis”	0

Full and parsimonious text string search algorithms with the number of patients identified by the search to have advance care planning. The first three terms comprise the parsimonious search algorithm, which identified 97% of patients with advance care planning documentation on clinician review.

^aThe false positives were identified and positive predictive value calculated by comparing these patients with those identified on gold standard chart review.

^bThis column shows the number of new patients who would be added by the term when searched in order of positive predictive value.

ACP documentation, for the purposes of this study, was defined broadly as any documentation from an outpatient visit containing code status terminology (Full Code, DNR/DNI, or CMO) or regarding preferences for end-of-life care. As the study also focused on the presence or absence of documentation, and not on the specific decision made, documentation regarding a discussion of goals of care without a final decision as to advance directives was included. We discarded documentation of discussions that occurred outside of an outpatient clinical encounter (e.g., “Notified by hospice that patient died at home on comfort measures” or a reference to a previous inpatient code status), a plan to discuss in the future (e.g., “will discuss code status at next visit”), or assignment of a healthcare proxy alone without discussion of preferences for end-of-life care. Signed advance directives were not analyzed as they are not available in the clinical EMR and thus would not have been retrievable in the immediate clinical setting.

Analyses

We characterized documentation of ACP discussions identified by each method as true positives or false positives based on the presence or absence, respectively, of ACP on gold standard chart review. Similarly, we classified each patient without a documented ACP discussion identified by our two strategies as a true negative or false negative based on comparison with chart review.

With the goal of identifying the most parsimonious text string search algorithm to identify true ACP documentation, we identified the individual text strings with the highest positive predictive value and iteratively refined the text string search algorithm, each time adding the text string with the next highest positive predictive value.

We calculated sensitivity, specificity, positive predictive value, and negative predictive value for the directive field, full-text string search, and parsimonious text string search to identify the most useful method for identifying outpatient ACP.

The Boston University Medical Campus institutional review board approved this study protocol (H34353).

Results

A total of 479 individual patients were admitted to the MICU for acute exacerbation of COPD during the study period. One hundred sixty-eight were excluded as they did not have outpatient visits at our institution in the years preceding or following the admission (Fig. 1). The remaining 311 patients, who were included for analysis, were racially diverse and severely ill with obstructive lung disease with a median FEV1 of 1.1 L (Table 2). In this cohort, 132 patients (43%) had at least one documented outpatient ACP discussion by gold standard clinician review (Table 1 and Fig. 2).

FIG. 1.

Patient flow chart detailing inclusions and exclusions. Advance care planning documentation was analyzed with full clinician chart review as the gold standard (middle) as well as with the advance directive field (left) and the parsimonious text string search (right). Results for each method, TN, TP, FN, and TP, sensitivity, and specificity compared with the gold standard are shown. TN, true negatives; TP, true positives; FN, false negatives; TP, true positives.

Table 2.

Patient Characteristics

Demographics
Mean age (years)	63.7 (SD ±12.5 years)
Male sex	172 (55%)
Racial distribution
Caucasian	131 (42%)
African American	133 (43%)
Hispanic	14 (5%)
Other (Asian, Native American, Middle Eastern, Other, NA/Unavailable)	33 (10%)
Measures of severity
Study period mortality rate (n = 286)	106 (37%)
Median FEV1 (n = 172)	1.1 L (SD ±0.6 L)
Percent requiring mechanical ventilation	90 (29%)
Percent requiring noninvasive ventilation only	121 (39%)

Demographics and Severity Indicators for the included patients (n = 311, unless otherwise noted).

FIG. 2.

The pie chart shows the percentage of included patients with advance care planning identified by gold standard clinician review. Text string searches and advance directive field, taken together, identified all patients with advance care planning documentation on gold standard review. Although there was considerable overlap (purple), the advance directive field alone (red) would not identify many patients with documentation that was identified using the parsimonious text string algorithm alone (blue).

Analysis of the directive field detected 71 (54% of those identified on gold standard) patients with documented ACP, who were all confirmed on gold standard chart review. Thus, relying on the directive field alone would miss 61 (46%) patients with documented ACP discussions identified on gold standard chart review (Fig. 2). Although the directive field was highly specific (100%) for identifying ACP documentation compared with the gold standard chart review, it was not very sensitive (54%, Fig. 1).

The full-text string search returned 304 individual patients. When compared with the gold standard clinician review, 175 charts identified by the search did not have documentation of an ACP discussion (false positives), leaving 129 with ACP documentation (98% of those identified by gold standard) as true positives on gold standard chart review (Fig. 1 for full sensitivity and specificity data).

The low specificity of the full-text string search is due to the inclusion of text strings that appear frequently in outpatient charts, but were often used in situations outside of an ACP discussion, such as “intubat*,” “DNI,” and “aggressive” (Table 1). The parsimonious search algorithm, which included only three text strings (DNR, Full Code, and Goals of…), was both sensitive and specific for ACP (95% and 97%, respectively, Fig. 1).

Discussion

ACP is known to be important in patients with advanced COPD as patients often cannot recall or discuss their goals of care at the time of acute exacerbation. Our study is the first to elucidate the rate of outpatient ACP documentation in patients who require MICU admission at an urban safety-net hospital for exacerbation of COPD. It is also the first to critically evaluate methods to retrieve ACP documentation using the EMR and identify a simple text string search algorithm that was more specific for identifying ACP documentation than the EMR directive field.

Our gold standard chart review found that 43% of COPD patients requiring MICU admission over five years had an outpatient discussion of ACP. Previous studies interviewing outpatients with severe COPD reported ACP in 9%–14% of their outpatient clinic visits.^7–9 There are several possible explanations for the higher degree of ACP noted in our study. First, patients included in our study are likely to have more severe COPD than those included in previous studies. Admission to the MICU may be a trigger for outpatient providers to document ACP, and patients admitted to the MICU are more likely to require rehabilitation stay where ACP is often discussed and documented in the record. Second, patient interviews are subject to a patient's recall of the conversation, which may underestimate the number of ACP discussions, especially in cases where there is no signed ACP document or limitation on end-of-life care. Third, secular trends regarding ACP may have led to increased discussion and documentation of ACP over time. For example, a study assessing ACP documentation in older patients (including those with and without COPD) from 1998 to 1999 reported an overall rate of documentation of 15%,¹⁴ while a study looking at documentation in a similar population from 2008 to 2011 reported an overall rate of documentation of 60%.¹⁵ Finally, the true rate of documentation of ACP will vary based on what documentation qualifies as ACP for the individual study. While some studies include only signed, legal advanced directives or healthcare proxy forms, others will consider a code status order or simply documentation of a discussion in a progress note or problem list as evidence of ACP documentation.

For existing ACP documentation to be useful, rapid identification of such documentation is imperative. Yet, studies have shown that clinicians in the acute setting often lack the documentation they need to appropriately manage patients.^11,12,16,17 Even if documentation is present, physicians and medical assistants were shown to have difficulty locating ACP documentation within two minutes.¹⁸ If clinicians spend several minutes searching for ACP documentation, the decision-making window for initiating emergency cardiopulmonary resuscitation or intubation has often passed.

While EMRs were expected to help bridge the gap, they offer many locations for documenting ACP and can make retrieval even more complicated. In a study evaluating geriatrics patients known to have ACP documentation based on review of scanned documents, progress notes, and problem lists, only 34% had this information present in a scanned document that was easily accessible to clinicians.¹⁵ EMR systems designed standardized fields and methods to document and identify ACP; however, their completeness has not been evaluated until now.

Our study found the EMR directive field to miss many patients with documented ACP discussions and thus was inferior to a simple text string search to identify ACP documentation in COPD patients requiring MICU admission. There are many possible reasons for this finding. The idea of a centrally located field to document ACP discussions is relatively new, and clinicians may not have been aware of the field or how to document discussions within it. In addition, providers may be unsure that their ACP discussion will reflect the patient's wishes in an acute setting and therefore hesitate to document it in a prominent centralized location. This may be especially true if the discussion does not include signing an official advance directive.

This study has limitations. As a single-center study, the exact text string search strategy may not be generalizable to all EMRs and clinical institutions. However, text string searching is available in two very popular EMRs (EPIC and Cerner¹⁸) and the phrases searched can be adjusted to suit the institutional culture. In addition, we cannot be certain that chart documentation alone always represents a full ACP discussion done on that clinic day. For example, while we used the phrase Full Code in our search algorithm as an indication of an ACP discussion, in some cases, this may represent default documentation that occurred without a true discussion or documentation carried over from a previous discussion. Finally, our study definition of ACP documentation did not require completion of a signed advance directive. While only signed advance directives are legally actionable in the setting of an acute clinical deterioration, it is often helpful in the clinical setting to be aware of a patient's previous opinions regarding aggressive care, regardless of whether he or she completed a legal advance directive. In addition, there are many hospitals where signed legal documentation of ACP is not available in the clinical chart. This study is the first among COPD patients at an urban safety-net hospital who previously required intensive care and thus represents an important contribution to the literature.

While a centralized location within the clinical chart to document and retrieve ACP discussions is an important step, more work is required to make this documentation both reliable and rapid to retrieve. Ongoing clinical studies are examining ACP documentation modules that increase the overall rate of documentation in the general population,¹⁹ but as yet there are no data regarding the ease of retrieval with these new modules. Our abbreviated text string search is a basic form of natural language processing, which has been previously used to rapidly identify ACP in medical student notes with a high positive predictive value, but only moderate sensitivity.²⁰ Our abbreviated text string search can form the basis for future natural language processing interventions with improved sensitivity for retrieving ACP documentation.

In addition, it would be helpful to know the reasons why directive fields are not used by clinicians. Clinicians may find the advance directive field too technically difficult to use or they may lack confidence in their ACP discussions. Clinicians may also worry that their patients' preferences for end-of-life care will fluctuate over the course of multiple hospital admissions.^21–23 Although previous studies have identified barriers to discussing ACP with COPD outpatients,²⁴ more work is required to identify barriers to documentation and retrieval of ACP.

Our study is the first to evaluate the rate of documentation of ACP in COPD patients requiring MICU admission at an urban safety-net hospital, which is a previously unstudied population. We have found the centralized directive field to miss more patients than a simple text string search when used to rapidly identify ACP documentation. We believe that our abbreviated text string search will facilitate retrieval of ACP documentation in clinical, research, and quality improvement settings and ultimately improve realization of patient-centered acute medical care.

Acknowledgments

The authors thank Linda Rosen, MSEE, of the Boston University Medical Campus' Clinical Data Warehouse for her assistance with acquisition of data.

This study was supported by NIH T32 HL007035.

Author Disclosure Statement

The views expressed in this article do not communicate an official position of the Boston University School of Medicine, the Department of Veterans Affairs, the US government, or the NIH.