Abstract
Objective
To develop and implement a customized toolkit within the electronic medical record (EMR) to standardize care of patients with brain tumors.
Patients and Methods
We built a customized structured clinical documentation support toolkit to capture standardized data at office visits. We detail the process by which this toolkit was conceptualized and developed. Toolkit development was a physician-led process to determine a work flow and necessary elements to support best practices as defined by the neuro-oncology clinical team.
Results
We have developed in our EMR system a customized work flow for clinical encounters with neuro-oncology patients. In addition to providing a road map for clinical care by our neuro-oncology team, the toolkit is designed to maximize discrete data capture. Several hundred fields of discrete data are captured through the toolkit in the context of our routine office visits. We describe the characteristics of patients seen at our clinic, the adoption of the toolkit, current initiatives supported by the toolkit, and future applications.
Conclusion
The EMR can be effectively structured to standardize office visits and improve discrete data capture. This toolkit can be leveraged to support quality improvement and practice-based research initiatives at the point of care in a neuro-oncology practice.
Abbreviations and Acronyms: AED, antiepileptic drug; CES-D, Center for Epidemiologic Studies Depression scale; EDW, Enterprise Data Warehouse; EMR, electronic medical record; GAD-7, General Anxiety Disorder 7-item scale; MDASI-BT, MD Anderson Symptom Inventory-Brain Tumor; NorthShore, NorthShore University HealthSystem; SCDS, structured documentation support toolkit
Brain tumors are commonly associated with significant symptoms that affect quality of life. The management of brain tumors is complicated due to the pathology of the tumors, associated symptoms, and prognosis. Brain tumors can be monitored with neuroimaging or treated with some combination of surgery, radiation, and chemotherapy, depending on the type of tumor.1 Response to treatment is assessed by the clinical status of the patient and neuroimaging.2 Most neuroimaging is through magnetic resonance imaging or computed tomography in patients who have contraindications to magnetic resonance imaging.
Medical management of brain tumors and their conditions improves survival and quality of life.3,4 Seizures are one of the most commonly managed conditions and can have a significant impact on quality of life.5, 6, 7, 8 Seizure outcomes can be measured by response to antiepileptic drugs (AEDs) in addition to interventions such as surgery, radiation, and chemotherapy. Other medical issues that can be assessed include mood disorders, fatigue, deep vein thrombosis/pulmonary embolism, and cerebral edema.9, 10, 11 The spectrum of potential symptoms and complications makes medical management complex.9,12 As such, standardizing care to address common issues and tracking patients to monitor outcomes is of interest.
We sought to develop a neuro-oncology “toolkit” to standardize care for neuro-oncology patients. The toolkit defines the important assessment and clinical information that our neuro-oncology practice prioritized as best practices. Furthermore, the toolkit relies on discrete data entry and thus provides an opportunity for systematically tracking outcomes over time and conducting quality improvement projects. The goal is to improve the quality of care for patients through systematic tracking of clinical outcomes and use of data for quality improvement practice-based research.
Patients and Methods
NorthShore University HealthSystem (NorthShore), located in the northern suburbs of Chicago, included 2 neuro-oncologists practicing at 3 outpatient centers at the time of the toolkit’s development. Our 7-stage process for quality improvement and practice-based research using the electronic medical record (EMR) has been previously described.13 We briefly describe the development of our highly customized brain tumor structured clinical documentation support (SCDS) toolkit that is used at each patient encounter.
The content of the toolkit was determined through frequent physician meetings that occurred every 2 weeks for 3 months. These meetings included neurology, the department chair, neuro-oncologists, and clinical support staff (such as clinical coordinators and nurses). The goal was to reach consensus on essential elements that conform to our practice’s definition of best practices in treating patients with brain tumor. In regard to important assessments, we reviewed the pertinent medical literature and National Comprehensive Cancer Network guidelines for tumors. We decided on standard information related to diagnostic history, treatments such as radiation and chemotherapy, current supportive medications, and so on, as shown in the Figure (screenshots). Each of these fields includes drop-down menus that expand when an affirmative response is entered. We also specified detailed pathologic information, including molecular subtypes for gliomas and metastatic tumors.
Figure.
Screenshots of the neuro-oncology structured clinical documentation support toolkit within the electronic medical record. A-B, MD Anderson Symptom Inventory-Brain Tumor (MDASI-BT). C-F, Short Test of Mental Status. G, Current medications. H, Prior diagnosis notes. HPI = history of present illness; MGMT = methyl guanine methyl transferase; ROS = review of symptoms; RX = medical prescription. © 2020 Epic Systems Corporation, used with permission.
Several score test measures were selected to address disability, as well as associated symptom severity. Because depression is common among patients with brain tumor and higher levels of depression are associated with poorer outcomes, we assessed depression and anxiety using the Center for Epidemiologic Studies Depression (CES-D) scale and General Anxiety Disorder 7-item (GAD-7) scale. Additionally, the CES-D and GAD-7 were selected for standardization with the other neurology toolkits. We chose the MD Anderson Symptom Inventory-Brain Tumor (MDASI-BT) because it is a standardized brain tumor assessment that has been validated and published in the literature.14, 15, 16 Also, we included a standardized assessment of performance status, the Karnofsky Performance Status. This was chosen over other measurement tools because it is the standardized scale used widely in neuro-oncology. The Short Test of Mental Status was selected because it is a standardized neurology test that is not copyrighted, making it freely available for use.
We developed an automated scoring system for radiographic response in patients with glioma based on the Response Assessment in Neuro-Oncology criteria. Patient outcomes were entered as discrete fields according to Response Assessment in Neuro-Oncology criteria and clinical criteria. We assessed the efficacy of AEDs according to patient reports of seizures. Additionally, we assessed the toxicity of treatments (radiation and chemotherapy) and supportive medications.
After deciding on the content, we conducted meetings with programmers from NorthShore’s EMR (Epic Systems) Optimization team every 2 weeks. These meetings additionally included clinical support staff, as well as a statistician and business intelligence analysts who were involved in data extraction and summary. These meetings occurred for 3 months. They built an the SCDS toolkit that included navigators (a sidebar index of processes to choose from), electronic forms (which had the ability to auto-score and auto-interpret), and summary flow sheets. We included free text fields to allow for additional information. We designed work flows (the order and assignment of tasks to a care team that includes a nurse and a neuro-oncologist) and mapped items to the progress notes (the order and layout in which the content would write).
During this time, we worked through the toolkit in a development format to ensure usability and consistency in definitions. Users were encouraged to work through the toolkit to identify potential barriers and bring them to the working group for resolution. The toolkit was designed to support the 2 main appointment types used at NorthShore, initial visits and follow-up visits (which occur annually or as interval visits, except for malignant brain tumors [every 6 months]). The standard appointment length for an initial visit is 60 minutes and for a follow-up visit is 30 minutes. As such, the toolkit was structured to conform to these times, with a shortened version of the toolkit used at follow-up visits.
After the SCDS toolkit implementation, we continued to meet every 2 weeks with programmers specialized in extracting, transforming, and loading data from the EMR’s data repository to data marts in NorthShore’s Enterprise Data Warehouse (EDW). The EDW programmers created enrollment reports for tracking patients and produced data quality reports indicating which required data were missing from office visits. These data quality reports are distributed to the care team monthly. Data not cleaned within 3 months were archived as permanently missing and were not listed on subsequent reports.
The care team learned where they were error prone from the data quality reports, and they remediated their use of the toolkits. When systematic errors occurred for many providers, the teams had the opportunity to improve their use of the toolkits or to request optimizations or a change in data requirements. The monthly reports produced only a few or no data checks per provider after the project was established. The weekly meetings continued for 3 months. The EDW team continues to support the toolkit use and updates/changes are made as needed when modifications (changes/additions/deletions) are deemed appropriate by the clinical team based on patient encounters. Additionally, ongoing support is provided for newly onboarded clinical staff to provide a learning environment for toolkit use and provide feedback on entry.
Results
We have successfully implemented a neuro-oncology–specific toolkit incorporating clinical assessment tools, including clinical presentation, pathologic and imaging results, symptom type/history, medical history, medication use, and functional assessment scores. A work flow was developed that assigns tasks and responsibility for specific assessments. Further, our toolkit writes progress notes and returns more than 800 fields of discrete data.
The process took 9 months from the time of initial physician meetings to implementation of the toolkit into a live clinical practice environment. Automated monthly reports are produced that summarize patient characteristics as they relate to demographic characteristics, clinical presentation, diagnostic/imaging information, symptoms, and score tests. These reports provide an opportunity to examine the characteristics of clinic patients and also address any issues with missing data or incomplete data entry. Repeat missingness or incompleteness suggests a need to reconsider the value of the data element and its relevance to best practices. Baseline demographic characteristics and data for score test measures are shown in Tables 1 and 2.
Table 1.
Demographic Characteristics of Patients With Brain Tumors
| Benign (n=207) | Malignant (n=252) | |
|---|---|---|
| Female sex, no. (%) | 150 (0.72) | 133 (0.53) |
| Age at initial visit (y), median (range) | 67 (24-97) | 63 (24-92) |
| Age at onset (y), median (range) | 57 (6-84) | 61 (18-92) |
| Race, no. (%)a | ||
| Caucasian | 139 (0.67) | 169 (0.67) |
| Hispanic/Latino | 4 (0.02) | 8 (0.03) |
| African American | 3 (0.01) | 5 (0.02) |
| Asian | 5 (0.02) | 4 (0.02) |
| Pacific Islander/Hawaiian Native | 0 (0.00) | 1 (0.004) |
| Education | ||
| Years, median (range) | 16 (6-25) | 16 (9-30) |
| Smoking, no. (%)a | ||
| Never | 103 (0.50) | 100 (0.40) |
| Former | 64 (0.31) | 116 (0.46) |
| Current | 2 (0.01) | 13 (0.05) |
Sum may not equal total patients due to missing values.
Table 2.
Score Test Measure Frequencies Among Patients With Brain Tumors
| Benign (n=207) | Malignant (n=252) | |
|---|---|---|
| General Anxiety Disorder-7, no. (%) | ||
| Normal | 127 (0.61) | 133 (0.53) |
| Mild anxiety | 33 (0.16) | 47 (0.19) |
| Moderate anxiety | 14 (0.07) | 32 (0.13) |
| Severe anxiety | 8 (0.04) | 15 (0.06) |
| Center for Epidemiologic Studies-Depression, no. (%) | ||
| No indication of depression | 134 (0.65) | 144(0.57) |
| Mild to moderate | 30 (0.14) | 38 (0.15) |
| Possibility of major depression | 16 (0.08) | 34 (0.13) |
| Karnofsky Performance Status interpretation,a no. (%) | ||
| Able to carry on normal activity and work, score of 80-100 | 169 (0.82) | 193(0.76) |
| Unable to work; able to care for most personal needs at home, score of 50-70 | 16 (0.08) | 41 (0.16) |
| Unable to care for self, score of 0-40 | 0 (0.0) | 3 (0.01) |
| MD Anderson Symptom Inventory-Brain Tumor, no. (%) | ||
| Mild | 93 (0.43) | 137 (0.54) |
| Moderate | 9 (0.04) | 24 (0.09) |
| Severe | 5 (0.02) | 15 (0.06) |
Scores are typically in standard intervals of 10.
Discussion
We have developed and implemented a customized EMR toolkit to evaluate patients with brain tumor in our neuro-oncology clinics. The development of the toolkit was a physician-led process with support from information technology partners. The toolkit is designed to formalize routine care and as such is not intended to extend the visit time compared to a standard visit.
In addition to standardizing care to conform to our definition of best practices, discrete and consistent data entry allows for longitudinal tracking of outcomes. This is of particular important for patients with brain tumors because managing patients’ symptoms and complications is complex. For example, seizures are common in patients with all types of brain tumors and are associated with significant morbidity.7 Therefore, we designed our toolkit to track seizures according to the efficacy of AEDs. Our toolkit has the potential for designing studies to compare the efficacy of commonly used AEDs. Rigorous assessment of longitudinal data supports quality improvement projects to determine the impact of interventions on patient outcomes. Given that the toolkit was developed by physicians for the purposes of supporting clinical care, our rates of missing data are low (as shown in Results). This reduces the potential for bias and improves the likelihood that findings are generalizable to our larger patient population.
Additionally, leveraging some of the standardized data elements, we have implemented Best Practice Advisories to identify opportunities for a provider intervention when indicated. For example, if a patient screens positive for depression or anxiety, a Best Practice Advisory fires and presents a mouse-click option to place an order, write a referral, or defer (which prompts selection of a reason for deferral). This is particularly important for patients with brain tumors because depression rates among those with brain tumors are among the highest of all cancers.17 It has been reported that approximately one-half of patients with brain tumors who screen positive for depression fulfill Diagnostic and Statistical Manual of Mental Disorders criteria for major depressive disorder.
Importantly, depression has been independently associated with decreased survival time in patients with glioma.18, 19 Additional symptoms captured through the MDASI-BT include fatigue, nausea, constipation, sleep disturbances, and cognitive changes. These symptoms are often caused by radiation and/or chemotherapy rather than the brain tumor itself. Identifying these symptoms is important because managing them can have a significant impact on a patient’s quality of life.
We are also using the toolkit to support larger research efforts. Concurrently, we are also conducting a DNA biobanking study under another Institutional Review Board–approved protocol. Blood samples will be used to generate data on more than 1 million single-nucleotide polymorphisms. This genetic information can be used to complement the clinical data and conduct novel studies of the influence of biomarkers on disease cause, progression, and treatment response. This practice-based research is enhanced through the collection of discretized data. Beyond our practice, our toolkit also supports data sharing with other neuro-oncology clinics. We plan to share our toolkit with clinics that use the same EMR (Epic), agree to implementation and deidentified data sharing for the purpose of standardizing care, benchmarking performance, and conducting multisite research collaborations across diverse patient populations.
There are some limitations to the approach we have described. Though the toolkits present standardized data fields, there are invariably differences between physicians as to how questions are asked and how equivocal responses are interpreted. For example, we rely on patient-reported seizure events, for which this may not be entirely accurate. Though there is space for comments in many fields, providers are generally limited to choosing from a pre-set response list, and this pre-set list is used for descriptive reporting and analysis. However, all physicians were involved in the development and agreed on the content and responses, and modifications can always be undertaken by the Health Information Technology team. Patients may also interpret the same question differently and we use subjective questionnaires. However, this is not an issue specific to our method and the questionnaires used in our toolkits have been validated previously.
Additionally, although the design of the toolkit was intended to be completed within our standard appointment length, there is a learning curve. Initially, there is an adjustment to provider flow and this may result in increased time to complete a patient encounter. However, over time, we have found that use of the toolkit does not increase the length of a routine office visit. By making edits to the toolkit of unnecessary or redundant data fields over time, we were able to make the toolkit much more user friendly than the original version.
Shortly after implementation of the toolkit, we discovered that some data fields originally conceived in the design stage of the project proved to be less useful in practice. This led to significant changes to the toolkit, in which some data fields were deleted from the toolkit. None of these fields were critical data fields for outcome measures. For instance, we removed the exact start date of treatment, replacing it with the starting year, because finding this information added extra time for the provider reviewing the chart. In our attempt to be exhaustive in our data collection, we removed fields we later discovered were redundant. For patients with metastatic tumors, initially we captured every treatment they had received. However, most chemotherapy drugs do not cross the blood-brain barrier. Therefore, we chose to focus on newer treatments that crossed that barrier into the brain. This finding is in accord with other SCDS toolkits at our institution. Additionally, after 1 year of data analysis, we discovered a tight correlation in the results of the CES-D/GAD-7 and MDASI-BT. This led to removal of the CES-D and GAD-7 assessments from the toolkit.
Patient experience has been helpful in additional modifications to the toolkit. One of the challenges we faced was patients filling out the CES-D, GAD-7, and MDASI-BT before the appointment. In accord with our correlative results for these outcome measures, we discovered patient dissatisfaction with redundancy of the forms. There were additional factors that led to incomplete filling out of forms, including patients not receiving the forms at the time of check in and patient fatigue. We have mitigated these factors by only using the MDASI-BT and creating an online version of this form that patients can fill out before the appointment.
Another patient experience that we have encountered is unwillingness to take the Short Test of Mental Status. Patients often refuse to have this test administered to them. This could be due to the self-consciousness at knowing that the brain tumor impairs cognition, anxiety related to taking a test, or fatigue. This has motivated us to look at alternative ways to assess cognitive functioning, such as patient-reported outcomes.
Conclusion
The SCDS toolkits and clinical documentation support features can be used to standardize office visits of patients with brain tumor. The toolkits can be built to enable physicians to provide care consistent with our definition of best practices, with the goal of optimizing patient care. This is particularly applicable to assess patients for measures of efficacy and toxicity of brain tumor treatments, such as radiation and chemotherapy. It is also applicable to supportive care treatments for seizures, fatigue, and venous thromboses. Additionally, the toolkits capture discrete data that can easily be translated into descriptive and analytic reports for the purpose of conducting quality improvement and practice-based research. We anticipate that these toolkits will improve the care of patients with brain tumors and provide future opportunities to engage in research to predict disease course, treatment response, and quality-of-life measures.
Acknowledgments
The authors thank Hannah Moulthrop Meng, MS, for assistance with editing, formatting, and submitting the manuscript for publication; the medical assistants, nurses, neurologists, electronic medical records Optimization and Enterprise Data Warehouse programmers, administrators, and research personnel at NorthShore University HealthSystem who contributed to the quality improvement and practice-based research initiative using the electronic medical record; and the neuro-oncology patients who inspire us to improve quality and innovate our clinical practice every day.
Footnotes
Grant Support: This work was supported by the Agency for Healthcare Research and Quality (R01HS024057) and the Auxiliary of NorthShore University HealthSystem.
Potential Competing Interests: The authors report no competing interests.
References
- 1.Owonikoko T.K., Arbiser J., Zelnak A. Current approaches to the treatment of metastatic brain tumours. Nat Rev Clin Oncol. 2014;11(4):203–222. doi: 10.1038/nrclinonc.2014.25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Villanueva-Meyer J.E., Mabray M.C., Cha S. Current clinical brain tumor imaging. Neurosurgery. 2017;81(3):397–415. doi: 10.1093/neuros/nyx103. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Avila E.K., Chamberlain M., Schiff D. Seizure control as a new metric in assessing efficacy of tumor treatment in low-grade glioma trials. Neuro Oncol. 2017;19(1):12–21. doi: 10.1093/neuonc/now190. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Zamanipoor Najafabadi A.H., Peeters M.C.M., Dirven L. Impaired health-related quality of life in meningioma patients-a systematic review. Neuro Oncol. 2017;19(7):897–907. doi: 10.1093/neuonc/now250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Politsky J.M. Brain tumor-related epilepsy: a current review of the etiologic basis and diagnostic and treatment approaches. Curr Neurol Neurosci Rep. 2017;17(9):70. doi: 10.1007/s11910-017-0777-3. [DOI] [PubMed] [Google Scholar]
- 6.Schiff D., Lee E.Q., Nayak L., Norden A.D., Reardon D.A., Wen P.Y. Medical management of brain tumors and the sequelae of treatment. Neuro Oncol. 2015;17(4):488–504. doi: 10.1093/neuonc/nou304. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Chen D.Y., Chen C.C., Crawford J.R., Wang S.G. Tumor-related epilepsy: epidemiology, pathogenesis and management. J Neurooncol. 2018;139(1):13–21. doi: 10.1007/s11060-018-2862-0. [DOI] [PubMed] [Google Scholar]
- 8.Englot D.J., Chang E.F., Vecht C.J. Epilepsy and brain tumors. Handb Clin Neurol. 2016;134:267–285. doi: 10.1016/B978-0-12-802997-8.00016-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Phillips K.A., Fadul C.E., Schiff D. Neurologic and medical management of brain tumors. Neurol Clin. 2018;36(3):449–466. doi: 10.1016/j.ncl.2018.04.004. [DOI] [PubMed] [Google Scholar]
- 10.Riedl J., Ay C. Venous thromboembolism in brain tumors: risk factors, molecular mechanisms, and clinical challenges. Semin Thromb Hemost. 2019;45(4):334–341. doi: 10.1055/s-0039-1688493. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Mohile N.A. Medical complications of brain tumors. Continuum (Minneap Minn) 2017;23(6, neuro-oncology):1635–1652. doi: 10.1212/CON.0000000000000540. [DOI] [PubMed] [Google Scholar]
- 12.Huang T., Mueller S., Rutkowski M.J. Multidisciplinary care of patients with brain tumors. Surg Oncol Clin N Am. 2013;22(2):161–178. doi: 10.1016/j.soc.2012.12.011. [DOI] [PubMed] [Google Scholar]
- 13.Maraganore D.M., Frigerio R., Kazmi N. Quality improvement and practice-based research in neurology using the electronic medical record. Neurol Clin Pract. 2016;5(5):74–78. doi: 10.1212/CPJ.0000000000000176. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Radloff L.S. The CES-D scale: a self-report depression scale for research in the general population. Appl Psychol Meas. 1977;1(3):385–401. [Google Scholar]
- 15.Spitzer R.L., Kroenke K., Williams J.B.W. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–1097. doi: 10.1001/archinte.166.10.1092. [DOI] [PubMed] [Google Scholar]
- 16.Armstrong T.S., Mendoza T., Gning I. Validation of the M.D. Anderson Symptom Inventory Brain Tumor Module (MDASI-BT) J Neurooncol. 2006;80(1):27–35. doi: 10.1007/s11060-006-9135-z. [DOI] [PubMed] [Google Scholar]
- 17.Krebber A.M., Buffart L.M., Kleijn G. Prevalence of depression in cancer patients: a meta-analysis of diagnostic interviews and self-report instruments. Psychooncology. 2014;23(2):121–130. doi: 10.1002/pon.3409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Noll K.R., Sullaway C.M., Wefel J.S. Depressive symptoms and executive function in relation to survival in patients with glioblastoma. J Neurooncol. 2019;142(1):183–191. doi: 10.1007/s11060-018-03081-z. [DOI] [PubMed] [Google Scholar]
- 19.Shi C., Lamba N., Zheng L.J. Depression and survival of glioma patients: a systematic review and meta-analysis. Clin Neurol Neurosurg. 2018;172:8–19. doi: 10.1016/j.clineuro.2018.06.016. [DOI] [PubMed] [Google Scholar]