Extracting Complementary and Integrative Health Approaches in Electronic Health Records

Abstract

Complementary and Integrative Health (CIH) has gained increasing popularity in the past decades. While the evidence bases to support them are growing, there is still a gap in understanding their effects and potential adverse events using real-world data. The overall goal of this study is to represent information pertinent to both psychological and physical CIH approaches (specifically, using examples of music therapy, chiropractic, and aquatic exercise in this study) in an electronic health record (EHR) system. We also aim to evaluate the ability of existing natural language processing (NLP) systems to identify CIH approaches. A total of 300 notes were randomly selected and manually annotated. Annotations were made for status, symptom, and frequency of each approach. This set of annotations was used as a gold standard to evaluate the performance of NLP systems used in this study (specifically BioMedICUS, MetaMap, and cTAKES) for extracting CIH concepts. Venn diagram was used to investigate the consistency of medical records searching by Current Procedural Terminology (CPT) codes and CIH approaches keywords in SQL. Since CPT codes usually do not have specific mentions of CIH approaches, the Venn diagram had less overlap with those found in clinical notes for all three CIH therapies. The three NLP systems achieved 0.41 in average lenient match F1-score in all three CIH approaches, respectively. BioMedICUS achieved the best performance in aquatic exercise with an F1-score of 0.66. This study contributes to the overall representation of CIH in clinical note and lays a foundation for using EHR for clinical research for CIH approaches.

Introduction

Complementary and Integrative Health (CIH) approaches are defined as “a group of diverse medical and health care systems, practices, and products that are not presently considered to be part of conventional medicine” by the National Center for Complementary and Integrative Health (NCCIH) in the National Institutes of Health [1]. “Complementary” is considered as being used together with conventional medicine. “Integrative” is not only the coordinative combination of conventional medicine and complementary approaches, but also the combination of different innervations such as “conventional medicine, lifestyle changes, physical rehabilitation, psychotherapy, and complementary health approaches in various combinations, with an emphasis on treating the whole person” [1–3]. CIH includes multiple approaches, such as herbal, dietary supplements, mindfulness and spiritual practices, yoga, and manual therapies.

CIH has received a great deal of attention and gained increasing popularity among specific patient groups in the past decades. National Health Interview Surveys in the USA estimated that all-CIH used in adults was 38% in 2007 and 35.5% in 2012 [4, 5]. According to a systematic review, over the past 30 years especially from 2000, there was an increasing number of people with cancer using CIH. About 40% of cancer patients in Australia, Canada, Europe, New Zealand, and the USA reported currently using CIH [6].

Most recently, NCCIH reclassified CIH as nutritional, psychological, and physical in their newly announced strategic plan, where psychological and physical approaches include mind-focused practices, body-based approaches, meditative exercise, art and music therapies, or integrative approaches combining several components [3].

Psychological and physical approaches play an important role in optimal health, which was widely used for “the management of pain and other troublesome symptoms” [3]. Neha Ghildayal et al. reported that over 40% of the low back pain (LBP) population in the USA used CIH approaches in the past year, among which herbal therapies, chiropractic manipulation, and massage had the highest use [7]. Given the high use of psychological and physical approaches, multiple research projects have shown evidence of the effectiveness of psychological and physical CIH approaches. Richard L. et al. reviewed the randomized controlled trial (RCT) data regarding the safety and effect of CIH, demonstrating that some CIH approaches may help patients manage their pain conditions. Specifically, massage therapy may help for the management of neck pain with adequate doses [8]. Published in Sept 2021, a recent review of the RCT studies of spinal manipulative therapy (SMT) regarding chronic LBP concluded that there is sufficient evidence indicating SMT is a good option to relieve chronic LBP [9]. Another review paper also found that 4 out of 5 RCTs indicated the effectiveness of music therapy toward both stress biomarkers and psychological stress [10]. In a summary study of 30 RCTs, Hilton et al. found evidence supporting an association between mindfulness meditation and reduced pain level [11]. However, according to NCCIH, unclear mechanisms of psychological and physical CIH approaches led to difficulty in determining the target populations and suitable specific conditions. The complexity of some psychological and physical CIH approaches also adds challenges to understanding individual effects [3]. Thus, more high-quality and rigorous studies are still needed to investigate the effectiveness of psychological and physical CIH approaches.

For this study, we mined data from the electronic health record (EHR) as an alternative resource to support our investigation regarding the real-world usage and safety of psychological and physical CIH approaches in patients. EHR contains structured information about patients’ medical history, diagnoses, medications, treatment plans, etc. However, unstructured clinical notes in the EHR contain useful information not easily found in structured data. Many studies have used both structured and unstructured data from the EHR to provide opportunity to enhance patient care and facilitate clinical research across multiple domains [12–14].

While some psychological and physical CIH approaches such as acupuncture have been recognized and have their unique Current Procedural Terminology (CPT) code to record in the EHR, a great proportion of psychological and physical CIH approaches remain vague in CPT code documentation. One therapy can be recorded by several CPT codes, and one CPT code can also represent several CIH approaches [15]. This increases the difficulty of tracking the CIH approaches by CPT codes. The use of clinical notes in the EHR however has rich documentation of CIH approaches for history and procedures. Natural Language Processing (NLP) techniques have shown promising results in extracting related information from unstructured clinical notes, especially in novel domains.

Our aim for this study is to (1) develop a data model to represent information regarding both psychological and physical CIH approaches (specifically music therapy, chiropractic, and aquatic exercise covered in this study) documented in unstructured clinical notes in the EHR and (2) evaluate existing NLP systems’s ability to annotate the CIH approaches. To the best of our knowledge, this is the first study to investigate the representation of CIH approaches in clinical notes. We manually annotated the entities (i.e., status, symptom, and frequency) describing these CIH approaches. We were also interested in trying to discover which symptoms or diseases the CIH approaches were used. Using the manual annotations as the gold standard, we evaluated the performance of NLP annotation systems (specifically BioMedICUS, MetaMap, and cTAKES) for extracting CIH information from clinical notes.

Methods

This pilot study mainly focused on three CIH approaches: music therapy, chiropractic, and aquatic exercise covering psychological and physical CIH. For each CIH approach, we followed the same steps: (a) randomly select clinical notes containing key terms; (b) build annotation guidelines; (c) manually annotate entities for each note based on the annotation guidelines; (d) collect CPT codes, extract medical records by querying CPT codes in SQL, and compare medical records querying by CPT codes versus approaches keywords (etc. “music therapy”); (e) extract information using automated NLP systems and evaluate their performance against the gold standard.

Data Source and Collecting Clinical Notes

We collected data from the University of Minnesota Academic Health Center’s Information Exchange (AHC-IE) clinical data repository (CDR). Data were from an Alzheimer’s disease cohort from 2010 to 2021. We extracted structured EHR data, including CPT codes and patient ID, and unstructured EHR data from clinical notes.

We used a word embedding tool (i.e., word2vec) [16] trained with clinical notes from the CDR to expand the terms related to the CIH approaches. We used a similar approach to expand lexical variants (including misspellings) in our corpus [17, 18]. Additionally, we conducted a thorough review of notes in our cohort to identify the specific expressions of CIH approaches. After identifying these terms, we retrieved clinical notes containing these expanded terms. The terms can be found in Table 1. Totals of 1446, 9505, and 2158 notes containing entities expanded by “music therapy,” “chiropractic,” and “aquatic exercise,” respectively, were identified. Among these notes, we randomly selected 300 notes (100 notes per CIH approach). Since the approaches-related sentences from the clinical notes of the same patient are more likely to be the same for each approach, we only selected one note per patient to avoid information overlap.

Table 1.

Example of expanding terms

CIH approaches	Terms expand type	Example of search terms
Music therapy	Abbreviation and other names	“music therapy”
		“music tx”
	Misspelling and alternative words of “music”	“musuic therapy”
Chiropractic	Misspelling and alternative words of “therapy”	“music approaches”
	Abbreviation and other names	“chiropractic,” “chiro”
	Misspelling and alternative words of “Chiropractic”	“chiropracty”
Aquatic exercise	Abbreviation and other names	“aquatic exercise”
		“aquatic HEP”
	Misspelling and alternative words of “aquatic”	“aquatics exercie”
		“pool exercie”
	Misspelling and alternative words of “exercise”	“aquatic exerise”
		“aquatic therapy”

Annotation Guidelines Development

The names and definitions of CIH approaches were collected from Natural Medicines [19]. Three approaches were selected to evaluate after searching from our cohort. Guidelines for statement-level annotations were developed through previous work reviewing existing standards [20] and notes (Table 2). For each approach, we annotated three entities: (1) status: status includes three labels: “past,” “current,” and “none”; (2) frequency; (3) symptom (see Tables 3, 4, and 5). This symptom could be several symptoms or diseases for an approach procedure. For example, music therapy may be used for depression and anxiety at the same time, which could both be labeled in the symptom entity.

Table 2.

Annotation guideline

Entity	Rule
Status
Past	The patient receives the CIH approaches in the past time
Current	The patient was receiving the CIH approaches
None	The patient declined the CIH approaches or did not receive the CIH approaches before
Frequency	To indicate how often the patient receives the CIH approaches
Symptom	To indicate the symptoms of diseases that the CIH approaches were used for

Table 3.

Annotation example of music therapy

Entity	Example
Status
Past	Pt did have OP, PT, music therapy in the past
Current	Appeared sad from facial expression today in music therapy
None	Pt did not attend music therapy group
Frequency	Music therapist will follow 1–2x/week
Symptom	Pt has intermittent pain breakthrough overnight, relieved with repositioning and music therapy on TV

Table 4.

Annotation example of chiropractic

Entity	Example
Status
Past	He has been to multiple pain clinics in the past including MAPS and TC pain and has had injections, PT, and chiropractic care
Current	Continue with physical therapy, OT and chiropractor and massage
None	Discharged due to patient declining chiropractic approaches
Frequency	She has been seeing a chiropractor 1–2 times a week for back and L foot pain
Symptom	Here for a trial of chiropractic adjustments for her NP and dizziness

Table 5.

Annotation example of aquatic exercise

Entity	Example
Status
Past	She has had success with aquatic therapy in the past
Current	At this time patient is independent with an aquatic exercise program and no longer requires skilled PT intervention to continue
None	She declines aquatic therapy referral
Frequency	Patient enjoys water exercise and reports that she has been able to attend a local pool to continue aquatic exercises 1x/week
Symptom	Pt is a 71-year-old female referred to physical therapy for aquatic exercise program for chronic bilateral knee and back pain, low activity tolerance

Manual Annotation of Clinical Notes

In all 300 notes, sentences containing entities about music therapy, chiropractic, and aquatic exercise were selected. In these sentences, we annotated entities indicating CIH approaches status, symptom, and frequency using the Inception annotation tool [21]. For example, in “Pt declined music tx services at this time,” “declined” and “music tx” were annotated, which indicated “current” in status and music therapy. Two annotators (ZH, ZN) with medical background annotated 10% of the notes. The inter-rater agreement was evaluated using Cohen’s Kappa. The remaining were independently annotated by each annotator.

Structure Data Collection

Our cohort data is maintained in the SQL database of the AHC-IE CDR. We can use CPT codes recorded in structured EHR data and keywords written in clinical notes to search medical records in SQL to see if the three CIH approaches have medical records. Despite the fact that CPT codes and keywords are stored in different SQL columns, they can be linked to the same patient ID and compared to see if they can identify the same approach records. The CPT codes for each approach were collected through a literature review [15] (Table 6). The approaches records containing unique patient ID were collected separately by searching CPT codes and keywords. For example, searching CPT codes 97530 yielded music therapy code patient ID list, while searching “music therapy” keywords yielded music therapy note patient ID list. These two lists were compared using a Venn diagram to see if medical records searching by CPT codes and approach keywords were consistent. The Venn diagram between CPT code and keywords would overlap if the same medical data, which we believed to be documenting the same parent patient ID, could be traced by both CPT code and keywords.

Table 6.

CPT codes of CIH

CIH category	CIH approach	CPT codes and summary of activity
Psychological and physical	Music therapy	97530 therapeutic activities direct contact
	Aquatic exercise	97113 aquatic therapeutic exercises
Physical	Chiropractic	98941–98943 chiropractic manipulation
		97140 manual therapy techniques
		97112 neuromuscular reeducation
		97810–97814 acupuncture
		97530 therapeutic activities
		99201–99215 generic office visit
		73510 diagnostic imaging

Evaluating Performance of Automatic Extraction Tools

To further evaluate the results of our annotation guidelines, we used the set of manually annotations as a gold standard. This gold standard was used to analyze the performance of the clinical NLP extraction tools employed in this study for annotating the three CIH approaches on the set of 300 notes. Specifically, we used the NLP-ADAPT-kube platform [22] to generate Concept Unique Identifiers (CUIs) from the Unified Medical Language System (UMLS) related to the three CIH approaches and entities mentioned above. Specific NLP annotation systems used were BioMedICUS, MetaMap, and cTAKES [23–25]. CUIs unambiguously map identified entities in clinical notes to UMLS Metathesaurus concepts. Only entities extracted by NLP extraction tools with specific CUIs of music therapy, chiropractic, and aquatic exercise were evaluated. We used two evaluation metrics proposed by the n2c2 2018 NLP challenge [26] to evaluate whether BioMedICUS, MetaMap, and cTAKES extracted entities that were manually annotated by our annotators. Strict matching refers to a true positive (TP) when a predicted entity’s span, which represents the slice of a document, exactly matches the annotation span from the gold standard. While lenient matching counts a TP if a predicted entity’s span overlaps with the gold standard. Based on these criteria for TP, measures of precision, recall, and F1-score were reported.

Results

Annotation Statistics

For the 300 notes, 115 sentences pertaining to music therapy, 128 sentences pertaining to chiropractic, and 136 sentences pertaining to aquatic exercise were identified (Table 7). The mention-level annotation yielded Cohen’s Kappa, $\kappa$=0.94 in approaches recognition, $\kappa$=0.83 in status, $\kappa$=1.00 in symptom, and $\kappa$=1.00 in frequency. For all three entities of CIH, status was documented the most. There was a total of 106 sentences documented for status in sentences of music therapy, 75 sentences in chiropractic, and 76 sentences in aquatic exercise. The most annotated label in status was “current.” Symptom and frequency mentions were documented relatively less frequently than status. Among all symptom labels, “anxiety” was most documented in music therapy, while in chiropractic and aquatic exercise, “chronic pain” was most documented.

Table 7.

Entities, values of CIH, and their prevalence in the corpus

Entity	Value	CIH
		Music therapy (n = 115)	Chiropractic (n = 128)	Aquatic exercise (n = 136)
Status	Total1	92.2% (106)	58.6% (75)	55.9% (76)
	Current	85.2% (98)	43.7% (56)	42.64% (58)
	Past	2.6% (3)	12.5% (16)	6.6% (9)
	None	4.3% (5)	2.3% (3)	6.6% (9)
Symptom	Total1	16.5% (19)	16.4% (21)	4.4% (5)
	Chronic pain	0.0% (0)	14.8% (19)	3.7% (5)
	Anxiety	13.0% (15)	0.0% (0)	0.0% (0)
	Dizziness	0.0% (0)	0.8% (1)	0.0% (0)
	Headache	0.0% (0)	0.8% (1)	0.0% (0)
	Sleep disorder	0.9% (1)	0.0% (0)	0.0% (0)
	Other	2.6% (3)	0.0% (0)	0.7% (1)
Frequency	Total1	0.9% (1)	2.3% (3)	8.8% (12)
	1/week	0.0% (0)	1.6% (2)	2.2% (3)
	2–3/week	0.9% (1)	0.8% (1)	5.9% (8)
	4–5/week	0.0% (0)	0.0% (0)	0.7% (1)

¹Total is the sum of all values within each entity and their proportion

Compare Information Extracted Using CPT Codes vs from Clinical Notes

The Venn diagrams between approaches records searching from CPT codes and clinical notes are shown in Fig. 1. The overlap in each Venn diagram indicates the records both identified by CPT codes and keywords. In music therapy, only 215 of the 769 total records recognized by keywords “music therapy” were also identified by CPT code 97530 Therapeutic Activities. In 295 total records identified by “aquatic exercise” and 292 records identified by CPT code aquatic exercise, only 69 records overlapped. In medical records identified by all chiropractic codes, 318 of the records can be identified by note “chiropractic,” while 574 records identified by “chiropractic” cannot be identified by codes.

Fig. 1.

Venn diagram of music therapy, aquatic exercise, and chiropractic

Performance of Extracting Tools

The performance of BioMedICUS, MetaMap, and cTAKES is shown in Fig. 2. The upper three bar charts were the performance for each tool in extracting entities representing music therapy, the center three bar charts were the performance in extracting entities representing chiropractic, and the bottom three bar charts were the performance in extracting entities representing aquatic exercise. Although performance evaluation by lenient matching is better than strict matching, the overall performance in these three tools was relatively poor with some of the entities not being extracted. Specifically, the average lenient matching F1-score of BioMedICUS (0.59) and cTAKES (0.36) were higher than MetaMap (0.28). In the three therapies, all tools extracting music therapy performed better than chiropractic and aquatic exercise. cTAKES cannot extract any information represented aquatic exercise. BioMedICUS outperformed the other two systems, achieving a lenient F1-score of 0.62 in music therapy, 0.48 in chiropractic, and 0.66 in aquatic exercise.

Fig. 2.

Performance of Biomedicus, cTAKES, and MateMap

Discussion

Psychological and physical CIH approaches had received increasing popularity and had indicated evident-based effects on specific symptoms. However, due to unclear mechanisms and complexity of psychological and physical CIH approaches, there is still a gap in understanding their effects. By leveraging notes curated for CIH-related concepts, we have demonstrated that our study may help future investigations of filling the gap by extracting information about music therapy, chiropractic, and aquatic exercise from clinic notes. A total of 379 sentences were identified from 300 notes, 257 sentences containing mentions about the status of the CIH approaches, indicating the potential for extracting pertinent information about CIH from clinical notes.

In the clinical notes of our cohort, most sentences were given in the past tense. Since we noticed that the even notes written on the same date of recorded date would also be documented in the past tense in the notes from our cohort, we considered all instances of past tense without any indication of a specific time as being “current,” which contributed to the prevalence of “current” status.

The use of CPT codes to track Complementary and Integrative Health (CIH) in structured data has some limitations. One of the main drawbacks is that many CPT codes are too general and not specific to certain CIH approaches. This can result in inaccurate identification of medical records that relate to CIH. For example, a single CPT code such as 97530 therapeutic activities can represent various CIH approaches such as music therapy, chiropractic, and other therapeutic approaches. As a result, using CPT codes alone to identify CIH approaches can lead to less accurate results and limited insight. Additionally, the Venn diagram matching between medical records querying by CPT code and keywords showed that medical records identified by keywords had less overlap with CPT codes. This suggests that relying solely on CPT codes to identify CIH-related medical records may result in missing relevant information, and a more comprehensive approach is needed. Therefore, using clinical notes with mentions is a more accurate way to find specific CIH approaches compared to relatively vague documentation pertaining to CPT codes regarding CIH. By accurately documenting CIH usage in EHRs, it can assist healthcare providers in comprehending the types of CIH approaches used by patients and their potential benefits and risks, thereby improving treatment decisions and patient outcomes.

Our study also compared our gold standard with NLP extracting tools by using the annotation as the gold standard. Not surprisingly, lenient matching produced better results than strict matching. For “music therapy,” the three extraction tools achieved 0.66, 0.66, and 0.63 in average lenient matching precision, recall, and F1-score. For “chiropractic,” the three extraction tools achieved 0.46, 0.43, and 0.41 in average lenient matching precision, recall, and F1-score. Specially, cTAKES could not identify words represented for “aquatic exercise” and their specific CUIs. Lower recall mainly resulted from the abbreviation of approaches that the tools cannot identify. For example, “music tx” stands for “music therapy” and “chiro” stands for chiropractic. Extraction tools also identified words that were not in the gold standard such as “therapy” and “D CHI,” accounting for part of the lower precision. The performance of BioMedICUS, cTAKES, and MetaMap showed that these tools can only extract part of the exact words standing for music therapy, chiropractic, and aquatic exercise, indicating the significance of extracting CIH information from clinical notes training with a well-established annotation guide.

This study is not without limitations. The investigation was limited to a small number of CIH approaches, and the corpus for each approach was relatively small due to the manual annotation process’s time-consuming nature. Future studies should aim to broaden the scope by selecting more notes from different note types and cohorts, exploring additional CIH therapies from different categories, and building NLP models to identify and extract specific CIH modalities mentioned in clinical notes. This will enable more comprehensive and accurate documentation of CIH usage in patients. Second, we considered all the sentences in the past tense without any words that indicated past time as “current” in status, which may result in bias in our annotation. For example, we annotated the status of “Participated in Music Therapy group by listening to self-chosen music.” as “current” though this sentence was written in the past tense. Third, our sentence-level annotation missed some information that was not in the same sentence containing words about approaches.

Conclusion

In summary, this study focused on identifying specific CIH approaches, such as music therapy, chiropractic, and aquatic exercise, in clinical notes. We developed an annotation guide to efficiently extract CIH information from unstructured clinical notes, which can facilitate more comprehensive and accurate documentation of CIH usage in patients. The study’s findings contribute to the overall representation of CIH approaches found in clinical notes and can inform future research on the impact of CIH on patient outcomes.

Author Contribution

Huixue Zhou collected the data, created the annotation guideline, and wrote the main manuscript text. Huixue Zhou and Zhongran Niu annotated the collected clinical notes. Greg Silverman evaluated the performance of existing Natural Language Processing systems. All authors reviewed the manuscript.

Data Availability

Not applicable.

Code Availability

Not applicable.

Declarations

Ethics Approval

Not applicable.

Conflict of Interest

The authors declare no competing interests.