Quantifying care coordination using natural language processing and domain-specific ontology

Abstract

Objective This research identifies specific care coordination activities used by Aging in Place (AIP) nurse care coordinators and home healthcare (HHC) nurses when coordinating care for older community-dwelling adults and suggests a method to quantify care coordination.

Methods A care coordination ontology was built based on activities extracted from 11 038 notes labeled with the Omaha Case management category. From the parsed narrative notes of every patient, we mapped the extracted activities to the ontology, from which we computed problem profiles and quantified care coordination for all patients.

Results We compared two groups of patients: AIP who received enhanced care coordination (n=217) and HHC who received traditional care (n=691) using 128 135 narratives notes. Patients were tracked from the time they were admitted to AIP or HHC until they were discharged. We found that patients in AIP received a higher dose of care coordination than HHC in most Omaha problems, with larger doses being given in AIP than in HHC in all four Omaha categories.

Conclusions ‘Communicate’ and ‘manage’ activities are widely used in care coordination. This confirmed the expert hypothesis that nurse care coordinators spent most of their time communicating about their patients and managing problems. Overall, nurses performed care coordination in both AIP and HHC, but the aggregated dose across Omaha problems and categories is larger in AIP.

Background

Coordination of healthcare services is vital for older adults who are vulnerable to the effects of illness, cognitive decline, disability, poverty, and limited social support.¹ The University of Missouri (MU)-Sinclair School of Nursing opened a home healthcare (HHC) agency to support an enhanced care coordination intervention, known as Aging in Place (AIP), designed to keep older adults living in their community homes for as long as possible.² After AIP ended, the HHC agency continued to provide traditional HHC services.³ Care coordination is a primary role of nurses in all settings, but it is particularly important in HHC services. Care coordination is an umbrella term that encompasses both case management and transitional care,⁴ and is defined as ‘the deliberate integration of care activities between two or more participants involved in a person's care to facilitate the appropriate delivery of health services’.⁵ Case management, while often used interchangeably with care coordination, is actually a more intense form of care coordination used to assist patients who have multiple, complex problems and are at significant risk of poor outcomes.⁶ Transitional care, yet another form of care coordination, is used to coordinate handoffs between healthcare settings and services.⁷ Over the last several decades, a number of care coordination models have been developed that focus on complex case management, transitions, and AIP.^2,8,9 Benefits of nurse care coordination include reductions in emergency room visits, increased patient survival after hospitalization, fewer readmissions, reduced costs, and increased transitional care safety.^3,10

Although the number of care coordination programs is growing and care coordination is generally viewed positively, there remain significant problems with care coordination measurement, including identifying the specific activities that constitute care coordination and quantifying how much care coordination (dose) was delivered to each patient. The current state of care coordination documentation and measurement is poor, relying on structured data fields that were not specifically designed to describe care coordination, or custom-built tools and surveys that capture the work of care coordinators in an abstract way. In addition to structured data, practicing nurse care coordinators routinely document detailed narrative notes that fully describe what they did for patients while rendering care coordination. Many of these narrative notes document complex activities, such as management, facilitation, and communication, that cannot be described in sufficient detail using structured input alone.⁴

It is time consuming and labor intensive to analyze narrative notes using traditional qualitative methods, and the number of notes that can be analyzed using such methods is small. The use of natural language processing (NLP) and domain-specific ontologies overcome this limitation, allowing mining of a large amount of unstructured narrative notes. Research in biomedical NLP has made enormous progress in developing techniques and tools that can analyze large corpora of unstructured text. One widely used tool, MetaMap, is based on the Unified Medical Language System (UMLS).¹¹ Another commonly used system for medical text mining is the Medical Language Extraction and Encoding System (MedLEE).¹² Finally, the clinical Text Analysis and Knowledge Extraction System (cTAKES) was built using Unstructured Information Management Architecture (UIMA)¹³ and is used for information extraction from electronic health record clinical text.¹⁴ A more comprehensive survey of systems for clinical text mining has been published.^15,16

While these state-of-the-art NLP systems are best suited for medical and biomedical text mining, they are less appropriate for mining care coordination notes because nurses describe social phenomena (eg, arranging transportation, setting appointments, obtaining food) using everyday language and not formal biomedical or clinical language. In the meantime, most NLP frameworks use machine learning algorithms trained on annotated biomedical and clinical corpora rather than nursing corpora.

The methodology described in this paper relied on AIP and HHC datasets that were rich in descriptions of care coordination. Therefore, it was important for the NLP tool to be guided by an ontology that reflects the language of practicing nurse care coordinators. There are nursing-specific ontologies that are integrated into UMLS such as the International Classification of Nursing Practice (ICNP). While the ICNP terminology contains many care coordination concepts, its objective is to establish common language describing nursing practice across different international settings¹⁷ and it is not completely integrated into existing ontologies.^18,19 In order to quantify care coordination, we needed an ontology that (1) contained care coordination-specific vocabulary, (2) organized concepts in a simple categorization specific to care coordination, and (3) described core practices of care coordination such as communication and management.

Care coordination is a critical component of nursing practice and organizations, such as the American Nursing Association, that are currently focused on assuring that care coordination is recognized and appropriately reimbursed. Currently, efforts are underway in the USA to develop and implement health information technology that includes documentation of care coordination delivery.^7,20 This paper reports on the development and use of a care coordination ontology, which was built expressly to extract data from an electronic healthcare record (EHR) that used the Omaha system. The title ‘Omaha’ comes from the location of the HHC agency that originally developed the structured, standardized language of problems, categories, and interventions for an EHR system. In this paper, we identify components needed to measure care coordination using the notes of practicing nurses. The following questions guided this research: (a) what specific activities did AIP nurse care coordinators and HHC nurses use when coordinating care for older community-dwelling adults and (b) how can those activities be quantified?

Methods

The process for capturing care coordination activities from narrative notes is summarized in figure 1.

Figure 1:

Activity extraction and dose-calculation process.

In this section we will describe in detail the concept extraction process, followed by the description of the patient and problem profiles, which are used to quantify the amount of care coordination received by each patient, hereafter referred to as dose.

Setting and sample

This study uses EHR data from two groups of patients. One group consists of 217 patients who were admitted to an HHC agency between 1999 and 2002 for enhanced AIP care coordination.^2,9 The other contains 691 patients who received traditional HHC without enhanced care coordination between 2003 and 2005. Nurses in both AIP and HHC documented patient interventions in an EHR that used the Omaha System. MU Institutional Review Board approval was obtained prior to the start of the study.

Dataset description

The Omaha System consists of three schemes. The problem classification scheme (table 1, column 1) is a structure of 42 terms and cues for a standardized assessment of the individuals, families, and communities.²¹ The intervention classification scheme is used by nurses to ‘describe health related care plans and services for individuals, families and communities’.²¹ At the top level of the intervention scheme is the ‘Omaha category’ (table 1, column 2), which contains four categories: (1) Teaching, guidance, and counseling, (2) Treatments and procedures, (3) Case management, and (4) Surveillance. The second level is the ‘Omaha intervention’ or ‘Target’ (table 1, column 3), which provides a more specific set of 75 interventions.²¹

Table 1:

A sample patient dataset

Omaha problem	Omaha category	Omaha intervention	Narrative note
Medication regimen	Case management	Medication set-up	The client said she would call (activity) in refills (focus) for coumadin tomorrow
			SN called (activity) in refill (focus) for thyroid rx
	Teaching, guidance, and counseling	Medication set-up	SN reviewed (activity) meds (focus) as SN filled (activity) the mediplanner (focus). Client was able to verbalize the name of the med and frequency
		Medication action/side effect	Reviewed (activity) the medications (focus) and effects, precautions to take, effects of amiodarone and Coumadin for irregular heartbeat. Instructed her re the need to consider a plan to continue (activity) the use of the med planner (focus). She initially verbalized a desire to learn to fill it herself, then decided to allow her son to fill it after she spoke with him
	Surveillance	Medication administration	Patient does not wish nurse to do medi-planner but allowed her to count pills. Able to state (activity) correct dose (focus) and frequency of meds when questioned. Had 19 pills of Synthroid, 70 pills of Lanoxin, 22 of 1/2 tabs of sotalol…
Communication with community resources	Case management	Social work/counseling	SN informed client of physician order for MSW to assist (activity) with community resources (focus) if needed
Healthcare supervision	Case management	Medical dental care	…will keep pcp informed (activity) of progress (focus) and any noted complications
			…continuing to assist (activity) client to understand (focus) differing opinions from her doctors
Circulation	Case management	Medical dental care	…received order (activity) to change the dose (focus) of meds and get repeat PT next Monday. Will arrange (activity) with her pcp to have PT [protime] (focus) drawn at the clinic next Monday at the appointment. Will change (activity) meds (focus) and instruct client this visit
			…will fax (activity) results (focus) when available

In every patient visit, nurses identified, assessed, and documented patients’ healthcare problems. The EHR had structured inputs, allowing nurses to select an Omaha category, interventions, and a problem. Nurses also used the comment box to document activities or findings that were not easily described in the defined structured language. For instance, in table 1 the nurse addressed the patient ‘Medication regimen’ problem using three main interventions (Case management, Teaching, guidance, and counseling, and Surveillance). Under Case management, the ‘Medication set-up’ intervention was used, while ‘Medication administration’ was used under Surveillance. The bold words in the narrative notes correspond to the care coordination activities and their corresponding foci used by nurses when coordinating the patient's care.

The data contain a total of 139 173 narrative notes for the two groups of patients divided into four Omaha categories (table 2). Of the four Omaha categories, Case management was the category most closely aligned with Care coordination, since both are viewed as approaches for managing complex care. Also, from an activity and task perspective, they are conceptually similar in orientation. Also, most of the professional literature identifies the following activities as case management: coordination, advocacy, and referrals used by clinicians to guide individuals, families, or communities in the appropriate use of resources and improve communication, all of which occurred in case management documentation.²² Therefore the Case management category, which represented 8% of the data, was used to build the ontology and construct the profiles. The remaining 92% of the data residing in the other three Omaha categories was used to test and validate the ontology and the overall approach.

Table 2:

Some characteristics of the dataset (AIP, n=217; HHC, n=691)

Category	Group	No of notes (%)	Average notes per patient (n)	Average problems per patient (n)	Average interventions per problem (n)
Build
Case management	AIP	6311 (4.53%)	29	3.17	3.83
	HHC	4727 (3.4%)	7	1.96	3.06
Testing & validation
Health teaching, guidance and counseling	AIP	7020 (5.04%)	32	3.97	5.05
	HHC	21 047 (15.12%)	30	3.35	5.22
Treatments and procedures	AIP	9156 (6.58%)	42	3.15	4.04
	HHC	17 593 (12.64%)	25	2.47	4.04
Surveillance	AIP	34 298 (24.64%)	158	8.46	4.08
	HHC	39 021 (28.04%)	56	5	3.95

AIP, Aging in Place; HHC, home healthcare.

Some characteristics of the data are summarized in table 2. The number of interventions is similar in both groups, ranging between 3 and 5 interventions per problem. In addition, nurses in AIP had more documentation because patients remained in the program longer and received continuous services from a care coordinator. The average AIP patient had 29 notes in Case management, while HHC had an average of 7 notes.

Care coordination ontology

The activities and foci shown in table 1 describe care coordination from the care coordinators’ perspective. As discussed above, it was necessary to build a care coordination ontology to guide the activity extraction. The 11 038 narratives under the Omaha category, Case management, were used in the construction of the domain-specific ontology of care coordination. The activity extraction for the ontology was validated using nearly 100 terms that had been abstracted by hand from 20 medical records from the care coordination narrative notes. All 100 terms were identified during the ontology construction process.

Candidate terms were organized using five top level concepts identified by the experts: (1) care coordination ‘activities’ contained action verbs used by nurses when coordinating care; (2) care coordination ‘foci’ represented the objects the activities acted upon; (3) ‘actors’ contained people who interacted with care coordinators; (4) ‘problems’ described specific patient problems identified by the care coordinator; and (5) ‘places’ included locations where patients’ resided when they received care. Candidate terms identified from Case management were associated with the appropriate class in the ontology using ‘is-a’ relationship. This process resulted in 394 concepts, of which, 66 were classified as care coordination activities, 156 as coordination focus, and the remaining concepts were distributed across the other three classes (table 3). The first-level concepts of care coordination activities and focus are shown in table 3. Protégé, an open source software, was used for editing and modeling the ontology.²³ Interested readers are encouraged to download the full ontology hosted on BioPortal (http://bioportal.bioontology.org/ontologies/NCCO).

Table 3:

Summary of the ontology

Class	Number of child nodes	Maximum depth
Activities	65	3
Administer	0
Assess	3
Assist	0
Attempt	0
Communicate	15
Identify	0
Instruct	0
Manage	33
Monitor	4
Obtain	0
Order	0
Foci	163	6
Ability	0
Access	0
Adherence	0
Appointment	0
Appropriateness	0
Care	76
Documentation	11
Follow-up	0
Information	2
Resource	12
Services	45
Supervision	0
Transportation	4
Understanding	0
Problems	90	6
Actors	53	3
Places	18	3

Mining nurses’ narrative notes

The ontology was used to mine nurses’ narrative notes for the care coordination activities. However, the activities alone were not informative, because they were out of context. For example, the activity ‘adjustment’ does not mean much by itself, but when combined with the foci of ‘medication’, it describes work carried out by care coordinators to ‘adjust medications’. For that reason, the problem profiles contain only those activities that co-occur with at least one care coordination focus from the ontology.

After sentence boundary disambiguation, tokenization, and stemming had been performed on the narrative notes using Natural Language Toolkit,²⁴ the text was searched using an activity–focus recognizer based on regular expression. The results of these searches were reviewed by clinical experts: three registered nurses and a licensed clinical social worker with extensive care coordination research and practice experience. The results made sense within the context of care coordination and were considered sufficient to build problem profiles and calculate the dose.

Initially, we identified a large number of potential activity–focus pairs, making the results hard to interpret. To overcome this problem, the ontology allowed us to control the specificity of information retrieval by using broader/narrower terms when searching the text.²⁵ Broader terms appear higher in the hierarchy and have wider or less specific meaning. We used this technique to collapse all activities found in the text under the first-level concepts in an activity set, denoted as , where and A defined as:

For example, to find ‘manage medication’ activity found in the second note listed in table 1, we used the regular expression ‘a\s+f’, where a∈ Manage={Arrange, Fill, Plan,…} and f∈ Medication={Antibiotics, Mediplanner, Prescription,…}. This means that ‘manage’ and ‘medication’ concepts are separated by one or more spaces (stop words are ignored, eg, ‘the’).

Problem profiles

Patient problems form the broader context of what care coordination activities were delivered. Problem profiles were developed to describe care coordination activities and foci used for specific Omaha problems in the Case management category. On the left side of figure 2, note that specific care coordination activities found in narrative notes were mapped to each Omaha problem resulting in problem profiles, which gave a deeper level of understanding of how nurses accomplished coordinated care for patients with specific Omaha problems.

Figure 2:

Activities provide a new level of detail describing how nurses coordinated care within each Omaha problem.

Every Omaha problem, i, is described by a profile, m_i, that consists of the first-level concepts from the activity set A. With each activity, , we associate a weight w_ik, which is the number of interventions in which the activity α_k was used to care for patients with problem i. For example, care coordinators may have used the activity ‘manage’ in three different interventions in problem i, meaning w_ik=3. Formally, we can write the profile m_i of problem i as a vector:

1

If activity α_k was not identified in any intervention in problem i, then we set w_ik=0.

Patient–problem profile

In order to compute the problem profile m_i, we first need to describe each patient by the activities used by nurses to coordinate care. The outcome of this process is the patient–problem profiles.

Each problem i and patient j can be described by a patient–problem profile P_ij, which is a vector

2

where w_ijk is the number of interventions where the activity α_k was used for patient j and problem i.

Example 1: Consider the sample patient data shown earlier in table 1. In Case management, the patient has two notes documented for the problem Medication regimen. The terms ‘call’ (communicate) and ‘refill’ (manage) were used in only one intervention in Case management, which is Medication set-up. In other words, the patient Medication regimen profile contains two concepts: communicate (w_ij5=1) and manage (w_ij8=1), resulting in patient–problem profile .

Medoid profile

The profile m_i of a given problem i over a group of N patients was computed by first identifying the profiles for a given patient j and problem i, as we demonstrated in example 1. The medoid profile is the patient profile most representative of problem i, meaning that the majority of patients have profiles for problem i somewhat similar to m_i. The medoid approach was favored over the mean to avoid partial activities in the profile definition (ie, ‘manage medication’ can either be present in some interventions or not, ie, 0, 1, 2,…, but not 0.5). To find the medoid profile, we computed the pair-wise Euclidean distance between each patient–problem profiles P_ij and P_ik. The patient, v, representing the problem profile was the one closest to all other N−1 patients, that is:

3

In other words, if patients with problem i are indexed from 1, …, N, then the variable v found in equation (3) is the index to the medoid patient–problem profile . That is, the problem profile m_i in equation (1) can be rewritten as m_i=P_iv or .

This process is performed for every Omaha problem, and eventually patient–problem profiles are compared with the problem profile m_i to determine the dose.

Care coordination dose

Care coordination is quantified by transforming the patient–problem profile into a value, which we refer to as the ‘dose’. A dose is the intensity of the activities used by the nurses to coordinate a patient's care, measured relative to the problem profile m_i.

To compute the dose, patients were followed from their date of admission to AIP or HHC for 360 days until their death, or until the end of the study (whichever condition occurred first). The narrative notes documented during that period for each patient were extracted and parsed, and finally used to compute patient–problem profiles as described in the section Problem profiles.

The dose D_ij is the sum of the patient–problem profile weights (w_ijk) normalized by the sum of the problem profile m_i weights (w_ik). When the dose for problem i was computed, only the activities that appeared in m_i with w_ik>0 were used in the calculation. Then, the dose of patient j in problem i is computed as a function of the problem profile m_i and the patient–problem profile P_ij as:

4

where the patient–problem profile weights w_ijk are given in equation (2) and the problem profile m_i weights (w_ik) are based on medoid patient–problem profile P_iv computed using equation (3).

Example 2: Given the profile for Medication regimen and the patient–problem profile from example 1, , the dose of Medication regimen for that particular patient is computed using equation (4)

5

Here, only the activities ‘communicate’ and ‘manage’ were used in the calculation (w_ik>0). A dose of D_ij=1 means the patient was fully coordinated, as specified in the problem profile (the same activities appeared in the patient and problem profile, ie, w_ik=w_ijk). More frequent activities in the patient–problem profile compared with the problem profile indicate a higher dose of care coordination (D_ij>1), while less frequent ones denote a lower dose (D_ij<1).

The aggregated dose of care coordination for a single patient is calculated by summing all individual problem doses (5) as:

4

Finally, we hypothesized that activities used in care coordination would also be used in other Omaha categories, since care coordination is a significant part of a registered nurses’ job. To test this, we used the problem profiles that were initially developed in the Case management category to calculate the aggregated dose for every patient within the remaining Omaha categories. The results of the aggregated dose are discussed in the next section.

Results

The problem profile m_i give us insight into the actions used by nurses to coordinate care. We identified the activities ‘communicate’ and ‘manage’ as the most widely used activities in care coordination, appearing in 23 (55%) and 29 (69%) out of 42 problem profiles, respectively, as shown in figure 3. On the other hand, the activities ‘administer’, ‘attempt,’ and ‘identify’ are not represented in any of the 42 profiles, since they either had a low prevalence among the patients or they did not co-occur with coordination foci.

Figure 3:

Number of problem profiles representing each activity.

Figure 4 displays the 20 most frequent activity–focus pairs in AIP and HHC. These activities are related to communication and management. Overall, more care coordination activities were documented in AIP than HHC. Sixty-seven per cent of activities that involved communication about durable medical equipment occurred in AIP, and about 92% of medication management took place in AIP. Only two of the 20 activities (‘manage care’ and ‘manage documentation’) occurred more often in HHC than in AIP. HHC is an extension of acute care, and the activity ‘manage care’ referred to the management of acute healthcare problems, while ‘management of documentation’ refers to locating and obtaining advance directives. This finding makes sense, since AIP patients had more chronic healthcare problems with established routines that did not require constant revision.

Figure 4:

Top 20 most frequent activities in the Case management category and the occurrence percentage in every group.

Figure 5 shows the number of unique activities documented in AIP and HHC for every Omaha problem in the Case management category. The Caretaking/parenting Omaha problem was the most diverse, where 103 unique activities were used by care coordinators in AIP, while only 16 were used in HHC. A greater number of unique activities documented in AIP than in HHC showed that care coordinators in AIP used more varied techniques to manage patients’ care. Moreover, there are some Omaha problems where HHC had no documented activities at all, such as Cognition, Family planning, and Pain. However, in four problems (Communication with community resources, Digestion-hydration, Personal care, and Skin), the HHC patients had more activities than AIP.

Figure 5:

Number of unique activities by Omaha problem in Case management category.

Table 4 reports the different doses of care coordination used for AIP and HHC patients as measured in the Case management category. This method detected differences between the AIP and HHC care coordination doses, with AIP having higher average doses of care coordination in all but three problems (Communication with community resources, Skin, and Healthcare supervision). Higher average dose for HHC patients was expected in Omaha problems such as Communication with community resources and Skin, where more activities occurred in the HHC than in the AIP (see figure 5).

Table 4:

Care coordination dose by problem and category in AIP and HHC groups

	Average dose		Median dose		Max dose
Omaha problem	AIP	HHC	AIP	HHC	AIP	HHC
Problem dose (dose by Omaha problem in Case management)
Environmental domain
Income	0.9	0.73	1	0.5	1	1
Psychosocial domain
Communication with community resources	0.64	0.67	1	0.5	1	1
Mental health	1	1	1	1	2	1
Caretaking/parenting	0.8	0.56	0.83	0.5	2	1
Physiological domain
Skin	0.71	1.13	1	1	1	2
Neuro-muscular skeletal	0.81	0.54	1	0.5	1.5	1
Circulation	0.88	0.4	1	0.5	1	0.5
Urinary function	0.83	0.75	1	0.75	1.5	1
Health-related behaviors domain
Nutrition	0.86	0.67	1	0.5	1	1
Personal care	0.95	0.91	1	1	2	2
Healthcare supervision	0.53	0.66	0.5	0.5	1	1.5
Medication regimen	0.86	0.71	1	0.5	1	1
Category dose (aggregated dose by Omaha category)
Health teaching, guidance, and counseling	1.39	0.99	1	1	6	4
Treatments and procedures	1.21	0.92	1	1	4	2
Case management	2.2	1.15	1.5	1	11.53	3
Surveillance	1.4	0.9	1	1	7	3
All category dose (p value<0.01)
Aggregated dose across all Omaha categories	3.54	1.74	2	1	24.53	9.5

AIP, Aging in Place; HHC, home healthcare.

Furthermore, we discovered that care coordination is not exclusive to the Case management category. We successfully used the problem profiles m_i to compute the dose in the other three categories. Results from a t test show that the aggregated dose of care coordination in all categories is significantly higher in AIP (µ=3.5; σ=4.1) than HHC (µ=1.7; σ=1.7), t(440)=6.5, p<0.01.

Discussion and conclusion

Care coordination relies heavily on communication with patients, family, and healthcare team members about healthcare management, all of which is usually detailed in narrative notes.

This paper presents a novel approach to the measurement of care coordination dose. We describe the development of an ontology to guide the extraction of care coordination activities. The ontology design took into consideration a simple categorization that describes care coordination core practices such as communication and management. The activities were then used to build patient–problem profiles. The profiles were not based on conceptual ideas about care coordination but used the words of practicing nurses as they documented their work with patients. These activities include communication (eg, calling and reporting) or management (eg, adjusting and scheduling) about patient needs (eg, transportation and medications). Organizations such as the American Nursing Association are calling for recognition and reimbursement of care coordination. It is critical that care coordination activities, derived from the work of practicing nurses, be fully documented, so they can be recognized as valuable to patient care outcomes and reimbursed appropriately.

Using 139 173 narrative notes for building and validating the problem profiles, we measured the care coordination dose in both AIP and HHC. We concluded that ‘communicate’ and ‘manage’ activities are widely used in care coordination, confirming the expert hypothesis that nurse care coordinators spent most of their time communicating about their patients and managing problems. Overall, nurses in both AIP and HHC preformed care coordination, but the aggregated dose across Omaha problems and categories was larger in AIP. The fact that this conclusion agrees with our intuition that more care coordination was delivered in AIP than in HHC provides a validation of our methodology and ontology.

We note that the care coordination ontology was constructed based on activities extracted from the Case management category for community-dwelling older adults. The activities are general enough to be applied across different populations, but the focus of the activities may vary. As has been noted by others, the broad concepts that guide care coordination can be used across populations⁶; nonetheless, the ontology would need to be validated for use in different populations, EHR systems, and across different clinical institutions. It is important to note that the EHR data spanned nearly 7 years; AIP began at the end of calendar year 1999 through 2002, and then in 2003 converted to a traditional HHC agency. There are nearly three times the number of traditional HHC as AIP patients. Given this disparity, we noted higher doses of care coordination in AIP, supporting our conclusion that the ontology is extracting terms significant to care coordination.

In this paper, we identified components needed to measure care coordination using the notes of practicing nurses. This is early work, which is intended to help guide future thought about how narrative notes may be used to quantify the work of nurse care coordinators. We recognize that more work is needed, including merging the care coordination ontology into existing ontologies²⁶ and annotation of more diverse EHR nursing unstructured text.

Contributors

LLP, MP, CG, MR, LH, FS: made substantial contributions to the design of the work, acquisition of funding, analysis, and interpretation; drafted and revised the work; gave final approval of the version; and agree to be accountable for all aspects of the work, including resolving questions regarding accuracy and integrity of the work. MAK, VL: made substantial contributions to the design of the work, analysis, and interpretation; drafted and revised the work; gave final approval of the version; and agree to be accountable for all aspects of the work, including resolving questions regarding accuracy and integrity of the work.

Funding

This work was supported by National Institute of Nursing Research grant number 1R15NR012940-01.

Competing interests

None.

Ethics approval

IRB Project #1188957.

Provenance and peer review

Not commissioned; externally peer reviewed.