A Framework for Information System Usage in Collaborative Care

Abstract

Clinical information systems (CIS) can affect the quality of patient care. In this paper we focus on CIS use in the collaborative treatment of chronic diseases. We have developed a framework to determine which CIS functions have general usefulness for improving patient outcomes.

Methods

We reviewed the use of clinical information systems within a collaborative care environment, identifying CIS functions important in chronic disease care. We grouped the functions into categories of Access, Best practices, and Communication (ABC). Three independent raters selected the most important collaborative care related functions from the HL7 Electronic Health Record Systems functional model, and mapped the HL7 functions against the ABC categories We then built a model of CIS use and tested it on data from a cohort of patients with chronic illnesses.

Results

Of the 133 HL7 elements in the ABC model, 60 (45%) were ranked as important for collaborative care by two reviewers. Agreement was moderate for importance (κ = .20) but high for ABC categorization (κ=.67). In our data tests, for the 1105 patients, access 4.4±6.5, best practices 0.8±1.6 and communication 2.9±4.5 CIS functions were used per episode of care. We were able to identify several key functions that may affect patient care. For example, certain CIS functions related to best practices were associated with higher clinician adherence to testing guidelines.

Discussion

This framework may be useful to assess and compare CIS systems for collaborative care. Future refinements of the model are discussed.

Introduction

Care for chronic diseases accounts for over half of health care expenditures in the United States. Beneficial treatment exists, but is only delivered approximately 50% of the time.¹ Models such as the Chronic Care Model (CCM) and continuous quality improvement can help create system-based changes that improve the likelihood appropriate care is delivered in a timely fashion.^{2, 3} Among other changes, the CCM recommends reorganizing health care delivery and adding clinical information systems and decision support to create a proactive practice team (including patients). ³ Some studies have shown improvement in cost and quality of care with this model. However, the particular functions important for clinical information systems in chronic disease care are unknown.^{4, 5} Descriptions of other models are perhaps intentionally vague about particular information systems components, using broad terms (e.g., decision support) rather than referencing specific CIS components. A review by Cretin et al. describing implementation and evaluation of the CCM, while useful in other areas, remains opaque about CIS use.⁶

In contrast, several frameworks for general information systems components exist. Health Level 7 (HL7), a standards development organization, has created an Electronic Health Record System (EHR-S) Functional Model which attempts to describe all pertinent functions in information systems. Other frameworks are based on outcomes. For instance, researchers have created information system feature sets, implementation practices, and internal policies and procedures which increase the likelihood of improved quality and safety in patient care.^7–9,¹⁰ In recent years, the Certification Commission for Healthcare Information Technology (CCHIT) has prioritized important functions for ambulatory EHRs (where most chronic disease care occurs).¹¹

However, standards or frameworks governing implementations of EHRs tend not to support the needs of team-based chronic illness care, focusing instead on individual physician needs (e.g., reimbursement).¹² Neither do these standards focus on collaborative care (where multiple roles are involved), instead relying on lists and functions undifferentiated by role.

This paper describes our attempt to identify and measure information systems components important for collaborative care. In our Care Management Plus project, at Intermountain Healthcare in Salt Lake City, we created systems of care management involving nurse care managers as part of primary care teams.¹³ To supplement the primary care team’s work, we created information systems components to improve collaborative care in a system with a fully functional EHR. Our analysis revealed three major needs for effective care management: monitoring and feedback about the plan of care; the ability to assure components of the care plan were completed; and communication and collaboration of all team members (including patients). Information system components recommended by the care teams focused on three major CIS functions complementary to these needs: facilitating information access, such as shared care plans, problem lists, and health status; supporting accepted standards of care, including standard reminders and care plan specific planning and follow-up lists such as tickler lists; and facilitating communication between team members, including e-messaging.¹³ We identified supporting themes describing clinical information systems functionality important for collaborative care. We sought to represent the themes as a concise and accessible framework.¹⁴ The result was the following framework for describing information systems functions that can aid collaborative care: providing information Access, supporting Best practices, and facilitating Communication (ABC). This ABC framework, while untested, was easy to communicate within our organization, and let others outside the organization know about our process.

The ABC framework was generated in a qualitative fashion, based on the developers’ expertise in collaborative care within the Chronic Care Model, information systems design, and requirements collection from care managers. While the framework was credible within the qualitative development process, it lacked the validation necessary to promote the model beyond our own use. The major risks to validity of the framework were that it was incomplete (one or more important components to supporting collaborative care were missing from the framework), or inaccurate (one or more of the components of the framework were not supportive of collaborative care). We therefore sought to validate both the completeness and accuracy of the framework by corroborating our findings with external models and with clinical data.¹⁵

Methods

Since care management and disease management programs currently suffer from a lack of standardized measurements, we concentrated on developing a broad functional model for EHRs. A set of functional measures was chosen for two reasons. First, functionality, while it may follow form, is not contingent on proprietary technology, intellectual property, or a particular infrastructure. Second, functions could be compared across different team members and settings within a single CIS intervention, focusing on the amount of CIS use for collaborative care of a patient.

To validate the completeness and usefulness of the ABC framework, we needed a comprehensive list of possible functions of information systems in collaborative care. The HL7 Electronic Health Record Technical Committee registered Draft Standard for Trial Use of the functional elements for EHR systems¹ represented such a comprehensive list.¹⁶ The Direct Care (DC) and Supportive (S) components of the HL7 EHR-S Functional Model consist of 133 functions arranged in a simple hierarchy around major concepts. In the DC and S components, there are 6 highest level (Care Management, Clinical Decision Support, and others), 24 second level, and 36 third level categories (referred to in this document as root categories). These draft functions have been used by the CCHIT to prioritize important functions for ambulatory EHR systems as part of its credentialing process. We used the HL7 EHR-S Functional Model to validate the completeness of our ABC framework. Our goal was to determine whether a) three of our primary developers and researchers would choose the same functions as key to the success of collaborative care for chronic illness, and b) whether such functions could be reliably grouped under our ABC framework.

To validate the accuracy of the ABC framework, we tested it against process and adherence and health outcomes in select chronic diseases. Using real patient care process and outcome data, we tested whether the use of selected model components did, in fact, affect the course of disease.¹⁷

Setting and Population

Intermountain Healthcare (IHC) is a large health system based in Salt Lake City, Utah, with a history of innovation in CIS development and use. In 1997, a special care management program was introduced in seven IHC primary care clinics. In these clinics, care managers and augmented information technology were added to the primary care team. The team was trained in enhanced chronic disease protocols and in educational and motivational techniques for patients. Patients were referred based on several chronic conditions, and individualized care plans based on protocols were created and enacted to improve the health status of the patient. For the validation component, patients who were referred with diabetes, hypertension and depression were identified through diagnoses from outpatient visits to the care management programs in 2003, and made at least one outpatient visit from January to June 2004. To establish the number of uses of any function per patient-visit, users of the CIS system (including a small number of patients) were limited to the seven care management outpatient clinics.

Study Design

Three raters were selected for their expertise in developing and evaluating local CIS components for collaborative care. Based on their experience in system development and research, these raters independently selected important concepts for collaborative care of chronic diseases from the HL7 EHR-S Functional Model, prioritizing them into important and very important categories.¹⁶ Researchers were trained to use important if they determined the concept had a place in the collaborative care system, and very important if the concept was crucial to the functioning of a system. If possible, they also categorized each function against the ABC framework. Criteria for selection included direct care and supportive elements as specified in the HL7 EHR-S Functional Model. The infrastructure components category was not included (as unrelated to collaborative care).¹⁷ A weight was calculated based on the overall agreement (percent of reviewers selecting) and importance (percent of reviewers who marked very important) of a concept; the most highly ranked concepts in each category were used to select concepts for measurement.

To test the effect of use of these components on patient care, we first related the concepts back to specific system functions and composed measures of those functions. First, researchers independently composed variable definitions that related to the most important concepts. Second, researchers identified computer system components within the IHC system that contained variables related to these concepts. If a system function did not exist and a measure could not be created, these functions were labeled untested and tracked. Finally, a metric was created for each category. Metrics were created by use of the information system component in question (or receipt of the product of the component, such as an alert). Information system use was defined as the number of uses of any function per patient-visit, adjusted for visit frequency.

Confounding elements for use, such as clinic visits, size of care team (from each clinic), and use of printed summary sheets were built into the queries to make the information use standard per patient per clinic visit variable, with the ability to separate individual provider information use as needed. Telephone or email access to communicate with patients was not considered as part of the overall information use statistics.

Analysis and Outcome variables

Agreement for selection of each component (Yes/No related to collaborative care) was measured by a weighted κ statistic.¹⁸ Internal consistency for categorization into the ABC grouping (with an extra class of ‘does not fit’) was measured using Cronbach’s α. Information system use per patient were calculated overall and by provider type (physician, nurse, other). Since high use overall was correlated with all components, an orthogonal set of measures of each component use was needed to create multivariate models. We performed a principle components analysis to create uncorrelated factors. These factors were then compared to appropriate testing of and changes in HbA1c, a measure of disease control for diabetes, and LDL levels, a risk factor for heart disease, using multivariate logistic and general linear models. Potential confounders, such as previous adherence (as measured at the patient level) to process guidelines for testing and follow-up, were included in the multivariate models.

Results

Function selection

As shown in Table 1, 102 concepts of 133 total (76.7%) from the EHR-S Functional Model were selected by at least one rater as important to collaborative care. All three raters agreed that 21 (15.8%) concepts were important and two (1.5%) concepts were very important, but there was only modest agreement between reviewers overall (κ=.21). Since some of the lowest levels of the functions were very similar, we grouped the root categories of the hierarchy (N=36) trended towards higher agreement (κ=.30; see table 1 for an example). Categorization by raters into Access, Best Practices, and Communication was highly consistent (Cronbach’s alpha=.65). However, 20 of the categories could not be melded into the ABC framework, indicating that it is an incomplete model.

Table 1.

Selection and categorization of HL7 EHR-S functional model components important for collaborative care

Selection	N (%)	Ave. weighted κ	p-value
Overall categories	133 (100.0%)
Selected	102 (76.7%)
… by 2+ raters	60 (45.1%)	0.21	0.02
… by 3 raters	21 (15.8%)
Very important	49 (36.8%)
… by 2+ raters	13 (9.8%)	0.162	0.23
… by 3 raters	2 (1.5%)
Root categories	36 (100.0%)
Selected	29 (80.6%)
… by 2+ raters	19 (52.8%)	0.302	0.14
… by 3 raters	10 (27.8%)
Very important	19 (52.8%)
… by 2+ raters	10 (27.8%)	0.237	0.31
… by 3 raters	2 (5.6%)
	Cochran’s α
Categorization*	60 of 133
Access	11 (18.3%)
Best Practices	20 (33.3%)	0.65	<.0001
Communication	9 (15.0%)
Unclear	20 (33.3%)

^*for categories chosen by 2+ raters.

Instancing of elements into specific information system components

The subset of categories selected for measurement based on highest weight is displayed in Figure 1; the entire set of categories selected is available from the authors. Functions important under access (11 in full set, 6 shown and 3 selected) included the management of summary lists (DC.1.1.3), the ability to summarize the health record, and capture patient-originated data. Key best practice functions (20 in full set, 6 shown and 3 selected) included the support of self-care, a fundamental principle of patient-centered care, care plans, and presentation of alerts. Context-sensitive components were also important since alerts may differ based on user and patient condition. Communication functions (9 in full set, 6 shown and 3 selected) included inter-provider communication, patient and family education, and the management of provider/patient relationships. The starred functions were then measured. For example, we measured every unique access of stored patient preferences. For best practices, alerts received on particular patients based on condition were chosen for context-based assessments. Provider communication between members of the clinical team was measured using notes generated from the chart-based electronic messaging system.

Figure 1.

Selected CIS functions potentially important for collaborative care from the HL7 EHR-S framework and metrics used to evaluate them

Function categories selected for measurement are in italics; DC = Direct care; CIS = Clinical Information Systems

Measurement and outcomes

Table 2 displays the measurements of information function use per patient per episode of care during the 6 month study period. For each clinic visit, 4.4±6.5 selected access functions were performed, .8±1.6 best practice alerts or suggestions were generated, and 2.9±4.5 communication functions were performed per patient per visit. Each specific component was used in variable amounts: self-care reminders were set very infrequently (.007 per patient per visit, or 1 per 142 patient-visits) while an average of 4.0 notes were generated per patient-visit. Role based information revealed certain functions were only performed by one role type (nurses), highlighting their importance in our collaborative care model. Where functions were accessed by individuals and the results printed, then reviewed by the team, individual roles could not be calculated.

Table 2.

Measurement of Clinical Information System function usage and relation to guidelines

		Usage per patient per visit
			Role
Concept	Metric(per patient)	Overall team	Physician	Nurse	Other
Access	Sum of below	4.4 (6.5)
DC.1.1.5 Summarize health record	Used worksheet	.2 (.2)	No role information available.*
DC.1.1.6 Manage clinical documents and notes	Entered note about care plan	4.0 (6.3)	2.6 (3.0)	1.2 (1.8)	.1 (.6)
DC.1.1.9 Patient preferences	Documented pref.	.2 (1.0)	None*	.2 (1.0)	None*
Best Practices	Sum	.8 (1.6)
DC.2.2.1.6 Support self-care	Set alert for reminders	.007 (.08)	None*	.007(.08)	None*
DC.2.1.2 Support for context-enabled assessments	Received contextual alert	.3 (.4)	No role information available.*
DC.2.2.1.2 Support for context-sensitive care plan	Reminded about care plan	.5 (1.5)	None*	.5 (1.5)	None*
Communication	Sum**	2.9 (4.5)
DC.2.2.1 Interprovider communication	Sent or received e-message	1.3 (1.6)	.9 (1.3)	.8 (1.3)	.8(1.4)
DC.3.2.4 Patient and family education	Education tracking	.2 (.8)	None*	.2 (.8)	None*
DC.3.2.3 Provider/patient communication	Contact tracking	1.4 (3.6)	None*	1.4 (3.6)	None*
Multivariate analysis		Overall	Adj. OR	p-value
Other variables significant in the models included: age, gender, comorbidity score, previous testing, and baseline scores.
HbA1c testing		c=.73	(95% CIs)
Best practices			1.4(1.1,1.8)	0.003
LDL testing		c=.65
DC.2.1.2 Contextual alerts			1.8(1.2,2.6)	0.001

^*CIS use for these functions was limited to a single role or shared use function and role could not be determined.

^**Roles do not sum because a single message is counted between two roles (sender/receiver) Grey shaded rows were highly correlated with other functions and dropped out of the final full analysis. Patient use of the medical record was not yet available in the EHR; their role is clearly important in collaborative care.

Multivariate analysis (shown on Table 2) revealed high correlation between function use (Pearson’s r = .21–.89); seven of the nine specific functions remained after principle factors analysis was used to remove correlation (drop-outs included patient preference documentation and provider patient communication which were highly correlated with care plan documentation and care team communication). All three of the general functions (access, best practices, and communication) remained in the model. After accounting for previous testing (p<.001), previous HbA1c levels (p<.001), team size (p=.452) and patient comorbidities(p<.001), CIS functions related to best practices increased the relative odds of appropriate diabetic testing by 36% (OR 1.36, 95% CI 1.1, 1.7). A specific function, contextual alerts, increased odds of LDL testing by 80% (OR 1.8 95% CI 1.2, 2.6). Higher use of best practice functions were associated with a non-significant decrease in HbA1c. No CIS functions were related to changes in LDL levels.

Discussion

This preliminary model gave us insight into the benefits and potential drawbacks of using information functions related to access, best practices, and communication. High use of the sub-elements of the model was associated with improvement in process-based measures. Patients with chronic disease who had a larger exposure to the ABC collaborative care information system were more likely to receive testing than those who did not, even after accounting for number of visits and other confounders.

However, larger numbers of clinical notes were associated with higher HbA1c levels, likely due to a co-occurring or worsening illness during the study period. Best practices functions were associated with a non-significant decrease in HbA1c levels. Several components ranked as important were not implemented and may be related to success with future implementations of systems to support collaborative care; these additional components are available at www.intermountainhealthcare.org/cmt/.

Our ultimate purpose in designing this framework was to reliably measure and evaluate CIS functions important for collaborative care. The ABC model categorized the majority of the concepts important for the topic with high reliability. All three raters agreed on a set of concepts, suggesting there is an underlying core set of functions for collaborative care that are not currently being exploited. We consider the preliminary results encouraging—patient care would appear to improve with information systems that allow role based information to be quickly and easily communicated to both providers and patients.

That said, there were important limitations to the study. First, the HL7 EHR-S Functional Model used was in draft form at the time of assessment, and may not have represented all the possible functions for an EHR system. Second, only moderate agreement was achieved in a small group of experts on the important elements for collaborative care use. Third, the ABC model is oversimplified. In our study, it did not identify all sub-functions important for collaborative care. Fourth, the outcomes measures presented here do not account for a number of confounders known to affect health states in patient populations.

Future research will be needed to validate and refine this model. A broader group of experts could build consensus on the CIS components needed to improve collaborative care. Alternatively, additional selection of concepts may better be driven by intervention than expert consensus. Some of the 133 concepts in the model may not be available; measurement should be driven by pragmatic concerns of making research reproducible through standard variable definition and accounting for different baseline amounts of information use. This paper simply represents an important first step in validating a framework for important CIS functions needed for collaborative care of chronic diseases in the outpatient setting.

Conclusions

We were able to identify key CIS components useful for collaborative care, and group them in a simplified model around the concepts of Access, Best Practices, and Communication. However, some components did not fit the simplified model, and there was not complete agreement about all functions. The model was useful in developing and testing metrics of information system use. These metrics then could be used to identify which information systems components were related to improved care of patients; best practices and especially contextual alerts were highly correlated. Different function use by role could also be calculated to help understand the needs of the collaborative care team.