The Knowledge Program: an Innovative, Comprehensive Electronic Data Capture System and Warehouse

Abstract

Data contained in the electronic health record (EHR) present a tremendous opportunity to improve quality-of-care and enhance research capabilities. However, the EHR is not structured to provide data for such purposes: most clinical information is entered as free text and content varies substantially between providers. Discrete information on patients’ functional status is typically not collected. Data extraction tools are often unavailable. We have developed the Knowledge Program (KP), a comprehensive initiative to improve the collection of discrete clinical information into the EHR and the retrievability of data for use in research, quality, and patient care. A distinct feature of the KP is the systematic collection of patient-reported outcomes, which is captured discretely, allowing more refined analyses of care outcomes. The KP capitalizes on features of the Epic EHR and utilizes an external IT infrastructure distinct from Epic for enhanced functionality. Here, we describe the development and implementation of the KP.

INTRODUCTION

Health information technology is becoming increasingly central to health care and will be a main strategy to improve quality of care, optimize patient management, enhance capabilities to perform research on a large scale and identify effective interventions in a real-world setting ¹. More effective use of information technology (IT) is a central recommendation of the 2001 Institute of Medicine report, Crossing the Quality Chasm, which cited a national healthcare IT infrastructure as essential to closing the large quality gap existing between the healthcare system that the United States has and the system that it could have¹. The United States Congress has passed legislation to encourage “meaningful use” of electronic health record (EHR) systems by providers and hospitals and the federal government will provide additional funding to hospitals that have implemented EHRs that comply with meaningful use criteria². Beginning in 2015, the Center for Medicare and Medicaid Services will reduce reimbursement to hospitals that do not have EHRs that satisfy meaningful use requirements³.

In addition, in the national debate over health care, many parties have begun to request evidence concerning treatment outcomes and the quality of care. In 2006, 47 states had mandatory or voluntary reporting systems for inpatient hospital data for both physicians and hospitals⁴. Moreover, interest is increasing in using comparative effectiveness research to help inform management decisions, and the 2009 American Recovery and Reinvestment Act included $1.1 billion for comparative effectiveness research⁵.

Due to the scope of the clinical dataset that it manages, the EHR presents an enormous opportunity to achieve these goals. However, the traditional EHR is designed for clinical care, and resembles a document management system. As such, it is not organized to provide data access for analyses of aggregated data. Most clinical information is entered as free text, which severely restricts one’s ability to later extract data. The specific clinical information entered into the EHR is not standardized, and content varies substantially between EHR providers. Tools for data extraction by providers and researchers are sparse and often unavailable. Further, relevant clinical data are often housed in multiple different systems, which may limit the utility of data queries from an EHR⁶.

Several approaches have been taken in attempts to overcome these limitations. Data warehouses typically provide a data retrieval mechanism. Such warehouses, which are becoming more common, improve the accessibility of clinical information^{6, 7} but are often limited to administrative data or clinical information that is captured discretely through the EHR. Other efforts to increase the usability of electronic health information have focused on structured documentation of clinical data, but typically have included only one disease process, a limited number of physicians, and only occasionally have collected measures of health status. One of the largest ongoing successful data capture initiatives is the National Spine Network, which provides a data collection system for participating physicians who treat patients with musculoskeletal symptoms and collects patient-reported information^8,9–11.

To solve these problems, the Cleveland Clinic Neurological Institute (NI) has developed the Knowledge Program (KP), a comprehensive initiative that uses a combination of strategies to improve the systematic collection of discrete clinical information into the EHR and the accessibility of data for use in research, quality measures, and patient care. A distinguishing feature of the KP is the incorporation of health status data collection, primarily supplied by the patient, as part of the normal clinical workflow. The Cleveland Clinic has been using the EpicCare EHR (Epic Systems; Verona, Wisconsin) since 2001. The KP capitalizes on the features of the Epic EHR, including its various clinical documentation options, and utilizes an external IT infrastructure that is distinct from Epic. Here, we describe the initial development and implementation of the KP.

DESIGN OBJECTIVES

The design objectives of the KP are based on the use of core EHR features with extension, when necessary, through the integration of custom external software. We chose this approach because it provides a comprehensive solution for systematically collecting clinically relevant patient outcome measures, along with an organizational infrastructure and a method for creating structured documentation within the EHR. Ultimately, the additional software layer allows for easy access to clinical information and serves as an accessible, scalable platform for collecting and analyzing clinical outcomes that is tightly integrated with the EHR.

Five specific design objectives characterize the KP:

• Systematic collection of patient health status measures (HSM) during the clinical encounter

  Collecting HSM provides clinically relevant outcomes that can be used for patient management, quality reporting, and clinical research. Our objective was to electronically collect patient-reported health status information prior to or at their clinic visits using an independent platform that is closely integrated with our EHR, with immediate availability within the EHR.

• Standardized documentation of key clinical data elements that are stored discretely, allowing electronic retrieval at a later date

  Standardized clinical documentation allows data to be more completely and consistently collected. In addition to HSM, we wished to collect selected elements of the clinical history and examination in a uniform and discrete fashion across different providers and store them discretely in the KP data warehouse. We chose to use the documentation processes available within Epic, with HL7 feeds from the EHR going directly to the KP data warehouse.

• Development of a system for obtaining follow-up for selected patient subgroups at standard intervals

  In clinical practice, both follow-up and timing of return appointments are variable and are affected by many factors such as patient and physician preferences, scheduling issues, and patient status. This variability can introduce a potential source of bias into the KP data warehouse and thus the potential for biased assessment of care outcomes. To minimize potential bias, we developed methods to systematically obtain follow-up data using both EHR and KP software.

• Consolidation of clinical and administrative data from multiple information systems into a clinical data repository (the KP data warehouse).

  The clinical data repository is a key component, serving as the central source for all KP-acquired data. It allows additional organization and codification of patient-entered data in context with other clinical and administrative data shared from ancillary systems----lab medicine, imaging, and scheduling, among others. Consolidating these data sets makes it possible to search efficiently across patient populations. Ultimately, the combined dataset, including a wide range of data domains, serves as the data source for a query tool.

• Development of web-based query tool allowing immediate access to data in the KP data warehouse:

  We have developed a web-based query tool that is protected by appropriate security measures. This data extraction tool allows users to identify or filter the subjects for which they would like data and provides an output selection system so users can choose and export the data elements to display for the selected cohort.

SYSTEM DESCRIPTION

Epic is fully implemented in all ambulatory settings at the Cleveland Clinic, including 12 family health centers, and in all hospital units. Clinical information is entered directly into Epic at the point of care. Clinicians can place orders and get results directly within the EHR, and it is the primary source of clinical information. For the KP, we have developed custom software that runs alongside the EHR. This software provides an added layer of clinical information, primarily HSM, that is stored discretely using structured documentation. The system includes a web-based data collection system, HL7 interfaces and web services (for system-to-system communication), a data-processing tier, and a relational database, including feeds from the different clinical and administrative systems within the Cleveland Clinic. The system was designed in C#, using the ASP.NET for the web-based data capture, relying primarily on XML and XSL. The HL7 communication is managed with Corepoint, and the backend dataset is managed with Sql Server. Custom classes were defined to handle data processing.

The KP development team consists of three closely integrated groups: a programming group comprising software engineers with expertise in .NET development and Epic-trained systems analysts, a clinical group that guides the clinical content and training, and a growing data utilization group consisting of an outcomes manager and a statistician (a database programmer will be added). The KP clinical working group, comprised of physician representatives from each clinical area, advises the core team on system functionality, clinical utility, and EHR integration. The KP leadership committee meets periodically and provides overall direction for the initiative.

The KP began as an initiative within the NI, which consists of 15 disease-specific centers that are organized to provide integrated, multidisciplinary care for patients with various neurological disorders.

Health Status Measures

Initial efforts focused on the systematic collection of HSM. Currently, both generic and disease-specific health-related quality-of-life measures are completed for each patient. Each provides different types of information and together allow a more complete assessment of a patient’s overall health status.^{12, 13} Two generic measures, the European Quality of Life Index (EQ-5D) and the Patient Health Questionnaire (PHQ-9), are collected for all patients seen in the NI ambulatory clinics. The EQ-5D is brief, has been used in many diseases, and provides health utilities that can be used in cost-utilization and cost-effectiveness analyses.¹⁴ The PHQ-9, a widely used scale measuring depression,¹⁵ increases the potential for treatment of a medical condition that affects a significant proportion of neurological patients. Ultimately, clinical action based on the analysis of PHQ-9 responses could significantly improve the quality of life for patients treated in the NI.

The disease-specific measures were chosen based on consensus by disease experts within the NI and were reviewed to ensure validity and their availability in the public domain. These measures are primarily patient-reported scales, which increasingly are being used to evaluate the impact of medical interventions on patient outcomes. Patient-reported measures also have the advantage of reducing healthcare provider workload. Patients complete the web-based questionnaires either in the waiting room using a wireless tablet or at a computer kiosk, or at home over a secure internet connection prior to their appointment. The patient responses are immediately available to the clinician, alleviating the need for exhaustive interviews with each patient.

Health Status Measures Workflow

The HSM data collection process has three major steps:

1. Patient completion of HSM questionnaires

2. Health care provider review of the patient responses and completion of additional provider-reported questionnaires

3. Discrete response data storage into the KP data warehouse and transmission of system-generated summary text for inclusion in the patient’s EHR.

Patients in the NI are scheduled, typically 20 minutes before their regular clinical encounter, to complete their HSM questionnaires. Upon arrival, patients are instructed on the HSM process and how to use the tablet or kiosk. Each patients logs in with either their medical record or social security number and date of birth. Upon patient authentication, disease-specific questionnaire content is generated automatically based on patient demographics, visit types, centers, and providers. The system then packages the relevant questionnaires and delivers them (question by question) to the patient using web-based data entry forms.

Figure 1 illustrates the data flow from the point of patient arrival and log in through provider approval and serialization to the KP data warehouse; the flow of scheduling information into the KP database is shown within the dotted lines at the bottom.

Figure 1.

Data flow through the KP from time of patient scheduling and log in to data entry into the data warehouse.

The data forms are response-driven, so questions may vary from patient to patient within a given HSM, depending on responses from either the patient or provider. For example, if a Spine Center patient indicates on a pain location question that he or she has neck pain, the Neck Disability Index will be included in that patient’s HSM form. When each question is completed, the data are sent in real-time to the KP data warehouse. Patients may also complete the assessment from home before their visit over the internet from Epic’s web-based patient portal, My Chart.

Healthcare providers have immediate access to the patient-reported dataset through a custom link in Epic that launches the KP HSM applicationand they are encouraged to review the responses with the patient during the current visit. While the clinician is assessing the patient and completing the patient encounter data, he or she will also answer provider-directed questions in the KP. After reviewing the patient responses and completing the provider section of the questionnaire, the provider approves the HSM, at which point all of the information is stored in the KP data warehouse and automatically sent as summary text back into the EHR. Thus, the patient outcome data are available as part of the patient’s permanent medical record. Accurate collection and processing of individual responses are critical, so validation is performed at data entry. Data forms cannot be approved and sent to the EHR unless they pass the validation.

Rollout of the KP with the HSM collection took place a few centers at a time initially on the main campus. Focused training was done separately for reception desk personnel, administrators, and healthcare providers prior to each rollout. The KP clinical group was present and available to aid healthcare providers as they began the HSM collection process. Once main campus rollout was completed, implementation in the 15 family health centers followed immediately. Collection of HSM is an NI-wide initiative and is considered part of our standard clinical workflow and care. Participation in the process is mandatory, and patient and provider questionnaire completion rates are collected and monitored.

Enhancement Tools for Health Status Measures

Completion Reports

Providing automated completion reports enables the individual centers to easily monitor patient and provider participation in the HSM collection and identify opportunities to increase participation. These real-time reports provide completion rates for the assessment forms at the center, provider, and patient levels, along with the percentage of forms that were fully completed (both the patient and healthcare provider completed the entire data form and the provider-approved the HSM form). Designated individuals within each center can log on and filter the completion report by time interval, center, and providers within their centers. Standardized completion reports are sent monthly to each Center so they can be reviewed at their business meetings. Currently, the minimum acceptable completion rate for the fully completed form is 75%.

Content Manager

The content manager is a software utility for developing new questionnaires (and their associated data forms) and revising existing questionnaire. Its development has led to a significant reduction in the time and effort required to add or revise questionnaires. Use of the content manager does not require any programming knowledge. As a result, authorized individuals can easily update forms or create new questionnaires without assistance from the KP software development team, even if the data forms are fairly complex. Authenticated users are able to assign the data source (patient or provider), title, validation checks, question type, values, and scoring methods. Because many of the questionnaires are relevant across specialties, they are automatically added to the KP questionnaire library, making them available to all participating centers.

Flow Sheet

Visualization of a patient’s individual scores and scales over time has been incorporated through KP flow sheets, enabling the longitudinal tracking of individual patient health status. (Figure 2). Healthcare providers access a custom tab within the clinical encounter in the EHR to automatically display graphs of HSM results. The provider can customize the graphical display properties and print out flow sheets for the patient’s reference. These flow sheets serve as a useful guide for individual patient management decisions and help to improve communication between the patient and provider concerning the patient’s progress over time

Figure 2:

Example of health status measure flow sheet for a patient from the Epilepsy

Follow-up Data Collection

We are currently identifying specific patient subgroups for whom follow-up information is particularly important, and are creating an electronic infrastructure that specifically permits follow-up data to be easily collected and transferred into the KP data warehouse. Patients for whom systematic follow-up at specific intervals is especially useful include those who (1) are actively managed by NI providers rather than those seen for a one-time consultation, (2) receive a procedural or medical intervention or undergo some type of surveillance over time, or (3) can be categorized precisely by diagnosis, procedure, or symptom complex. To identify the appropriate follow-up date, all patients must have at least one “anchor date” that provides a starting point from which to evaluate changes over time and to calculate the time for follow-up assessment. The anchor date(s) may refer to time of symptom onset, intervention, or when first seen by an NI provider. Most centers have incorporated anchor dates into their HSM forms.

The patients selected for follow-up are identified by using a combination of diagnostic and clinical information from procedural logs and the HSM. Those selected patients who have not been seen in the outpatient clinic within a designated time window are followed up by telephone. The KP flags the record and an e-mail containing relevant clinical information and a direct link to the EHR is sent automatically to the designated health care provider groups. Nurses receive the e-mail, open the patient’s electronic chart via the link, telephone the patient, and obtain the follow-up information in structured format that is entered into both the EHR and the KP data warehouse. The prototype for this process has been developed for patients discharged from the hospital with a principal diagnosis of stroke.

Structured Documentation

Our approach to structured documentation is to whenever possible enter information directly into the EHR and maximize the use of its structured documentation features. We categorized the components of the history and physical by priority and complexity. Past medical history was chosen as the initial component, followed by past family history, past surgical history, review of systems, history of present illness, and physical examination. We identified seven diagnostic categories in the past medical history that are important components of risk adjustment models of mortality and many HSM: hypertension, chronic renal insufficiency, cancer, stroke, coronary artery disease, depression, and diabetes. Providers must indicate whether these comorbid conditions are present or absent for each patient (Figure 3). Entering more specific information about these conditions, such as type of cancer, is optional and can be entered by clicking on the “notes” figure and entering a more specific diagnosis. In addition, disease-specific conditions designated by each NI disease-based center are presented in the same format. The native Epic forms for family history and review of systems are employed, using a standard group of elements for each as composed by the KP clinical working group, with subsequent tailoring by each center. We envision the history of present illness sections to be entirely disease-specific.

Figure 3:

Structured past medical history

The KP Data Warehouse and Query Tool

Production data captured through the HSM application and obtained from ancillary clinical and administrative systems are processed nightly and sent to the KP reporting data warehouse. Data are securely acquired via HL7, SQL Server Integration Services, and direct database-to-database querying. The data are stored in a relational fashion, enabling efficient use of disk space, indexed search operations, and object level security. The KP reporting data warehouse has been modeled with querying and aggregation in mind, and operates in its own segregated environment. This buffers production data processing from the resource-intensive queries that will be run against the reporting dataset. All KP data storage and access models have been approved by our Institutional Review Board.

KP Reports is a web-based service that provides a library of existing reports as well as ad-hoc reporting capabilities for administrative and quality intelligence. Every report can be exported to a number of common formats, as well as configured for auto-delivery via email on a recurring schedule.

An additional option available for end-utilization is the KP query tool application. This application provides end-users with immediate access to KP data from their desktops for use in research, quality, and patient care activities. The KP query tool is an ASP.NET application written in C#. The application sits behind the Cleveland Clinic firewall and can be accessed on the intranet. The nature of user access varies by the purpose of the query (patient management, quality, preparatory for research, and research) and type of data requested (aggregate, limited dataset, identifiable data). Requests for data for research require separate IRB approval. Users first select characteristics of the cohort they would like data on and then select the data elements to be retrieved. Data can be downloaded once requirements are met. A full audit trail is recorded for all actions taken within the query tool. Figure 4 shows the data categories and dataflow of the KP data warehouse.

Figure 4:

Data flow of KP data warehouse

Data Quality and Backup

Data quality is central to KP success, and data quality and backup are controlled through a combination of personnel and technology. The application framework provides data protection and the tools for active data analysis. The system offers immediate feedback at the point of data entry, and the user interface allows expert users to review and refine the data for accuracy.

The primary method to protect the quality of data entered directly into the KP and EHR involves data checks. Integers, decimals, dates, and strings have specific characteristics that can be assessed at runtime. The HSM applications content manager places data types at the forefront of the design process so that content manager users determine and set the response type, data domain, and ranges for each data element when constructing questionnaires. Appropriate data typing and domain assignment during content management provide the necessary metadata for the software to perform data validation during data entry. As a result, each data point can be evaluated using regular expressions and range-checking. Identifying data entry errors immediately allows data correction that would otherwise be impossible.

Periodic data reviews with participating disease-based centers add a high level of data validation. Reports are generated for each center, including patient volumes, key outcome metrics, clinical markers and relevant dates (such as onset of symptoms and interventions). This information is reviewed jointly by the clinical and KP technical staff members. Problems are identified, logged, and corrected in short cycles through this collaborative approach. The period data review process helps improve the quality of the system from data modeling through to data reporting.

The dataset is physically protected through a series of safeguards that have been established within the KP. The production servers are housed in a centralized data center. Physical access to the data center is secured via an electronic swipe card system, the server room has a climate control system, an emergency power backup system is installed, and the server cabinet is equipped with battery backup to prevent power loss. Consistent, redundant backup plans, including off-site long-term storage, have been implemented to protect against data loss.

EVALUATION AND CURRENT STATUS

Health status measures:

At the end of 2010, 23 distinct KP HSM questionnaire sets had been rolled out that consisted of 206 provider-reported scales, 123 discrete provider-entered questions, and 113 scales and 87 individual questions answered by providers. Data from KP questionnaires were available from 213,455 clinical encounters, representing 123,455 patients. Mean completion time for the HSM questionnaires was 9.5 minutes (range, 4.5 to 16.3 minutes) with a median of 8 minutes. In response to questions routinely asked in all questionnaires, most patients (73.2%) reported that completing the questionnaires was at least somewhat helpful in relaying information about their condition to their provider. Completion rates for questionnaires by patients and providers were 75% and 88% respectively.

The HSM collection has expanded to 4 of the 17 clinical institutes within the Cleveland Clinic and become a routine part of the clinical workflow for reception desk personnel, patients, and providers.

Structured documentation:

A standardized method for entering past medical history, family history, and surgical history was implemented in all NI centers in June 2010. This includes the required completion for the presence or absence of the seven items in the past medical history. We are in the process of implementing a standardized method for discrete, retrievable entry of the review of systems.

Systematic follow-up:

In December 2010, a systematic approach to obtaining follow-up information was implemented for patients discharged with a primary diagnosis of stroke. To aid in tracking HSM completion when follow-up is via telephone, a weekly completion report is sent to nursing personnel and administrators. An identical data form with discrete fields has been developed for in-person follow-up visits so exactly the same data is collected in both telephone and in-person follow-ups.

Data warehouse and query tool:

Version 1 of the KP query tool was released to NI physicians and administrators in December 2010. Training and education are ongoing, but as of March 2011, the query tool has been used to perform 651 queries. Data elements currently available for retrieval using the tool include demographic characteristics, HSM information, time to complete questionnaires, medications, radiology tests, free-text searches of radiology results, and diagnoses. We are expanding the data elements available for retrieval to include laboratory values and procedures and full-text clinical notes.

DISCUSSION

Improving the availability of electronic clinical data for various purposes and the systematic collection of HSM will become a necessary aspect of healthcare in the near future. Each will play an important role in comparative effectiveness and outcomes research. Our approach to achieving these goals has been to maximally leverage our EHR and then supplement its capabilities with an external IT infrastructure that integrates with the EHR. The external software system provides much more flexibility than the EHR but cannot replace the extensive features of a comprehensive EHR system. The literature contains relatively few papers on these objectives. The KP is distinct from other clinical data capture initiatives in both the breadth of disease it spans and its scope, which, includes the systematic collection of measures of patient-reported health status.

Collecting patient-reported HSM is central to the KP as this will allow us to better evaluate outcomes of care. Collection of patient-reported information is rapidly becoming a new standard for patient care and reduces the data collection burden for providers. Systematic collection of patient-reported information, however, requires greater coordination of workflow and the involvement of more members of the healthcare team, including reception and scheduling personnel. It also involves the modest cost of tablets or kiosks for patient data entry. Our process for systematic collection of patient-reported data has been successful, with questionnaire completion rates above 75%.

We are standardizing the information we enter as part of our clinical documentation, starting with past medical history. Along with the HSM and the discrete data elements already available in our EHR and administrative clinical databases, this will improve our ability to perform comparative effectiveness research, risk adjustment of outcomes, and predictive modeling. The highly variable documentation practices and autonomous practice of clinicians as well as individual patient heterogeneity pose challenges to implementing structured documentation within the EHR; thus, such standardization is an evolving process.

A critical component of this initiative is identifying the HSM and specific clinical data elements to collect. This requires careful thought and, ideally, the consensus of all groups collecting and using this information, so it is unavoidably time-consuming and resource-intensive. To optimize patient satisfaction, it is necessary to keep the patient-reported elements as brief as possible and limit patient questions to those that are directly relevant to the patient visit. Initially identifying key questions to be answered helps to ensure that the selection process is focused and will include elements that can be used later.

Technical challenges include the inflexibility of the EHR, continual evolution and changes to EHR functionality, and the intensive resources required to adapt and individualize the EHR for new uses. Although we envisioned and planned for some KP features to occur within EHR, we subsequently had to move these features outside the EHR’s constrained environment and into the more flexible KP software infrastructure. This feature set includes the ability to display serial HSMs for review in flow sheets, the data collection of “anchor dates”, which provide context for the clinical interpretation of HSMs, and automated notification of patients recently discharged from the hospital that require follow-up. In addition, the relational nature of the KP data warehouse, as with other clinical data warehouses that have similar designs, will lead to a slightly slower query response speed if the data warehouse is expanded to include other healthcare institutions. Currently, the query tool is best suited for simple queries, and more complex data requests will continue to require a programmer to retrieve the data.

The KP provides a test environment for studies on multiple topics such as the clinical impact of providing patient-reported outcomes available at the point-of-care, the comparative effectiveness of various interventions, and ideal methods for structured documentation of clinical information. We have begun to use the data to better understand neurological outcomes according to disease characteristics ¹⁶. We have also started a research program focusing on KP data utilization. The KP is continuing to expand in breadth and depth. The HSM data collection effort is spreading across the Cleveland Clinic to include other disease-based institutes. The program is extending data utilization through the incorporation of disease-based standards-of-care protocols or “care paths” into the electronic clinical workflow of patient care. In the care path model, standardized protocols and algorithms for managing diseases defined by clinical experts are integrated directly into the EHR. Through a combination of Epic-based functionality and external KP software features, the disease-specific care path will automatically present the appropriate order sets, clinical templates, and checklists and will provide best-practice alerts for selected patients. Reports on clinical characteristics and adherence to quality measures are generated at both an aggregate and patient level. The care path concept has been piloted for stroke patients and is being extended to other disease processes.

CONCLUSION

The approach that we have taken in developing and implementing the KP, along with the care path concept, will allow the realization of the long-sought goals to improve and standardize the data collected at the time of the clinical encounter. Moreover, we will be able to use this deep, clinically relevant dataset in analyses and future studies that will improve both the efficiency and outcomes of care.