An Internet Service Supporting Quality Assessment of Inpatient Glycemic Control

Abstract

Background

Several studies have linked the maintenance of normoglycemia in acutely ill inpatients with improved clinical outcomes. We previously proposed a few standard definitions for monitoring inpatient glycemic control, or “glucometrics.” In clinical practice, limited data management resources for developing and refining measurement protocols can slow quality improvement efforts. With regard to glucometrics, there are few baseline data regarding the quality of hospital glycemic management. Furthermore, there are no reliable methods for hospitals to gauge the progress of their quality improvement efforts.

Methods

We built a novel Web application that calculates glucometrics on anonymized blood glucose data files uploaded by registered users. This Web site also collects many key characteristics of the users and institutions utilizing the service. This application will allow us to pool data from several institutions to calculate aggregate glucometrics, providing baseline data for quality improvement efforts and ongoing metrics for institutions to gauge their progress.

Results

The application, accessible at http://metrics.med.yale.edu, has already drawn visitors from several countries. A number of users have registered formally, and some have begun to upload institutional glucose data. The application delivers detailed glucometrics reports to registered users, complete with visual displays. Quality improvement staff from large health systems have been the predominant users.

Conclusions

We have created an open access Web application to facilitate quality monitoring and improvement efforts—as well as clinical research—regarding inpatient glycemic management. If employed widely, this application could help establish national performance standards for glycemic control.

Introduction

As a result of studies associating rigorous inpatient glycemic control with improved clinical outcomes, clinicians, hospitals, and standards organizations have shown increased interest in the glycemic management of hospitalized patients.^1–6 Authors publishing on this topic have employed a variety of definitions for glycemic control. To facilitate valid comparisons among studies or institutions, we previously proposed standard definitions for glycemic control, or “glucometrics.”⁷

We have also noted a gap between the development of new methods or standards and their routine use in clinical settings. The logistics of implementation often slow the adoption of quality improvement initiatives. Information technology can play a key role in overcoming such practical barriers, and in bridging the divide between research and clinical practice.⁸

All efforts to measure the quality of medical care face the twin hurdles of collecting and compiling the requisite clinical data, then developing protocols to calculate, report, and compare metrics. For glucometrics,⁷ the availability of point-of-care meters capable of storing bedside glucose measurements from many patients eases to some degree the burden of data collection. However, the calculation of glucometrics is not straightforward. To provide useful information, a glucose data set must be partitioned several times, then statistics and event rates must be calculated. The entire process must then be repeated for multiple data sets.

Not all institutions have ready resources to develop algorithms for these complex and laborious analyses. To remove this barrier to assessing the quality of inpatient glycemic control, we built an open access Web application capable of computing glucometrics on uploaded data.

Glucometrics

In outpatients with diabetes, measurement of hemoglobin A1c (HbA1c) is a valid and reliable proxy for glycemic exposure over the previous two to three months. Unfortunately, such a measure is not appropriate for short inpatient stays. Instead, glucometrics analyze a patient's bedside glucose measurements, using three time units relevant to hospitalized patients.

Just as HbA1c is a three-month measure of glycemic control, the mean bedside glucose from a patient's hospital stay is an n-day measure of glycemia. This varies with the patient's length of stay, measured in days. Call this the patient-stay metric. Similarly, the mean of a patient's glucose values for one day is a one-day measure of glycemia. Call this the patient-day metric. A single glucose measurement is a spot measure of glycemia during the few hours between measured samples. Call this the patient-sample metric (Figure 1).

Figure 1.

Glucometrics, units of analysis. Glucometrics uses three units of time for analyzing a patient's glycemic control. For each unit, it calculates glycemic exposure using mean glucose measurements, percent in the target range of 70–149 mg/dl, and the rate of hypoglycemia (glucose <70 mg/dl) and hyperglycemia (glucose >299 mg/dl).

Glucometrics are calculated on an inpatient cohort, for example, all patients on a single patient care unit with bedside glucose readings, over a one-month period. The cohort's mean, median, and variability of glucose measurements (whether individual samples or mean values) are then calculated for each of the three metrics discussed earlier.

Additionally, glucometrics calculate the percentage of measurements that fell within a prespecified target glucose range, and the percentage of measurements that met prespecified criteria for adverse events, namely hypoglycemia or marked hyperglycemia. These are the percentages of patient-stays, -days, or -samples that:

1. fall within a target glucose range, for example, 70–149 mg/dl; or

2. contain hypoglycemic readings (<70 mg/dl); or

3. contain markedly hyperglycemic readings (>299 mg/dl).

The values chosen as endpoints for these ranges are rather arbitrary. Other individuals may differ in their definitions for target levels or adverse metabolic events.

Adverse event rates vary widely, depending on the time unit used for calculation. For any cohort, the number of patient-stays is smallest, the number of patient-days is intermediate, and the number of patient-samples is largest. As an example, two patients hospitalized for five days with four glucose measurements daily, count for two patient-stays, ten patient-days, and forty patient-samples. One episode of hypoglycemia during a patient-stay marks that entire stay as “hypoglycemic.” This fact, combined with the smallest denominator, patient-stays (n = 2 in the example), yields the highest hypoglycemia rate. Arguably, the patient-stay metric gives an overly pessimistic impression of hypoglycemia rates during that hospitalization. On the other hand, one hypoglycemic sample divided by the largest denominator, patient-samples (n = 40 in the example), yields the lowest hypoglycemia rate. Arguably, this is an overly optimistic impression.

A patient-day is recorded as “hypoglycemic” if any measurement on that day was in the hypoglycemic range. However, the denominator here is patient-days (n = 10 in the example), an intermediate number between patient-stays and patient-samples, yielding an intermediate hypoglycemia rate, perhaps most reflective of the quality of care. Note that of the three metrics, only the patient-day metric represents a standard, unvarying time interval. Conversely, both the length of patient-stays and the time between patient-samples can vary widely from one cohort to another, making comparisons difficult. For these reasons, we recommend the patient-day metric for measuring adverse metabolic event rates. This metric most accurately reflects the quality of glycemic management.

Glucometrics Calculation

The process of glucometrics calculation begins with the collection of glucose data. While data flow will vary from one institution to another, here is one model: A patient has a bedside glucose measurement, and an individual glucose meter stores this glucose value, the patient's medical record number, and the time of the measurement. After many such measurements on several patients over a week, the meter is docked to a connection module on or near the ward. The meter then uploads a data file into a central hospital workstation. This workstation, which stores all data files sorted by floor and by glucometer, can also export files for analysis. Such files may contain data from a single floor, or may contain aggregate data from many floors.

A data file consists of rows of glucose measurements with many columns or fields for various other attributes, such as meter serial number, location, or operator. The calculation of glucometrics requires only three fields: the patient's unique numerical identifier (e.g., medical record number), the timestamp, and the blood glucose result.

Patient-sample metrics may be calculated using the whole data file. For patient-stay metrics, the file is partitioned into patient subsets using the medical record number. For patient-day metrics, each patient subset is further partitioned into individual calendar days, using the timestamp. Summary statistics and event rates are then calculated for each of these data subsets. For further details on the analysis of glucometrics, please see the Glucometrics™ Web site.

While each institution wishing to monitor its inpatient glycemic management could develop methods for the data manipulation described earlier, the process is laborious. In addition, if subsequent analyses are not fully automated, operator error is likely during the multiple steps involved.

Methods

We developed a database-backed Web application that calculates and reports glucometrics. It is accessible at http://metrics.med.yale.edu A user logs securely into the system, and then creates an institution name under which to upload anonymized data files. This institution can be further subdivided into wards or patient care units. Upon uploading a data file, the application computes glucometrics, generating a report as a Portable Document Format (PDF) file. The original data file and its PDF report are stored together under the institution and/or ward, and can be accessed at a later time. This feature allows each user to follow a ward's (or hospital's) performance over time. A single user may aggregate data from several wards or institutions for metrics reporting. However, each user cannot access other users' data.

To protect patient privacy, the Web site gives directions on how to anonymize medical record numbers in data files. When registering at the site and requesting access to the application, users are reminded via e-mail that they must anonymize all data prior to upload. They are also instructed to use a code name for all institutions. The site uses Secure Sockets Layer, a cryptographic protocol for secure communications, when communicating login information or any glucose data. This project was approved by the Yale Human Investigations Committee.

Although institutions remain anonymous, the system asks several questions when a user creates an institution or ward. The questions for institutions are as follows:

• Is it a member of the Council of Teaching Hospitals specified at the Association of American Medical Colleges Web site?

• Is it non-profit?

• Is it part of the Veterans Health Administration system?

• Is it a public hospital, that is, owned by the city, county, state, or federal government?

• Is it located in a rural health area as defined by the University of Washington School of Medicine Rural Health Research Center? (A link to a definition map is given.)

• What is its bedsize: <100, 100–400, or >400?

• In which of 10 U.S. regions (as designated by the Centers for Medicare & Medicaid Services) is the institution located? Non-U.S. institutions are asked for country of origin.

For specific wards, the following questions are asked:

• Is the unit an intensive care unit, and if so, what type: medical, surgical, mixed, cardiothoracic, coronary care, neurosurgical, neurological, neonatal, pediatric, or burn unit? Otherwise, is it a step-down unit or a general ward?

• What is the unit's specialty: internal medicine, surgery, medical oncology (and so on)?

• Finally, what is the age group of the unit's patients: adult, adolescent, pediatric, or neonatal?

The Web site administrators can use these institutional and ward characteristics to generate anonymous, aggregate glucometrics from the combined data of several institutions. This allows for rough comparisons of glycemic management, for example, between small and large hospitals, between U.S. regions, between medical and surgical units, or between intensive care units and regular wards. Biases inherent within such analyses (e.g., the nature of institutions that utilize such a Web site) will need to be recognized explicitly.

The Web site also provides a detailed description of glucometrics definitions, a forum where users may discuss the metrics and suggest refinements, and a place to share ideas for quality improvement of glycemic management.

The Web application was built using the open source Web application toolkit, OpenACS (available from http://openacs.org/); the open source web server, AOLserver (available from: http://www.aolserver.com/); and the open source database, PostgreSQL (available from: http://www.postgresql.org/.) Calculation of glucometrics, graphics, and report generation were accomplished with Mathematica and webMathematica, Wolfram Research's mathematical software system.⁹ The application runs on a Linux server at the Yale Center for Medical Informatics in New Haven, Connecticut.¹⁰

Results

Although development and early testing of the Web site began in May 2006, a completed version of the site went live in November 2006. From November 1, 2006 to November 25, 2007, the site had 2877 visits, of which 1966 were first time visitors and 911 were repeat visitors. Visitors came from 71 countries; the top five nations in number of visits were the United States, Australia, United Kingdom, Canada, and India (Table 1).

Table 1.

Countries with at Least Ten Visits to Glucometrics^TM Web Site

Country/Territory	Visits
United States	2387
Australia	108
United Kingdom	81
Canada	49
India	42
Belgium	16
Philippines	14
Spain	10
Malaysia	10
Germany	10

By November 12, 2007, 66 legitimate users had registered with the site (Table 2). Almost all registered users were from the United States, except for 2 from Australia and 1 from Spain. Twenty-five of these users obtained permission to upload data (Table 3). Tables 1–3 show users' institutional affiliations and roles. As of December 2007, users had uploaded more than 160 glucose files for glucometrics reports.

Table 2.

Registered Users of Glucometrics^TM Web Site

User Affilliation	Number
Community hospital	16
Community hospital and health system	4
Multihospital health system	19
University affiliated hospital	13
University	3
Public or government hospital	3
Veterans Health Administration health system	1
Diabetes center	2
Commercial organization	4
Healthcare management company	1
TOTAL	66

Table 3.

Users Asking to Upload Data to Glucometrics^TM Web Site

Role	Number
Hospitalist	1
Endocrinologist	4
Endocrinology Fellow	1
Nurse or Inpatient Diabetes Care Coordinator	1
Medical Student	1
Diabetes Educator	2
Pharmacist	3
Hospital Decision Support	1
Medical Technologist/Point of Care Coordinator	3
Diabetes Center Program Director/Coordinator	1
Diabetes Quality Improvement Team Member	8
TOTAL	25

Figure 2 (a–c) presents a sample glucometrics report for one month, from an adult medical intensive care unit. The report first displays the distribution of bedside glucose values using a composite graphic: a range bar showing data percentiles lying over a histogram (Figure 2a). This graphic is shown for each of the three units of analysis: patient-stays, patient-days, and patient-samples. Next, a table displays all of the metrics in full detail. This includes the percentage of glucose values that fell within the target range, as well as the percentages of glucose values fulfilling criteria for the adverse events of hypoglycemia and marked hyperglycemia. Figure 2a shows a data file with 1405 samples from 64 patients; there were 374 patient-days. The median glucose measurement for all patient-samples was 146 mg/dl. For patient-stays and patient-days, statistics were computed on the mean glucose measurements of the stay or the day. The number of patient-day mean glucose levels within the target range was 199 (53.2%). The number of patient-day means in the hypoglycemic range was 14 (3.7%).

Figure 2a.

A sample glucometrics report. The report shows glucometrics graphically and in tabular form. The patient's glycemic control is analyzed at three time units. Individual glucose measurements control for the shortest interval, the time between samples (patient-sample). Mean glucose measurements control for the longest interval, the entire hospital stay (patient-stay). A day's mean glucose measurements control for an intermediate interval, one day (patient-day). This is the only existing method that has a fixed interval that allows better comparison of one patient to another. Frequency distributions are shown; the dark bar on the x−axis between 70 and 149 shows a target or goal glucose range. Percentiles of the data are shown by the lines and dot over the histogram: 5——25 •50 75——95 (http://metrics.med.yale.edu/main, Yale Center for Medical Informatics & the Yale School of Medicine, Section of Endocrinology).

The number of patient-day means in the markedly hyperglycemic range was 26 (7.0%).

Figure 2b shows patients with just a single glucose measurement. Although these values have been excluded from the glucometrics analysis, a histogram shows the distribution of these single values. These measures may be useful for quality improvement. For example, if many of these values are over 200 mg/dl, one may ask why no one performed additional glucose measurements on these patients.

Figure 2b.

Patient-stays with only a single glucose measurement were excluded from the analysis.

Lastly, three sets of histograms and tables show the frequency of bedside glucose measurements, as well as the total duration of monitoring, in Figure 2c. This information may be helpful in making comparisons between patient care units. For example, a large difference in sampling frequency between units may undermine the validity of comparisons.

Figure 2c.

Glucose sampling characteristics. In the histograms, data outliers beyond the 95th percentile are not shown.

The Web site's database stores all uploaded data files and their glucometrics reports. Users may download reports for local sharing or printing, or they may access them later for comparisons to their most recently computed metrics. In addition, users may aggregate several of their data files and generate a composite metrics report over time.

As the Web site administrators, we can aggregate data files from many separate institutions to compute overall glucometrics. We can use these summary results to develop baseline performance benchmarks.

Discussion

National agencies studying quality of care, including the U.S. Joint Commission, have spotlighted optimization of inpatient glycemic control. Defining performance measures and calculating a set of baseline metrics takes the first step toward improving glycemic management across health systems. Presenting these results to frontline clinical staff then provides feedback, which is important to change behavior. This Web service aids institutions in both measuring and reporting glucometrics. The growing response from hospitals and health systems during the year since the Web site's inception (2006), despite little marketing, shows the rising interest in managing inpatient hyperglycemia. While most registered users of the site were from the United States, the visit log attests to the subject's international appeal. User affiliations and roles suggest that the most common user will be an active member of a quality improvement team in a multihospital health system.

We are aware of only one other application that collects bedside glucose data for quality improvement analysis.¹¹ However, it requires purchase and installation of proprietary middleware, with an add-on module for glycemic control. In contrast, our Web site is freely accessible over the Internet. Since the application runs on a server maintained at the Yale Center for Medical Informatics, we do not burden the user with software installation or updating. In addition, the other middleware application simply computes its statistics on all blood glucose results, without further analysis by patient-stay or by calendar day. As mentioned previously, this presents problems in the calculation of adverse event rates, which vary widely depending on the unit of analysis. Our Web site offers analysis of glycemic control over three different time units: the patient-stay, the patient-day, and the patient-sample. Such flexibility is valuable to users with various analytic needs.⁷ Finally, our Web site also analyzes the frequency and duration of bedside blood glucose sampling on patients in a cohort. This is important in assessing the validity of a data set.

After having collected glucose data from several institutions, we will aggregate data files and compute initial performance benchmarks. These benchmarks should be helpful to institutions wishing to compare their glycemic management to others. Of course, questions will surely arise about the validity of such benchmarks. One concern is lack of data about the prevalence of diabetic patients in patient care units; wide variation in the prevalence of diabetes may produce differences in glucometrics that have nothing to do with diligence in glucose management. Another problem in comparing different types of units, say medical and surgical patient care units, is a mixed patient population; some strictly “medical” patients may reside in a “surgical” unit due to logistical issues. Other questions may arise about associating institutional characteristics with glucometrics performance. For example, public hospitals may treat patients who, for a variety of reasons, have hyperglycemia that is difficult to control. While we have made an effort to characterize the institutions and patient care units uploading data to our server, the addition of other factors relevant to glycemic management will likely be necessary to draw more accurate comparisons.

Our Web site also encourages feedback from users by providing forums for discussion about glucometrics definitions and other material on the site. One pertinent topic is the choice of glucose ranges used in calculating targets and adverse event rates. While the current site does not allow users to modify these ranges, this dynamic feature can be added if the demand arises. In addition, users may share quality improvement strategies that have been successfully implemented at their institutions. Others can then study the effect of such interventions by comparing pre- and postintervention glucometrics within their own organization. By facilitating such interaction, the Web site should help to close the loop from research to clinical application to further research.

Abbreviations

HbA1c

hemoglobin A1c

PDF

portable document format