Abstract
Background
We have developed a prototypical case-based reasoning system to enhance management of patients with type 1 diabetes mellitus (T1DM). The system is capable of automatically analyzing large volumes of life events, self-monitoring of blood glucose readings, continuous glucose monitoring system results, and insulin pump data to detect clinical problems. In a preliminary study, manual entry of large volumes of life-event and other data was too burdensome for patients. In this study, life-event and pump data collection were automated, and then the system was reevaluated.
Methods
Twenty-three adult T1DM patients on insulin pumps completed the five-week study. A usual daily schedule was entered into the database, and patients were only required to upload their insulin pump data to Medtronic’s CareLink® Web site weekly. Situation assessment routines were run weekly for each participant to detect possible problems, and once the trial was completed, the case-retrieval module was tested.
Results
Using the situation assessment routines previously developed, the system found 295 possible problems. The enhanced system detected only 2.6 problems per patient per week compared to 4.9 problems per patient per week in the preliminary study (p = .017). Problems detected by the system were correctly identified in 97.9% of the cases, and 96.1% of these were clinically useful.
Conclusions
With less life-event data, the system is unable to detect certain clinical problems and detects fewer problems overall. Additional work is needed to provide device/software interfaces that allow patients to provide this data quickly and conveniently.
Keywords: artificial intelligence, case-based reasoning, decision-support software, insulin pump therapy, type 1 diabetes mellitus
Introduction
We are developing the 4 Diabetes Support System™ in an attempt to enhance the management of patients with type 1 diabetes mellitus (T1DM) on insulin pump therapy. This is a case-based decision support system that is now capable of automatically analyzing large volumes of life events, self-monitoring of blood glucose (SMBG) readings, continuous glucose monitoring system (CGMS) results, and insulin pump data to (1) detect abnormal excursions in glucose patterns, (2) recall solutions that successfully correct specific clinical problems, and (3) remember which solutions work for a particular problem case. A preliminary study resulted in the development of 50 clinical diabetes cases that are stored in the system’s clinical case base. The prototypical system can now scan up to three months of patient data and detect 12 distinct clinical problems from this information.1,2 We believe that this system will help overcome the “data overload” experienced by most diabetologists who are attempting intensive glucose control in patients with T1DM,3,4 as well as other forms of diabetes.
Case-based reasoning (CBR) is an artificial intelligence approach that capitalizes on experience with past problems and solutions to determine solutions for current problems.5 It complements algorithmic and rule-based approaches to decision support for diabetes management, which are more commonly utilized.6 Case-based reasoning has found use in medical domains due, in part, to its incorporation of patient case histories and its ability to integrate multiple types of data.7,8 We felt very strongly from the onset of the system’s development that the integration of life-event data along with glucose levels and basal/bolus insulin doses would be key to helping physicians and the system identify glucose trends more readily than routine data analysis. However, a major burden identified by patients in our preliminary study was the amount of time and effort required of them to supply the necessary data. In that study, each patient wore a CGMS sensor three times, for three days at a time, and entered extensive self-care and life-event data using a Web-based interface to a database. In the present study, the information already collected by the Medtronic Paradigm insulin pump, including real-time CGMS data and SMBG data from linked glucose meters, was uploaded to the CareLink® Web site and imported into our system.
This system enhancement has improved data entry accuracy and reduced time demands on the patient. In this study, assessing the detection and retrieval modules of the system, we purposely collected only typical daily schedules for the patients rather than specific data about actual life events experienced during the study. In addition, we collected CGMS data only for those patients who normally used it as part of their routine care. We report our findings, which demonstrate that the additional life-event and CGMS data is very important for the detection routines of a CBR system.
Patients and Methods
This study was approved by Ohio University’s Institutional Review Board and conducted to assess the ability of the 4 Diabetes Support System to detect abnormal glucose patterns with less life-event data. Twenty-six adult T1DM patients on insulin pump therapy enrolled in the five-week study. All subjects were patients from the Appalachian Rural Health Institute Diabetes and Endocrine Center who volunteered to take part in the study. These patients were moderately well controlled [average glycosylated hemoglobin (A1C) level: 7.51% ± 0.77%]. Patients with A1C <6.5% and those who had participated in the earlier study were excluded. Twenty-three of the 26 patients who entered the study completed the entire protocol. No participants dropped out of the study because of time required for data collection. The study differed from typical clinical research in that data collected from patients was used solely to assess the ability of the system to detect problems and propose solutions. However, information obtained during the study that was deemed clinically significant and necessitated a change in an individual patient’s management was communicated to each patient by their personal physician during the study.
Background data were collected from each patient at the beginning of the study and entered into an Oracle database. This included personal data, a diabetes history, occupational information, pump information, insulin sensitivity, carbohydrate ratios, A1C levels, presence or absence of diabetic complications, other chronic diseases, medications, and typical daily schedules for work, exercise, meals, and sleep. Since our original study, Medtronic implemented a feature in their CareLink software to allow the extraction of pump and meter data into a comma-delimited file. A program was written to parse this file and input the data into the 4 Diabetes Support System’s database. During the study, patients performed SMBG 6–15 times per day as per their routine care. Pump data, linked meter data, and CGMS data when available (6 patients out of 23) were directly uploaded to the Medtronic CareLink Web site by each patient weekly and then imported into the 4 Diabetes Support System’s database. Data that were automatically transferred in this study but manually entered in the preliminary study included bolus times, dosage, and waveforms; Bolus Wizard usage; temporary basal rates and pump suspensions; times of infusion set changes; SMBG values; and anticipated grams of carbohydrate. Data that were approximated by typical daily schedules in this study included times of meals, snacks, work, sleep, and exercise. Previously entered data that were not collected in this study included illness; stress; menses; symptoms and treatment of hypoglycemia; type, duration, and intensity of exercise; physical intensity of work; specific foods consumed; infusion set sites; known pump problems; and other miscellaneous events.
Once a week, the situation assessment routines were run for each participant to detect possible problems. These routines detect the following 12 clinical problems: (1) overcorrection for hypoglycemia; (2) hypoglycemia after exercise; (3) possible pump problem affecting insulin delivery; (4) overcorrection for hyperglycemia; (5) prewaking hypoglycemia, as evidenced by prewaking CGMS data; (6) overbolus for a meal; (7) hyperglycemia upon waking, as evidenced by the first SMBG value obtained postwaking and prebreakfast; (8) hypoglycemia upon waking, similarly determined; (9) premeal hyperglycemia; (10) premeal hypo-glycemia; (11) postmeal hyperglycemia; and (12) postmeal hypoglycemia. Physicians evaluated the system’s output for their patients each week. For each detected problem, the physician was asked if the problem detected was correct and if the problem detected was clinically useful.
A separate evaluation of the case-retrieval module was performed after all patients completed the five-week study. To provide decision support for determining appropriate therapy for a detected problem, the case- retrieval module finds the “most similar” case in the clinical case base, as described by Marling and colleagues.2 The retrieved case contains the therapeutic adjustments made for a similar problem in the past. To assess a broad sampling of retrieved cases, 102 case retrievals were examined, corresponding to the first instance of each type of problem detected for each patient. For each type of problem detected, one to three similar cases were retrieved from the system’s clinical case base. Using this information, the 102 retrievals were reduced to 17 distinct problem/case-retrieval pairs. Four physicians (one endo-crinologist, one diabetologist, and two diabetes fellows) and one advanced practice nurse specializing in diabetes evaluated the results. Each completed a questionnaire asking two questions about each of the 17 problems and the cases retrieved for them. The first question of each set was to evaluate the similarity between the original problem case and the problem in the retrieved case. The second question was to evaluate if the therapeutic advice contained in the retrieved case would be beneficial or detrimental to the patient experiencing the problem. The questions were answered using ordinal qualitative scales for similarity and benefit.
Results
Problem Detection
Over the course of the evaluation study, the 12 situation assessment routines were used to scan the data from each patient weekly. The system found a total of 295 possible problems for the 23 patients, 280 of which were reviewed by the patients’ personal physicians. Physicians concluded that 274 (97.9%) of the 280 problem detections were correct and 6 (2.1%) were incorrect. Physicians were also asked to rate whether the detections were clinically useful. Physicians rated 269 (96.1%) of the 274 correct detections as useful, 9 (3.2%) as not useful, and 2 (0.7%) as possibly useful. Table 1 shows the correctness of the problem detections for each of the 12 situation assessment routines, and Table 2 shows the usefulness of these detections. The correctness and usefulness of the problem detections for each individual patient are shown in Tables 3 and 4, respectively. The system detected on average 12.8 problems per patient during five weeks of data collection, or 2.6 problems per patient per week.
Table 1.
Correct Assessment by Routine
| Routine | Yes | No | Not evaluated |
|---|---|---|---|
| 1 | 0 | 0 | 0 |
| 2 | 30 | 1 | 0 |
| 3 | 12 | 3 | 7 |
| 4 | 7 | 0 | 0 |
| 5 | 0 | 0 | 1 |
| 6 | 1 | 0 | 0 |
| 7 | 31 | 0 | 0 |
| 8 | 67 | 2 | 0 |
| 9 | 27 | 0 | 0 |
| 10 | 66 | 0 | 3 |
| 11 | 2 | 0 | 0 |
| 12 | 31 | 0 | 4 |
| Total | 274 | 6 | 15 |
Table 2.
Useful Assessment by Routine
| Routine | Yes | No | Maybe |
|---|---|---|---|
| 1 | 0 | 0 | 0 |
| 2 | 30 | 1 | 0 |
| 3 | 12 | 3 | 0 |
| 4 | 7 | 0 | 0 |
| 5 | 0 | 0 | 0 |
| 6 | 0 | 0 | 1 |
| 7 | 31 | 0 | 0 |
| 8 | 67 | 2 | 0 |
| 9 | 24 | 3 | 0 |
| 10 | 65 | 0 | 1 |
| 11 | 2 | 0 | 0 |
| 12 | 31 | 0 | 0 |
| Total | 269 | 9 | 2 |
Table 3.
Correct Assessment by Patient
| Patient | Yes | No | Not evaluated |
|---|---|---|---|
| 1 | 9 | 0 | 0 |
| 3 | 1 | 0 | 1 |
| 5 | 6 | 0 | 0 |
| 6 | 15 | 0 | 0 |
| 7 | 10 | 0 | 2 |
| 8 | 9 | 0 | 0 |
| 9 | 19 | 0 | 0 |
| 10 | 8 | 0 | 1 |
| 11 | 18 | 0 | 3 |
| 12 | 41 | 3 | 1 |
| 13 | 7 | 0 | 2 |
| 14 | 0 | 0 | 0 |
| 15 | 13 | 0 | 1 |
| 16 | 15 | 0 | 0 |
| 17 | 14 | 0 | 2 |
| 18 | 8 | 0 | 1 |
| 20 | 14 | 0 | 1 |
| 21 | 11 | 0 | 0 |
| 23 | 21 | 0 | 0 |
| 25 | 9 | 0 | 0 |
| 26 | 6 | 0 | 0 |
| 27 | 3 | 2 | 0 |
| 30 | 17 | 1 | 0 |
| Total | 274 | 6 | 15 |
Table 4.
Useful Assessment by Patient
| Patient | Yes | No | Maybe |
|---|---|---|---|
| 1 | 9 | 0 | 0 |
| 3 | 1 | 0 | 0 |
| 5 | 6 | 0 | 0 |
| 6 | 15 | 0 | 0 |
| 7 | 9 | 0 | 1 |
| 8 | 9 | 0 | 0 |
| 9 | 19 | 0 | 0 |
| 10 | 7 | 0 | 1 |
| 11 | 18 | 0 | 0 |
| 12 | 41 | 3 | 0 |
| 13 | 7 | 0 | 0 |
| 14 | 0 | 0 | 0 |
| 15 | 13 | 0 | 0 |
| 16 | 15 | 0 | 0 |
| 17 | 11 | 3 | 0 |
| 18 | 8 | 0 | 0 |
| 20 | 14 | 0 | 0 |
| 21 | 11 | 0 | 0 |
| 23 | 21 | 0 | 0 |
| 25 | 9 | 0 | 0 |
| 26 | 6 | 0 | 0 |
| 27 | 3 | 2 | 0 |
| 30 | 17 | 1 | 0 |
| Total | 269 | 9 | 2 |
Problem-Detection Capability with Less Life-Event and Continuous Glucose Monitoring System Data
At 2.6 problems per patient per week, approximately half as many problems were detected compared to the initial study containing the additional life-event and limited CGMS data,1 which detected 4.9 problems per week (p = .017). Figure 1 shows the number of problems detected per patient per week in each study for each of the 12 situation assessment routines listed earlier. No significant differences for age, gender, occupational status, marital status, number of years with diabetes, years using a pump, A1C scores, or average number of SMBG values per week were found between the study groups to bias the outcome. While it is possible that other factors may have influenced the outcome, this finding suggests that the additional life-event and CGMS data in the preliminary study nearly doubles the sensitivity of the detection routines.
Figure 1.
Problem detections in the preliminary study compared to detections with the enhanced interface.
Case-Retrieval Assessment
Once the 23 patients were finished with the study, the case-retrieval capacity of the system was assessed by evaluating the case similarity and usefulness of solutions found by the case-retrieval system. Overall, matching cases were judged to contain similar problems 79% of the time. The system retrieved a matching case with a problem that was judged to be very similar 34% of the time, somewhat similar 45% of the time, somewhat dissimilar 14% of the time, and very dissimilar 7% of the time. Overall, the system retrieved a matching case with a beneficial solution 82% of the time, with 36.5% of suggestions deemed to have been very beneficial, 45.9% somewhat beneficial, 8.2% neither beneficial nor detrimental, 8.2% somewhat detrimental, and 1.2% very detrimental. This was comparable to results obtained during the preliminary study.1,2 Testing performed at that time found that retrieved problems were similar 80% of the time and that associated solutions were beneficial 70% of the time, neither beneficial nor detrimental 23% of the time, and detrimental 7% of the time. Note that, for testing purposes, the closest matching case was always retrieved, even when no case in the clinical case base was truly similar to the problem at hand. In practice, a similarity score threshold would prevent dissimilar cases with potentially detrimental solutions from being used. The quality of decision support depends upon the availability of enough similar cases in the clinical case base; therefore, additional cases are currently being developed.
Discussion
It is estimated that patients with T1DM spend more than 2 hours per day on diabetes self-care.9 In our initial study, participants felt that the additional time required to enter life-event and insulin-pump data into our system was too burdensome to be beneficial to their management. In fact, 8 of 20 patients dropped out of the preliminary study due, in part, to the burden of data collection.1 As we continued developing the 4 Diabetes Support System, improvements were made to reduce the time required for patient data entry by directly obtaining the data stored on each patient’s pump. This enhanced the 4 Diabetes Support System’s ability to incorporate diabetes monitoring data automatically, which reduced patient effort. However, we only collected typical patient schedules, not specific data about life events that actually occurred during the study.
With the decreased clinical life-event information available, the system was only half as effective at detecting problems compared to the original study.1 This indicates that we must determine which life-event data are truly necessary for a more robust detection capacity and then automate this data input as much as possible to reduce patient time requirements and yet capture this critical information. In this study, only 6 of 23 patients used CGMS as part of their routine care; however, the detection rate was greater for them compared to those who did not. The precise contribution of CGMS alone is not addressed in the current study but warrants further research. Continuous glucose monitoring systems have been shown to improve glucose control in T1DM patients using continuous subcutaneous insulin infusion or multiple daily insulin injections.10 One patient who used CGMS continuously throughout the study had a detection rate of nine problems per week. The other 5 patients who used CGMS, 3 continuously and 2 intermittently (ranging from 16 to 35 of 35 days), had an average detection rate of three problems per week, which compares to the overall average detection rate of 2.6 problems per patient per week. In the preliminary study, all patients provided CGMS data for only 9 of 42 days. The higher detection rate in that study suggests that life-event data are extremely important for the detection capacity of the system.
A weakness of our detection studies is that there is no way to precisely determine the number of actual patient problems that the system or the physicians developing the system fail to detect. Although the physicians did review the data downloads utilizing the graphic presentation of patient information developed in the first study,1 the lack of life-event data may have reduced their ability to detect additional problems, which may have been buried in the volume of data from patient downloads. In fact, this is one of the motivating factors behind providing an automated problem-detection system in the first place.
We have previously stressed that computerization of glucose data management has actually created the paradox of having too much data and not enough data at the same time.1 Surveys have shown that most diabetologists are overwhelmed by the volume of data and the time required to analyze it.4 Previous attempts at glucose pattern analysis, including assessment of trends by date, time of day, relationship of blood glucose to meals, postprandial excursions, the effects of day of the week, and interactions between time of day and day of the week, have demonstrated improved glucose control with decreased risk of hypoglycemia.11,12
The 4 Diabetes Support System is now capable of screening large volumes of glucose and life-event data obtained from persons with T1DM on insulin pumps and detecting multiple problems. Additional work is needed to enhance and expand the current prototype into a system that would be safe and effective for clinical use by physicians managing patients with T1DM. Observations made during this study are being used to guide the enhancement and expansion of the system prototype. These issues are being addressed in a separate ongoing study in which 28 patients are using CGMS for the entire duration of a 3-month study. One goal is to enlarge the clinical case base so that it will contain more beneficial solutions for retrieval. We envision that the system will evolve with increased use; as new cases and solutions are recognized, they can be continuously incorporated into the system. Plans also include the development of additional problem-detection routines. As a result of this study, five new routines detecting abnormal glucose patterns commonly seen while a patient is sleeping have been developed and are awaiting formal evaluation. These routines detect (1) nocturnal hypoglycemia, (2) dawn phenomena, (3) Somogyi phenomena, (4) sustained nocturnal hyperglycemia, and (5) hyperglycemia shortly after going to bed. Another focus area for problem-detection is excessive glycemic variability.
Conclusions
In this study, we have been able to automate collection of much of the clinical data needed for a rich CBR diabetes management system, which reduced the amount of work that patients had to perform while participating in this study. However, with less life-event and CGMS data, the 4 Diabetes Support System was unable to detect certain types of clinical problems and detected fewer problems overall. In addition to expanding the clinical case base and developing new problem-detection routines, further efforts in the development of the 4 Diabetes Support System will focus on using hand-held devices such as smart phones with drop-down menus and macros that will enable important life-event data to be input to the system with minimal additional effort by the patient.
Acknowledgments
We thank Tessa Cooper, Eric Flowers, Thomas Jones, Anthony Maimone, Wesley Miller, Donald Walker, and Matthew Wiley for software development and knowledge engineering. We also thank our research nurses Cammie Starner, R.N., Lynn Petrik, R.N., Barb Pellegrin, R.N., A.N.P., and Regina Reed, R.N., A.N.P.
Glossary
Abbreviations
- (A1C)
glycosylated hemoglobin
- (CBR)
case-based reasoning
- (CGMS)
continuous glucose monitoring system
- (SMBG)
self-monitoring of blood glucose
- (T1DM)
type 1 diabetes mellitus
References
- 1.Schwartz FL, Shubrook JH, Marling CR. Use of case-based reasoning to enhance intensive management of patients on insulin pump therapy. J Diabetes Sci Technol. 2008;2(4):603–611. doi: 10.1177/193229680800200411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Marling C, Shubrook J, Schwartz F. Towards case-based reasoning for diabetes management: a preliminary clinical study and decision support system prototype. Computat Intel. 2009;25(3):165–179. [Google Scholar]
- 3.DeVries JH, Snoek FJ, Kostense PJ, Masurel N, Heine RJ, Dutch Insulin Pump Study Group A randomized trial of continuous subcutaneous insulin infusion and intensive injection therapy in type 1 diabetes for patients with long-standing poor glycemic control. Diabetes Care. 2002;25(11):2074–2080. doi: 10.2337/diacare.25.11.2074. [DOI] [PubMed] [Google Scholar]
- 4.Shah BR, Hux JE, Laupacis A, Zinman B, van Walraven C. Clinical inertia in response to inadequate glycemic control: do specialists differ from primary care physicians? Diabetes Care. 2005;28(3):600–606. doi: 10.2337/diacare.28.3.600. [DOI] [PubMed] [Google Scholar]
- 5.Kolodner JL. Case-based reasoning. San Mateo: Morgan-Kaufmann; 1993. [Google Scholar]
- 6.Rodbard D, Vigersky R. Development of algorithms for clinical decision support for primary care providers. Diabetes Technol Ther. 2005;7(2):409. [Google Scholar]
- 7.Holt A, Bichindaritz I, Schmidt R, Perner P. Medical applications of case-based reasoning. Knowledge Engineering Rev. 2005;20:289–292. [Google Scholar]
- 8.Bichindaritz I, Marling C. Case-based reasoning in the health sciences: what’s next? Artif Intell Med. 2006;36(2):127–135. doi: 10.1016/j.artmed.2005.10.008. [DOI] [PubMed] [Google Scholar]
- 9.Shubrook J, Schwartz F. Time demands for diabetes self-care. Presented at: American Diabetes Association 66th Scientific Session; 2006. [Google Scholar]
- 10.Rodbard D, Jovanovic L, Garg SK. Responses to continuous glucose monitoring in subjects with type 1 diabetes using continuous subcutaneous insulin infusion or multiple daily injections. Diabetes Technol Ther. 2009;11(12):757–765. doi: 10.1089/dia.2009.0078. [DOI] [PubMed] [Google Scholar]
- 11.Rodbard D. Optimizing display, analysis, interpretation and utility of self-monitoring of blood glucose (SMBG) data for management of patients with diabetes. J Diabetes Sci Technol. 2007;1(1):62–71. doi: 10.1177/193229680700100111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Freeman J, Gillespie S. Pattern management. Looking for clues. If stable blood glucose eludes you despite your best efforts, it’s time to investigate. Diabetes Forecast. 2004;57(8):57–60. [PubMed] [Google Scholar]