Data-Driven Personalized Feedback to Patients with Type 1 Diabetes: A Randomized Trial

Abstract

Background: A mobile phone-based application can be useful for patients with type 1 diabetes in managing their disease. This results in large datasets accumulated on the patient's devices, which can be used for individualized feedback. The effect of such feedback is investigated in this article.

Materials and Methods: We developed an application that included a data-driven feedback module known as Diastat for patients on self-measured blood glucose regimens. Using a stepped-wedge design, both groups initially received an application without Diastat. Group 1 activated Diastat after 4 weeks, whereas Group 2 activated Diastat 12 weeks after startup (T1). End points were glycated hemoglobin (HbA1c) level and number of out-of-range (OOR) measurements (i.e., outside the range 72–270 mg/dL).

Results: Thirty patients were recruited to the study, and 15 were assigned to each group after the initial meeting. There were no significant differences between groups at T1 in HbA1c or OOR events. Overall, all patients had a decrease of 0.6 percentage points in mean HbA1c (P<0.001) and 14.5 in median OOR events over 2 weeks (P<0.001).

Conclusions: The study does not provide evidence that data-driven feedback improves glycemic control. The decrease in HbA1c was sizeable and significant, even though the study was not powered to detect this. The overall improvement in glycemic control suggests that, in general, mobile phone-based interventions can be useful in diabetes self-management.

Introduction

It is well known that digital tools such as mobile applications (apps) can benefit patients in their management of chronic diseases.^1,2 Many such apps are available, in particular targeted toward diabetes management,³ and the potential for improved self-management is substantial.^4–7 However, relatively few of the available apps are properly validated and evidence based.

Type 1 diabetes (T1D) is a disease that requires a large effort on behalf of the patient, as well as being one that the patient needs to actively manage at all times. Using continuous glucose meters may improve glycemic control in patients,⁸ but the cost and discomfort of these devices mean many patients still manage their disease through self-measured blood glucose (BG) regimens. Also, although closed-loop systems may be technologically feasible, they are unlikely to be offered as a solution for patients for many years to come. Hence self-management of diabetes will continue to be the key part of the daily life for many patients worldwide. Thus, with the proliferation of smartphones and suitable apps, patients are likely to use these as part of their disease management in the foreseeable future.

Meta-analyses suggest that telemedicine is an effective method for managing diabetes, particularly in terms of reducing glycated hemoglobin (HbA1c) levels,^6,7 with one study estimating a pooled effect of 0.44% (95% confidence interval, 0.61–0.26%) reduction.⁶ As an example, the TeleDiab-1 study did show a substantial decrease in HbA1c level using a combination of an analytics system suggesting insulin and teleconsultation.² In a follow-up study it was shown that the insulin suggestion system was effective when used alone.⁹

Because, by the nature of the disease, the apps are to be used long term, they need to be designed with this in mind. In particular, ease of use and low intrusion while still being perceived as useful are key factors to maintain usability.¹⁰

Patients need to take a large number of variables into account in everyday situations in order to make good decisions in their disease management. In particular, counting carbohydrates can improve insulin administration and BG control, which in turn can improve patients' quality of life.¹¹ The proliferation of mobile phone-based digital diaries means that many patients carry a large volume of data on their disease and past management. Designing and utilizing a system for using these data, it would be possible to provide data-driven decision support to the patients, as a complement to a knowledge-driven approach.

The current study investigates whether a data-driven feedback module can improve self-management for T1D. To this end, a mobile phone-based diabetes diary was developed, and an additional feedback module called Diastat, which could be enabled on the device, was implemented. Thus users could use the diary without this module during an initialization period such that baseline characteristics could be measured before Diastat was enabled.

The goal of the self-management in T1D is to maintain a stable BG concentration within prescribed limits, usually 72–180 mg/dL, through balancing factors of current BG concentration, insulin administration, carbohydrate intake, and other factors such as physical activity. The process requires knowledge of the disease and controlling factors, awareness, and self-efficacy. How each individual manages these factors are largely left to the patients themselves. Hence, the main hypothesis of this study was whether providing feedback mechanisms through analysis of the patient's own data could be a useful way to make the patient more knowledgeable about his or her own disease and disease management, in turn reducing HbA1c and out-of-range (OOR) measurements. The system tested in this study aimed at increasing the knowledge and self-efficacy and providing advice at the critical time in disease management. The design was user-driven in the sense that feedback from previous studies was used, and test users were given the opportunity to give feedback on the components of the system.

The status of diabetes management is often measured in terms of HbA1c, which reflects the long-term BG level of the patients. However, more recently measures of variability have been shown to be important qualities that affect the overall health status of the patients.^12,13 In data from continuous glucose meters, variability is often quantified in terms of mean amplitude of glucose excursions, but this is hard to compute consistently based on self-monitoring of BG data. In the current study we have therefore chosen to focus on the number of OOR measurements during a prefixed period as a crude measure of variability. Patients who manage their diabetes better should experience fewer OOR events, and along with the HbA1c level this yields a more complete picture of the patient's diabetes management than any one of them alone.

Materials and Methods

Design

The trial was designed as a randomized stepped-wedge trial with two groups.¹⁴ After a run-in period of 5 weeks, Group 1 served as an intervention group, and Group 2 served as a control group for 8 weeks. Subsequently, Group 1 was dismissed, and Group 2 served as an intervention group for 10 weeks. The same number of individuals was allocated to each group.

Participants and ethics considerations

Participants were recruited among eligible patients registered at the Division of Internal Medicine, University Hospital of North Norway, Tromsø, Norway. Inclusion criteria were as follows: older than 18 years of age; have had a diagnosis of T1D for at least 1 year prior to enrollment; and have basic familiarity with mobile phones and use a mobile phone on a daily basis.

Exclusion criteria were as follows: pregnancy; inability to understand or conform to the guidelines when presented with the app; and severe complications attributed to the diabetes that would render participation unethical or medically challenging, as determined by a physician. Use of insulin pumps or continuous glucose meters was not an exclusion criterion.

The study was presented to and a waiver was received from the Regional Ethics Committee of North Norway (protocol number 2011/1939-3). The data protection officer at the University Hospital of North Norway approved the data handling protocol.

Interventions

We designed a mobile app designed for patients with diabetes and have called it the Diabetes Diary (DD).¹⁰ The basic version was offered to the public after the study started, whereas the participant's version additionally had wireless transfer of BG values. This was achieved by pairing the mobile phone with a Bluetooth^® (Bluetooth SIG, Kirkland, WA) adapter connected by wire to a BG meter.

Additionally, we created a data-driven feedback module called Diastat that was designed based on feedback and data in a preceding study.¹⁵ The Diastat module consisted of three parts:

• 1. BG periodicity graph. When sufficient BG data points were recorded in DD, a smoothed daily graph and weekly graph were presented to the users as a representation of their typical daily or weekly BG level, respectively. The graph was computed as a smoothed regression with periodicity. The main line was computed using a Nadaraya–Watson estimator:

  

  where (t_i,y_i) are the observed BG values with time stamp and K_σ(·) is a Gaussian kernel with SD σ. Empirically, we set σ=(1, 4) h for the daily and weekly graphs, respectively. Adjustments to ensure continuity across midnight or end of the week were used. Hence, a patient who often had an elevated BG level at a certain time of day would get that presented as a spike in his or her graph. In addition, a gray area indicating uncertainty was overlaid to reflect variance. An underlying assumption of homoscedasticity (i.e., constant variance through each period) leads to the SD being estimated as¹⁶

  

  The gray area would thus reflect the large spread in values or sparse recordings. The daily graph was presented at the home screen of the app and could be further investigated by tapping this graph.

• 2. BG trends. Multiscale causal trends were computed based on cSiZer, an algorithm for change detection on any scale based on kernel regression methods.¹⁷ cSiZer detects a trend and labels it as significant when there is an increasing or decreasing trend over one or more time spans. A merging is applied to the trends to avoid cluttering the display such that only distinct trends were visible to the user. Because the trends were multiscale, a user could simultaneously have an increasing and decreasing trend or any other combination. However, at most times the user would not have an active trend or could have only one single trend line. When a user had a significant short-term trend, this would be indicated on the home screen, although trends on any scale could be investigated in the graph.

• 3. Situation matching. When a user was in a situation where insulin was to be injected, he or she could get information on similar situations to the present in his or her own data, along with how much insulin was administered at that time and the subsequent BG measurement. The similarities of two situations were computed as follows. Six features were defined: (1) time of day (ToD); last BG measurement (BG); last insulin recording (I); second to last insulin recording (I2); last carbohydrate registration (C) (summed over a window of 15 min); and physical activity over the last 24 h (A). All features were normalized by root mean squared over the entire dataset to obtain a feature vector v_K(i) for feature K and situation i. A weight was assigned to each observation, where ToD and A were weighted constant as 0.5, whereas all other features were assigned weights , where Δt is time since the observation was made and τK is the “half-life” for feature K. Half-lives were heuristically set at 2 h for I, I2, and C and 30 min for BG. Finally the distance between two situations i and j was defined as

  

  and situations most similar to the current one were presented to the user.

Screenshots of the application and the Diastat modules are presented in Figure 1.

FIG. 1.

Screenshots of components of the Diabetes Diary with the Diastat module installed: (left panel) the main page of the diary; (upper middle panel) periodicity graph over 24 h; (upper right panel) periodicity graph over 1 week; (lower middle panel) trend display showing one short-term increasing trend and one longer-term decreasing trend; and (lower right panel) situation matching, where the top section shows the best matched situations to the current and the lower section shows the chosen situation in context. Color images available online at www.liebertonline.com/dia

At the outset of the study, all participants received the basic version of DD along with thorough instructions. Because carbohydrate content in the food was a key feature in the situation matching, a course in carbohydrate counting was given by nutritional experts at the initial meeting. Participants were randomized after the initial meetings were completed. At 4 weeks after commencement, participants allocated to Group 1 were invited to a follow-up meeting where they received a new version of DD that included Diastat. Twelve weeks after commencement, Group 1 was invited to a final meeting, and Group 2 was invited to a follow-up meeting where they received DD with Diastat. This point is denoted T1 subsequently. At 23 weeks, Group 2 was invited to a final meeting. This is denoted T2. Figure 2 summarizes the design.

FIG. 2.

Flowchart. Vertical lines indicate physical meetings with adjacent numbers indicating number of patients met. Dashed lines are meetings where the Diabetes Diary was deployed. Shaded areas indicate periods where registrations are used for calculations of out-of-range measurements. White numbers are active patients (i.e., who had sufficient recordings for analysis in the period). Numbers in parentheses are how many met for glycated hemoglobin (HbA1c) measurements. Dates shown are those used for analysis and when the majority of the patients met. Because of practical considerations not all patients were able to meet on the day indicated, but they did meet on dates as close as possible. Randomization was done immediately after the initial meeting.

HbA1c was measured during the meetings at 0, 13, and 23 weeks or as close as possible in time to the meetings. Data on self-monitored BG values were recorded in their mobile phones and transferred anonymously to our in-house server.

Outcomes

Primary outcome was defined as the number of hypoglycemic and hyperglycemic events during observation periods. These were combined into number of OOR events. The optimal range was defined as 4–15 mmol/L (72–270 mg/dL), and any measurement outside this range was counted as an OOR event. The observation periods were the 4 weeks prior to any of the follow-up or discharge meetings (see Fig. 2). Change in HbA1c level for each group from baseline to the first observation was a secondary outcome.

Sample size

Sample size was determined by estimating the number of OOR events based on the prior study assuming a Poisson distribution in these and a 20% reduction in the intervention group with 80% power and 5% significance level. This resulted in 15 patients per group, and our target was to recruit 40 patients altogether.

Randomization

An independent statistician who had no contact with patients performed the randomization. An equal number of participants was allocated to each group. Randomization was performed after the initial meeting to blind participants and researchers to the group allocation at this meeting. Subsequent blinding was not possible because of the nature of the intervention.

Statistical methods

HbA1c levels were compared using a paired t test, whereas changes in OOR measurements were compared using the Wilcoxon signed-rank test. Analysis was performed both as per protocol and intention to treat. Unless otherwise specified, intention-to-treat results are reported. For the intention-to-treat analysis we imputed missing values by carrying the last observation forward.

Analysis was performed in R (version 3.0.1) software with the package “exactRankTests”¹⁸ to compute exact statistics for OOR.

Results

Thirty individuals met at the initial meetings, and all agreed to participate in the study. They were subsequently randomized to two equal groups. The baseline characteristics for the groups are shown in Table 1. There were no significant differences between the groups in terms of the observed variables at baseline. One person actively withdrew before the end of the study, and observations showed that one patient in each group did not record any BG measurements in the second observation period. These individuals were considered withdrawn. All patients had an HbA1c level measured at baseline, but eight of the 30 (27%) missed follow-up measurement at time T1, and six of 15 (40%) missed it at T2. Reasons for missed follow-up were not recorded.

Table 1.

Baseline Characteristics

	Group 1	Group 2	Overall	P
Age (years)	41.07±13.5	38.33±7.3	39.70±10.8	0.50a
Women (%)	66.67	60.00	63.33	1.00b
HbA1c (%)	8.33±0.87	8.06±1.32	8.20±1.11	0.51a

Data are mean±SD values or percentage, as indicated.

^aBy unpaired t test.

^bBy test of proportion.

HbA1c, glycated hemoglobin.

In Figure 3 each individual patient's HbA1c and OOR measurements are shown over the observation periods. The data are summarized in Table 2.

FIG. 3.

Changes in primary end point and glycated hemoglobin (HbA1c) level over the study: (left panel) HbA1c and (right panel) number of measurements outside the optimal range (out of range). Each line shows one eligible patient. Mean (left panel) and median (right panel) values are shown as diamonds for each time period and group.

Table 2.

Results for the Changes in Out-of-Range Measurements and Glycated Hemoglobin for the Different Time Periods and Associated P Values

		n	Change	95% CI (ITT)	P (PP)	P (ITT)
OOR
Baseline
All	46.0	30
Group 1	46.0	15
Group 2	46.0	15
T1
All	31.0	28	−14.5a	−18.0, −9.0	<0.001	<0.001
Group 1	32.5	14	−7.5	−15.5, −0.5	0.02	0.02
Group 2	29.0	14	−18.0a	−25.0, −13.0	<0.001	<0.001
T2
Group 2	34.5	10	−17.5	−29.0, −4.0	0.13	0.02
Reference T1			3.5	−10.0, 21.5	0.43	0.38
HbA1c
Baseline
All	8.20	30
Group 1	8.33	15
Group 2	8.06	15
T1
All	7.63	22	−0.60a	−0.90, −0.30	<0.001	<0.001
Group 1	7.89	11	−0.63a	−1.02, −0.24	0.005	0.007
Group 2	7.37	11	−0.57a	−1.10, −0.05	0.04	0.04
T2 versus
Group 2	7.49	9	−0.21a	−0.64, −0.22	0.29	0.09
Reference T1			0.16	−0.37, 0.05	0.11	0.11

P values were computed by the Wilcoxon signed-rank test for out of range (OOR) and paired t test for glycated hemoglobin (HbA1c).

^aSignificant difference.

CI, confidence interval; ITT, intention to treat; PP, per protocol.

A difference between the groups would be detectable at time T1 (see Fig. 2 and Table 2). The difference between the groups at T1 was not significant for either OOR or HbA1c level. If Diastat were an effective intervention, we would expect that Group 1 would have had a larger decrease in OOR events and HbA1c level than Group 2. At T1 the difference in mean between the groups was 0.51 percentage points in HbA1c (Group 1 higher, P=0.51) and in median 3.5 events in OOR (Group 1 higher, P=0.96). The difference in decrease between the groups from baseline to T1 was significant for OOR, with Group 2 having the largest decrease (P=0.012). This difference was not significant for HbA1c (P=0.86). Further details are given in Table 2.

Overall, the patients had a drop of 0.60 percentage points in HbA1c level from baseline to T1. This was most clearly pronounced in Group 1 but was still significant in Group 2. As seen in Figure 3, the effect is pronounced and fairly consistent across all the patients. There was a corresponding drop in OOR measurements, although this was most pronounced in Group 2. Running a linear regression model on the difference between HbA1c at baseline and T1 using age, sex, group, and baseline value showed that only the latter was significant. A larger decrease was observed in those with higher baseline (regression coefficient r=−0.30, P=0.004). A corresponding analysis for OOR showed a significant effect of group (P=0.019) and baseline OOR (r=−0.23, P=0.002) but not baseline HbA1c.

Discussion

There was little to no difference between the groups, and thus the study does not provide evidence that data-driven feedback to patients is useful in terms of medical indications. This does not mean that such feedback is not useful, only that the specific kinds of feedback considered in this study do not have the necessary impact to generate a difference between our two groups.

The study shows that the patients benefited from using the designed diabetes diary as an aid in controlling their diabetes in terms of both HbA1c and OOR measurements. Indeed, the decrease in HbA1c level is substantial and significant, which is remarkable given that the study was not powered to detect this difference. The overall effect can be attributed to study enrollment and thorough documentation of carbohydrate intake, activity, and insulin as well as training in carbohydrate counting. It is obviously not possible to disentangle the effect of the different factors in this study.

It is well known that interventions of any kind tends to improve health outcomes, and the decrease reported here from baseline to T1 is in line with other telemedical interventions.^1,2,5 The effect on HbA1c level reported here is at the higher end of what is seen in studies with educational interventions¹⁹ and meta-analyses,⁶ indicating that the tool provides good support in diabetes management.

It is notable that the design of the app and Diastat was not guided by theory or models on health behavior. The main driving force behind development was patient input and considerations on which analytical methods were applicable to these data.¹⁵ The lack of model guidance has been noted as a problem for the design of interventional tools with mobile technology.²⁰ However, the rapid pace of technological development means that applying theoretical guidance in this domain is challenging. It should nevertheless be done to the extent possible. The intervention was a combination of three different modules, which makes it impossible to distinguish which elements were more or less effective. Trend detection and periodicity were aimed at informing and thus empowering the patient as a means of behavioral change. If one of these showed a disadvantageous pattern, the patient should take corrective action, such that the feedback acts as a cue to action. The situation matching was a method to provide decision support at the relevant time, as a “just in time” intervention. In all cases, using the patient's own data was intended to be motivational. The failure of effectiveness could be a combination of several factors where lack of theoretical guidance in the design process, too complex a system, and a heterogeneous study population are likely to be main contributors. A theory-driven design targeted at applicable groups could be a way to improve these techniques.

The design of the DD as a whole and the Diastat components in particular was performed with usability and ease of use in mind, and patient feedback was requested throughout the process. Thus, we believe that the resulting app was usable and effective.

We chose OOR events as absolute values rather than rate as an outcome in order to try to get the most accurate picture of the actual number of undesirable events. This choice is arguable, but because the intervention itself is likely to alter the patients' recording habits, changes in actual events may be masked if using rates as outcome. The patients were encouraged to record when they suspected hypo- or hyperglycemic events, and thus the number of OOR events should reflect the glycemic control.

The study has some limitations. The interventions were designed using participatory design with a limited number of patients but were not validated in terms of how the users perceived the information and if they correctly understood it and the implications. The target sample size was not reached, and hence there could be effects of the intervention that the study is underpowered for. There were relatively many patients who were withdrawn from the study in terms of HbA1c level. In the intention-to-treat analysis, any bias resulting from this should in principle be removed. The study population may not be representative of the diabetes patient group but be biased toward patients with a technological interest and interest in improving their condition. Thus the results may not generalize to groups that are less motivated or less technically inclined.

We included users of continuous glucose meters in the study so as not to deplete the patient pool. This choice may induce a bias, although it is not clear which way. These patients may be more technologically induced and interested in contributing to the study, or they may not see an added benefit and consider the system as an additional burden in an already complex disease management.

One of the major questions relating to digital tool for chronic disease management is the long-term effect of the tools. It is likely that there is a wear-out effect for many patients, hints of which may also be seen in our trial as the outcomes do not improve and indeed deteriorate after the first 3-month period for Group 2. Digital solutions need to be designed such that the tools are easy to use and have low intrusion while they are perceived as useful for the user.²¹

No adverse effects were reported.

Conclusions

There were no significant differences between the groups in terms of HbA1c or OOR measurements, and thus this study does not provide evidence that the data-driven feedback module described here is effective in diabetes management. The overall reduction in HbA1c level strengthens conclusions from other studies that simply providing patients with tools leads to short-term improvement of disease management.

Acknowledgments

The study was funded by the Regional Health Authority of North Norway for 2012–2014 (ID number 6934/HST1065-12). We thank diabetes nurses Solrunn Coucheron, Mona Iren Torsteinsen, and Tord Hagen for their effort throughout the study. Kari Saxegaard and Aasa Løvfall at Norsk Diabetikersenter are thanked for giving the course in carbohydrate counting.

Author Disclosure Statement

No competing financial interests exist.