Effects of consumer-oriented health information technologies in diabetes management over time: a systematic review and meta-analysis of randomized controlled trials

Abstract

Objective: To reveal the effects of consumer-oriented health information technologies (CHITs) on patient outcomes in diabetes management over time through systematic review and meta-analysis.

Methods: We searched 5 electronic databases (from database inception to July 2016) for studies that reported on randomized controlled trials examining the effects of CHITs on glycemic control and other patient outcomes in diabetes management. Data were analyzed using either meta-analysis or a narrative synthesis approach.

Results: Eighty randomized controlled trial studies, representing 87 individual trials, were identified and included for analysis. Overall, the meta-analysis showed that the use of CHITs resulted in significant improvement in glycemic control compared to usual care (standardized mean difference = −0.31%, 95% confidence interval −0.38 to −0.23, P < .001) in patients with diabetes. Specifically, improvement in glycemic control was significant at intervention durations of 3, 6, 8, 9, 12, 15, 30, and 60 months, while no significant differences were found at other time points reported. The narrative synthesis provided mixed effects of CHITs on other clinical, psychosocial, behavioral, and knowledge outcomes.

Conclusions: The use of CHITs appears to be more effective than usual care in improving glycemic control for patients with diabetes. However, their effectiveness did not remain consistent over time and in other patient outcomes. Further efforts are required to examine long-term effects of CHITs and to explore factors that can moderate the effects over time.

INTRODUCTION

Diabetes is an important public health care concern due to its high prevalence, morbidity, and mortality and escalating costs.^1,2 Challenges in diabetes management require innovative self-care strategies to improve glycemic control, which is the major goal in the care of diabetes and key to reducing the risk of microvascular complications and death related to the disease.² In light of this, various health information technologies (HITs) have been developed to facilitate patients’ care of diabetes, among which consumer-oriented HITs (CHITs) have gained interest with the promise of providing more effective self-care support. CHITs refer to consumer-centered electronic tools, technologies, applications, or systems that are interacted with directly by health consumers (ie, individuals who seek or receive health care services) to provide them with data, information, recommendations, or services for promotion of health and health care.^3–7

The advantages of using CHITs have been well documented.^8–10 For example, CHITs can provide patients with prompt personalized health records, enhance the accessibility of health care resources, promote communication and relationships between clinicians and patients, and overcome geographical barriers and logistical inconvenience when seeking health care services.^9,10 In addition, CHITs have the potential to engage patients as active participants in their health care process, and are therefore likely to empower them to gain more control of their care.^3,11,12

Although the benefits of CHITs in diabetes management are clear in theory, adopting the technologies has proven difficult and rate of use has been limited,^7,10 largely because their effectiveness relative to usual care is yet to be fully determined. Previous review studies have reported mixed effects of CHITs on patient outcomes in terms of the magnitude and significance of effect sizes.^13–21 On the one hand, the mixed effects might be a result of the heterogeneity of the technologies examined, as most of the previous studies examined CHITs together with other types of HITs (eg, nurse- or physician-led HITs).^13–19 On the other hand, the inconsistent results may in part be due to differences in duration of intervention for which the technologies are implemented. In fact, recent meta-analysis studies found that HITs used for patient diabetes management showed different efficacy in glycemic control with different intervention durations.^17,21 It is possible that patients develop different usage patterns for a technology as time progresses (eg, frequent use due to curiosity and expectation at early stages vs less frequent use due to increased fatigue and technology resistance at later stages). Thus, the influence that a technology has on health outcomes is likely to vary over time. In addition, it is expected that CHITs produce persistent effects over time in order for them to provide continuous self-care support for diabetic patients, as self-care is a lifelong task for most patients. Therefore, it is important to understand how the effects of CHITs evolve over time. However, no previous review studies have made such an endeavor.

While previous reviews examined a diverse range of HITs, including CHITs, the present study aimed to systematically review and assess the current state of evidence concerning the effects of CHITs only on patient outcomes, particularly on glycemic control in diabetes management, and to examine how the effects evolve over time. Glycemic control, indexed by glycated hemoglobin (HbA1c), was chosen as the main indicator in this study, as it is a widely recognized indicator of treatment effectiveness in diabetes management.²

Methods

The review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines,²² and the Cochrane Handbook for Systematic Reviews of Interventions.²³

Literature search and study selection

A literature search was conducted using MEDLINE and Academic Search Complete via EBSCOhost Research Databases, Cochrane Central Register of Controlled Trials via Cochrane Library, and Health and Medical Complete and PsycINFO via ProQuest for studies published from the inception of the databases to July 5, 2016. The search strategy included combinations of 4 sets of terms for technology (Web* or Internet or computer* or telemonitor* or telemedic* or e-health or cellular phone or mobile phone or technolog*), disease management (glucose monitor* or glycemic control or self monitor* or self care or self manag* or disease manag* or management), diabetes (diabetes or diabetic*), and randomization (random*). We intentionally used broad search terms, including both keywords and some associated controlled vocabularies, to reduce the chance of missing relevant studies (See Supplementary Appendix 1 for the detailed search strategy for the 5 databases). The titles and abstracts of the citations identified in the initial search were first screened to determine their relevance. The full texts of the potentially relevant studies were further reviewed for final inclusion. The reference lists of the included studies and several relevant review studies^13–21 were also manually searched to catch any missed articles.

Inclusion and exclusion criteria

Studies were included if they: (1) were original randomized controlled trials (RCTs), (2) focused on using CHITs directly by patients or other consumers for self-care of diabetes, (3) examined the effects of a CHIT on HbA1c changes and other patient outcomes in comparison to a control group without the technology, (4) were published in peer-reviewed journals, and (5) were written in English. We excluded studies that compared multiple technologies and that examined technologies for provision of health care by medical professionals.

Data extraction

A coding scheme was preconstructed to guide the data extraction. Data were independently extracted by 2 authors (DT and TYW). The information extracted included the characteristics of the study and quantitative data on HbA1c, and the statistical significance on other patient outcomes at all available time points reported in each study. We split the control group for multiple-arm studies and used first-stage data for crossover studies, as suggested by the Cochrane Collaboration Guidelines.²³ DT and TYW independently assessed the studies in all stages of the study selection and data extraction. Any discrepancies were resolved through discussion and consensus.

Data analysis

Meta-analysis was done to assess the evidence for the effects of CHITs on HbA1c. An overall meta-analysis was conducted by aggregating data on HbA1c changes using data reported at the latest time point in each trial, as was done in previous reviews.^16–21 We then pooled together data reported at the same time point and presented meta-analysis results for each time point examined. The standardized mean difference (SMD) in HbA1c between the intervention and control groups from baseline to a specific time point was calculated to represent the effect size for each trial at the time point. Random-effects models were used to pool the effect sizes, as the trials were assumed to be a sample of trials that could possibly be conducted in this area, unless a low level of heterogeneity was detected, where fixed-effects models were applied.²⁴ The I² statistic was used to determine heterogeneity, which measures the extent of inconsistency among the results of trials, with I² values of 25%, 50%, and 75% representing low, moderate, and high levels of heterogeneity, respectively.²⁵ The possibility of publication bias was examined using Egger’s regression test, with P < 0.05 considered to be significant for publication bias.²⁶ We also performed sensitivity analyses to examine whether the effects of CHITs on HbA1c remained unaltered, by excluding trials with a sample size smaller than the median value (77) of all included trials, with low study quality (2 studies^27,28), with a sample attrition rate >20% (4 studies^29–32), and with extreme outcomes (SMD > 1; 4 studies^33–36). However, sensitivity analyses were performed only on the overall meta-analysis and on analyses at the time points 3, 4, 6, and 9 months, as there were insufficient trials to allow for sensitivity analyses at other time points.

Patient outcomes other than HbA1c were summarized using a narrative synthesis approach.^37,38 The outcomes were grouped into 4 categories (clinical outcomes, psychosocial outcomes, behavioral outcomes, and knowledge outcomes), as proposed in a previous study.²⁰ Similarly, we pooled together data reported at the same time point and presented narrative synthesis results for each time point examined.

Study quality

The methodological quality of the included studies was assessed using the widely recognized 12-item criteria list proposed in the Updated Method Guidelines for Systematic Reviews.³⁹ A study that met 6 or more of the 12 criteria was considered high quality; otherwise, the quality was considered low.³⁹

Results

Figure 1 illustrates the literature search and study selection process. Eighty eligible studies were identified from 2394 initial citations and the manual search.^{27–36,40–109} We extracted 2 trials from studies by Forjuoh et al.,²⁹ Franklin et al.,⁵⁸ and Glasgow et al.,³⁰ and 3 from studies by McKay et al.⁸² and Quinn et al.⁸⁹ Therefore, a total of 87 individual trials were included for analysis. Table 1 summarizes the characteristics of the 87 trials (See Supplementary Appendix 2 for detailed information on these trials).

Figure 1.

Study search and selection procedures.

Table 1.

Characteristics of the 87 trials analyzed

Characteristics	n (%)
Year of publication
Before 2000	4 (5)
2000–2005	14 (16)
2006–2011	38 (44)
After 2011	31 (36)
Region where trials were conducted
North America	42 (48)
Europe	18 (21)
Asia	26 (30)
Oceania	1 (1)
Intervention duration
≤6 months	55 (63)
7–12 months	29 (33)
>12 months	3 (3)
Disease type examined
Type 1 diabetes	18 (21)
Type 2 diabetes	45 (52)
Both types 1 and 2 diabetes	13 (15)
Not specified	11 (13)
Type of CHITs involved
Computer-based system	38 (44)
Mobile phone–based system	27 (31)
Other	22 (25)
Environment in which technology was used
Home	53 (61)
Non-home	9 (10)
No location restriction	25 (29)

Study characteristics

The number of patients in individual trials ranged from 14 to 1382, with a median of 77. The median of mean age reported in the trials was 53.2 years (range 8.5–70.8 years). The median intervention duration was 6 months (range 2–60 months). The majority of the technologies (44%) were computer-based systems that provided comprehensive support for diabetes management, including self-monitoring of blood glucose, patient monitoring, insulin and medication management, diet management, and other self-management services. Some (31%) were mobile phone–based systems mainly designed to provide educational materials and messaging systems for care and communication through short messaging. The technologies involved in the rest of the trials (25%) were mainly telemonitoring systems developed to collect and transfer patients’ vital signs and diabetes-related symptoms. Most of the technologies evaluated (61%) were used in patients’ homes; some (29%) had no location restriction, as they utilized mobile phone–based systems; and the rest (10%) were implemented in primary care offices, clinics, or other non-home health care environments.

Quality assessment

The results of the quality assessment of the 80 studies are presented in Table 2. The quality assessment scores ranged from 5 to 11, with a mean (SD) of 7.5 (1.3). Seventy-eight studies were rated as high quality, while the other 2 had low study quality.^27,28 The most frequently identified potential sources of bias concerned absence of blinding patients/care providers/outcome assessors to the intervention, intention-to-treat analysis, and treatment allocation concealment.

Table 2.

Quality assessment of the 80 included studies

No. of criteria that were met		Criteria met	No. of studies
Low-quality trials			2
Five criteria met		f, g, h, k, l	2
High-quality trials			78
Six criteria met		c, f, g, h, k, l	15
		a, c, f, g, k, l	1
		c, f, g, i, k, l	1
Seven criteria met		a, c, f, g, h, k, l	18
		c, f, g, h, i, k, l	7
		a, c, f, g, i, k, l	3
		c, d, f, g, h, k, l	1
		c, f, g, h, j, k, l	1
Eight criteria met		a, c, f, g, h, i, k, l	6
		a, c, f, g, i, j, k, l	1
		a, b, c, f, g, h, k, l	3
		a, b, c, f, g, i, k, l	1
Nine criteria met		a, c, f, g, h, i, j, k, l	1
		a, c, e, f, g, h, i, k, l	1
		a, b, c, f, g, h, i, k, l	10
		a, b, c, f, g, h, j, k, l	1
		a, c, f, g, h, i, j, k, l	1
Ten criteria met		a, b, c, f, g, h, i, j, k, l	5
Eleven criteria met		a, b, c, e, f, g, h, i, j, k, l	1

The 12-item criteria list used for quality assessment was as follows: (a) adequate randomization method, (b) treatment allocation concealment, (c) similarity of groups at baseline, (d) patients blinded to the intervention, (e) care providers blinded to the intervention, (f) co-interventions avoided or similar for all groups, (g) acceptable compliance in all groups, (h) acceptable dropout rate, (i) intention-to-treat analysis, (j) outcome assessor blinded to the intervention, (k) similar timing of outcome assessment for all groups, and (l) selective outcome reporting.

Meta-analysis of HbA1c

The results of meta-analyses are presented in Table 3. In general, in the pooled analysis of 11 820 patients, use of CHITs resulted in a significant reduction in HbA1c compared to usual care (SMD = −0.31%, 95% confidence interval [CI]: −0.38 to −0.23, P < .001) (see Figure 2 ). There was large heterogeneity (I²= 72) across the trials. Egger’s regression test indicated the presence of publication bias (P = .003).

Table 3.

Meta-analysis results for HbA1c over time

Intervention duration (months)	No. of trials	No. of patients	SMD (95% CI)	P-value	I2	Egger’s regression testa
						t	P-value
2	3	572	−0.07 (−0.23 to 0.10)	.424	0	10.12	.063
3	35	3157	−0.40 (−0.55 to −0.25)	<.001	74	1.80	.081
4	7	1171	−0.15 (−0.44 to 0.15)	.334	80	1.76	.139
5	3	318	−0.21 (−0.66 to 0.25)	.377	69	0.23	.850
6	33	4133	−0.36 (−0.50 to −0.23)	<.001	75	1.91	.065
8	2	231	−0.30 (−0.56 to −0.04)	.022	0
9	8	529	−0.61 (−1.05 to −0.17)	.007	83	3.21	.018
12	23	4822	−0.22 (−0.35 to −0.09)	.001	72	2.75	.012
15	1	71	−0.51 (−0.99 to −0.04)	.033	0
24	1	1231	−0.04 (−0.15 to 0.07)	.436	0
30	1	71	−0.70 (−1.17 to −0.22)	.004	0
36	2	1027	−0.04 (−0.17 to 0.08)	.484	0
48	1	930	−0.10 (−0.23 to 0.03)	.133	0
60	1	727	−0.17 (−0.32 to −0.03)	.020	0
Overallb	79	11820	−0.31 (−0.38 to −0.23)	<.001	72	3.05	.003

Note: SMD, standardized mean difference; CI, confidence interval.

^aEgger’s regression test was performed only when there were 3 or more trials.

^bOverall meta-analysis pooled data from the latest time points for the trials.

Figure 2.

Overall meta-analysis results of the effects of the use of CHITs on reduction of HbA1c.

Data on 14 time points were synthesized, and the results are shown in Figure 3 . Specifically, compared with usual care, use of CHITs led to significant reductions in HbA1c at 3, 6, 8, 9, 12, 15, 30, and 60 months. No significant differences were found at any other time points. Heterogeneity was large for trials examined at 3, 4, 5, 6, 9, and 12 months. Egger’s regression tests indicated the presence of publication bias for results at 9 (P = .018) and 12 months (P = .012). We also found that the pooled effect size at each time point was closely correlated with the average sample size for the trials examined at the time point (Pearson r = −0.617, P = .019).

Figure 3.

Effects of the use of CHITs on reduction of HbA1c over time.

All the meta-analysis results remained unchanged in the sensitivity analyses, except that the positive effects of CHITs at 12 months turned nonsignificant when we excluded trials with small sample sizes (SMD = −0.04, 95% CI: −0.11 to −0.03, P = .260).

Narrative synthesis of patient outcomes

The results of narrative synthesis of patient outcomes over time are summarized in Table 4 (See Supplementary Appendix 3 for detailed information on patient outcomes examined in each trial). In total, patient outcomes were assessed 480 times across different time points. For 99 times (21%), use of CHITs was significantly associated with improved outcomes (positive effect) compared to usual care, while for the remaining 381 times (79%), use of CHITs was found to have no effect on patient outcomes. For clinical outcomes, all trials that assessed waist circumference, lipid ratio, creatinine concentration, and the incidence of diabetic ketoacidosis showed no effects of CHITs; a small portion of trials that examined blood glucose level, blood pressure level, total cholesterol level, and the incidence of hyperglycemia and hypoglycemia demonstrated positive effects. For psychosocial outcomes, all trials that assessed diabetes distress, patient attitudes toward disease care, and patient satisfaction with care received found no effect on outcomes with use of CHITs. A minority of trials that tested patient self-efficacy, perceived health status, social support, and patient activation found positive effects. Findings on the majority of behavioral and knowledge outcomes, such as patients’ physical activity levels, diet and eating behaviors, medication adjustment behaviors, and adherence to self-care behaviors, were mixed. No psychosocial, behavioral, or knowledge outcomes were examined in trials with a study duration longer than 12 months.

Table 4.

Narrative synthesis results for patient outcomes over time

Intervention duration (months)	No. of trials	Patient outcomes
		Clinical outcomes		Psychosocial outcomes		Behavioral outcomes		Knowledge outcomes
		Positive effect	No effect	Positive effect	No effect	Positive effect	No effect	Positive effect	No effect
2	2	1	4		3		2	1
3	31	11	103	1	13	3	13	1	3
4	4		11		7	3
5	2	1	1	1	2	1
6	28	21	71	5	13	5	12	1	3
8	1	2	2
9	6	4	8	2	4	1	3	1
12	23	9	46	4	27	12	7		3
15	1		8
24	1	2
30	1	1	7
36	2	2	4
48	1	2
60	1	1	1

Note: HbA1c was not included in the narrative synthesis. The data indicate cumulative numbers of patient outcomes for the effects.

Positive effect: significant improvement in the intervention group compared with the control group.

No effect: no statistically significant difference between the intervention and control groups.

Usage of CHITs

Fourteen trials reported data related to patients’ use of CHITs.^{30,32,43,52,55,61,81,82,86–88,96,101,104} Among them, 9 trials^{32,43,52,55,61,81,82,96,101} found a large portion of noncompliant patients (ie, patients who used the technologies less frequently than recommended by care providers), ranging from 24%¹⁰¹ to 61%.³² Five trials reported that patients’ use of technologies declined over time.^{30,43,61,88,104} Two trials found that use of the technologies was associated with a significant reduction in HbA1c.^86,87 Another 2 found a larger improvement in glycemic control for compliant patients compared with noncompliant patients.^43,52

DISCUSSION

While an increasing number of CHITs are available for consumer utilization, their wider application is largely inhibited by a lack of evidence on their (particularly persistent) effectiveness in improving patient outcomes. As such, this study reviewed existing RCTs that assessed the effects of CHITs in diabetes management and examined the effectiveness of the technologies over time. In general, we found evidence of improvement in glycemic control among patients who used CHITs for diabetes care compared with patients receiving usual care. Although the improvement holds true in the majority of intervention durations examined, the conclusion is tempered by our findings of nonsignificant improvement for some other intervention durations. In addition, our study found mixed evidence for the effects of CHITs on other clinical, psychosocial, behavioral, and knowledge outcomes.

Effects of CHITs on patient outcomes

To date, several reviews related to CHITs have been published.^15–21 These reviews addressed the use of telemedicine,^16–18 mobile phones,¹⁹ web-based applications,¹⁵ or self-management HITs^20,21 in diabetes management. In general, the reviews suggested that those technologies have the potential to improve glycemic control and some other patient outcomes. However, firm conclusions on the effectiveness of CHITs could not be drawn from these reviews, because they did not limit their reviews to CHITs (other types of HITs that are mandatorily used by care providers in their work environment were also included).^16–19 They also did not provide evidence on how the effects of CHITs evolved over time. Unlike previous reviews, the present study focused on CHITs only and examined their effectiveness over time.

In addition, existing studies have focused on comparing the use of technologies with usual care regardless of the difference in intervention duration.^16–21 This has therefore limited our understanding of how these technologies exert influence on patient outcomes over time. Unlike previous reviews,^16–21 we argued for the importance of reporting the effects of CHITs by intervention duration, and proposed to synthesize outcome data obtained from the same intervention duration and presenting the results in a time-series manner. Therefore, findings from this study are unique in that they allow a more accurate understanding of the effectiveness of CHITs over time.

In particular, we found that, compared to usual care, use of CHITs led to a significant improvement in glycemic control at 3, 6, 8, 9, 12, 15, 30, and 60 months, while no significant difference was found for other intervention durations, such as 2, 4, 5, 24, 36, and 48 months. This finding seems to indicate that CHITs might not be equally effective for all intervention durations. Reasons for the fluctuation in effectiveness over time could be complex. First, it is worth noting that the trials were disproportionately distributed among the intervention durations examined. Results reported for some intervention durations could be inconclusive because of a lack of sufficient trials to be pooled. Second, the finding that effect size and sample size were negatively correlated seems to suggest the widely known “size trial effect” (ie, small sample trials are more likely to detect larger effects and are therefore more likely to be published).¹¹⁰ This might overestimate effect sizes for intervention durations with a large number of small sample trials. In addition, the usage patterns of CHITs could also vary at different stages of their implementation process and as patients’ health status changes. Variations in usage patterns may lead to fluctuations in effectiveness. For example, it has been reported that use of CHITs declined over time,^{30,43,61,88,104} and the decrease in usage was associated with less improvement in glycemic control.^43,52,86,87 Finally, as proposed by health care work-system theories (eg, the Systems Engineering Initiative for Patient Safety model),^111,112 there are various elements in the work system where a CHIT is employed (eg, patient characteristics, technology features, self-care activities supported by the technology, the environment and context where the technology is used, and health care organizations involved). These elements are dynamically and interactively connected with one another throughout the implementation process. It is speculated that the roles the elements play can change, and therefore they may exert varying influence on the effects of the technology over time. However, this speculation awaits confirmation by future studies.

While much work has been done in short-term evaluation of CHITs, evidence available in the literature for their long-term assessment was limited. For instance, we only identified 3 trials with an intervention duration >2 years. Data are also lacking on long-term assessment of psychosocial, behavioral, and knowledge outcomes, all of which are important health indicators in diabetes management. Therefore, one needs to be cautious when claiming the effectiveness of CHITs before more research efforts are made on long-term evaluation of the technology.

Implications and future directions

This review raises many issues central to the adoption, evaluation, and design of CHITs in diabetes management. First, while the adoption of CHITs has become one of the most widely supported and advocated initiatives in health care,¹¹³ our findings showed that their effectiveness did not remain consistent for all intervention durations. This suggests that it might be premature to promote adoption of the technology by claiming its effectiveness in improving outcomes in patients’ care of diabetes. The findings also call for more clinical trials to be conducted so that firm conclusions can be made with sufficient evidence.

Long-term assessment of this technology is urgently needed. For a valid assessment, researchers need to improve patient compliance with the technological intervention, as compliance with technology usage is associated with improvements in patient outcomes. Otherwise, the true effects of the technology might not be detected due to patients’ misuse or nonuse. To ensure that maximum data are available to inform future research, long-term RCTs are needed to report data at more time points.

In addition, the findings of this study should be used to motivate further primary research to explore factors that could moderate the effects of the technology. Although a number of moderators, such as duration and type of diabetes, age, and baseline HbA1c level, have been examined in previous studies,^19,21 they were not investigated with the consideration of intervention duration. In addition, the elements in the work system where the technology is implemented have not been well studied. These elements are believed to be key to successful application of the technology.^111,114 Therefore, systematic scrutiny of the roles of these elements during the implementation process will likely facilitate our understanding of the fluctuation in the effects of the technology, and help with better design of CHIT-based health care interventions so as to maintain effectiveness in disease care.

Strengths and limitations

The present study has several strengths. To the best of our knowledge, this was the first to synthesize the effects of CHITs on patient outcomes in a time-series manner, methodologically different from previous reviews.^{16,17,19–21} Our results not only reflected more precisely the effects of CHITs on patient outcomes, but also provided evidence on how the effects evolve over time. In addition, our review was limited to RCTs only, most of which had low risk of bias. This guaranteed the quality of the data sources that this study was based on.

The limitations of this study should also be noted. First, as a recurrent problem in meta-analysis, the publication bias detected in our study suggests that caution should be exercised when interpreting the findings. Second, there was moderate to high heterogeneity among the trials in the meta-analyses. This may indicate the existence of potential moderators other than intervention duration, such as modes of information technology interventions. However, the limited descriptions of the CHITs and their implementation process in the majority of included studies prevented us from assessing moderating effects of these factors. Finally, to be simple, we treated patient outcomes equally in the narrative synthesis by only counting their statistical significance. However, they might in fact have different clinical and practical importance in patients’ disease management and deserve additional consideration.

CONCLUSIONS

This study represents the first meta-analysis of the effects of the use of CHITs in diabetes management over time. It demonstrates that, overall, use of CHITs appears to be more effective than usual care in improving glycemic control; however, the effectiveness did not remain consistent over time and in other patient outcomes. Further efforts are required to examine long-term effects of CHITs and to explore factors that could moderate the effects over time.