Process evaluation of Dulce Digital-Me: an adaptive mobile health (mHealth) intervention for underserved Hispanics with diabetes

Abstract

Type 2 diabetes disproportionately impacts ethnic minorities and individuals from low socioeconomic status. Diabetes self-management education and support has been shown to improve clinical outcomes in these populations, and mobile health (mHealth) interventions can reduce barriers to access. Dulce Digital-Me (DD-Me) was developed to integrate adaptive mHealth technologies to enhance self-management and reduce disparities in the high-risk, underserved Hispanic population. The objective of the present study was to evaluate reach, adoption, and implementation of an mHealth diabetes self-management education and support intervention in this underrepresented population. The present analysis is a multimethod process evaluation using the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. The study was effective in reaching a sample that was representative of the intended population; only modest but significant differences were observed in sex and age. The DD-Me health coach (HC) cited several important facilitators of intervention adoption, including outreach frequency and personalization, and the automated HC report. Implementation fidelity was high, with participants receiving >90% of intended interventions. Participants who received DD-Me with support from a HC were most engaged, suggesting utility and acceptability of integrating HCs with mHealth interventions. Perceptions of implementation among study participants were positive and consistent across study arms. This evaluation revealed the target population was successfully reached and engaged in the digital health interventions, which was implemented with high fidelity. Further studies should evaluate the efficacy and maintenance of the study following the RE-AIM model to determine whether this intervention warrants expansion to additional settings and populations.

A process evaluation of the Dulce Digital-Me mobile health intervention study revealed successful reach, adoption, and implementation, suggesting potential for future dissemination and implementation.

Implications.

Practice: For practitioners, this research suggests that a tailored, adaptive diabetes self-management intervention can be effective in reaching underserved, high-risk individuals.

Policy: For policy makers, this research adds to the important body of work focused on engaging those underrepresented in research and mitigating barriers that have historically precluded these populations from participating in research.

Research: For researchers, this research indicates that the Dulce Digital-Me RCT was effective in reaching the target underrepresented population and engaging them in the intervention.

INTRODUCTION

Nearly one-third of U.S. adults are projected to have diabetes by 2050 [1], and certain minority populations, including Hispanics [2, 3] and individuals from low socioeconomic status, are disproportionately affected [4, 5]. Good glycemic control and management of risk factors can prevent complications of type 2 diabetes (T2D) and improve long-term survival [6–9]. However, Hispanic individuals tend to show less diabetes self-management behaviors (e.g., physical activity; healthy eating), poorer glycemic control, and worse outcomes relative to non-Hispanic White individuals [10–12].

Diabetes self-management education and support (DSME/S) can improve glycemic control and other important diabetes outcomes [13–16]. For many individuals at highest risk for suboptimal outcomes, practical barriers limit access to DSME/S. Mobile health (mHealth) technology has been widely adopted to mitigate many of these barriers [17–23]. The Dulce Digital (DD) intervention, which included transmission of educational and motivational text messages combined with remote glucose monitoring by a care-team nurse, resulted in improved glycemic control across 6 months compared with usual care [24]. While DD was both feasible and acceptable [25], participants expressed preference for intervention content tailored to their individual needs and progress, as opposed to a static, “one-size-fits-all” approach. Despite evidence that mHealth interventions improve T2D outcomes [26–28], there has been little consideration about patient and provider needs in integrating these technologies in underserved populations or with existing healthcare practices [27, 29]. Further, there is a paucity of mHealth interventions that utilize adaptive components (e.g., personalized feedback) [29–31].

The Dulce Digital-Me (DD-Me) intervention [32] was developed to address these gaps by integrating adaptive mHealth technologies to personalize and extend the reach of DSME/S to help reduce disparities in a high-risk, underserved, Hispanic population. Between 2016 and 2021, a large RCT was conducted to compare the adaptive DD-Me intervention with the original DD intervention among N = 310 patients at a Southern California Federally Qualified Health Center (FQHC) serving a low income, predominantly Hispanic population. The DD-Me intervention included the original DD educational text messages and remote glucose monitoring plus the addition of real-time, personalized behavior change strategies (e.g., feedback, goal setting) to target self-management mechanisms that underlie clinical control of diabetes (e.g., medication adherence, blood glucose monitoring, physical activity, diet, stress management). The DD-Me tailored feedback and goal setting was implemented via one of two modalities (automated, algorithm-driven text messaging or health coaching telephone calls) to allow for the direct comparison of these unique feedback delivery methods. Outcome analyses evaluating the effectiveness of the intervention in improving clinical control, patient-reported outcomes, and cost effectiveness are currently underway.

Prior to interpreting the clinical outcomes of the present study, we aim to understand whether the study was successful in reaching this historically medically underserved population—at-risk Hispanic individuals seeking care at an FQHC—in digital health intervention research, and whether our interventions effectively engaged participants and were delivered with high fidelity. To achieve this aim, we conducted a multimethod process evaluation of the DD-Me trial. We applied the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) model, which has been used to evaluate behavioral interventions for chronic disease, to evaluate the potential for dissemination, future implementation, and translation of the research [20–23, 33–35]. Here, we use components of the RE-AIM framework to analyze reach, adoption, and implementation in the DD-Me study.

MATERIALS AND METHODS

The study was approved by the Scripps Health and San Diego State University IRBs and all participants provided written informed consent. Details regarding approvals, methods for cohort retention and intervention development, and implementation protocols, including interventionist training for the DD-Me trial were described in Philis-Tsimikas et al. [32]. Briefly, this was a randomized, controlled, parallel groups, comparative effectiveness trial with target N = 414 participants. Due to the COVID-19 pandemic, recruitment for this study stopped early with total enrollment being N = 310 participants. The early conclusion was approved by the funder (NIH) and the participating IRBs and was deemed to preserve statistical power to test the study Aims given our a priori attrition estimate. Eligible participants were Hispanic adults (≥18 years), registered patients of Neighborhood Healthcare, with T2D and at least one of the following within 45 days of enrollment: glycated hemoglobin (HbA1c) ≥8.0% and/or systolic blood pressure (SBP) ≥160 mm Hg and/or low-density lipoprotein cholesterol ≥100 mg/dL. After completing a baseline assessment, enrolled participants were randomized to one of three groups: DD, Dulce Digital-Me Automated (DD-Me-Automated), or Dulce Digital-Me Telephonic Health Coach (DD-Me-Telephonic-HC).

Participants in the DD group received culturally and health literacy-appropriate, DSME/S text messages spanning five “Core Content” domains (healthy eating, physical activity, psychological well-being, medications, and clinical indicators) in their preferred language—either English or Spanish. Participants were encouraged to check glucose using the cellular-enabled blood glucose meter (Telcare, Bethesda, MD), manage their oral medication(s) using the cellular-enabled pill box (WisePill, Somerset West, South Africa), and respond to brief self-report ecological momentary assessments (EMAs) assessing their health behaviors and emotional well-being. For each data source, if no data were received for 2 weeks, or if blood glucose reached critical values (see Philis-Tsimikas et al. [32]), an alert prompted staff to call the patient, as needed.

In addition to components described for the DD group, participants in the DD-Me-Automated group were able to tailor the order of the core content messages to their preference and received real-time, algorithm-driven feedback/goal-setting text messages tailored to their EMA responses and weekly summary feedback messages on their blood glucose control and medication adherence based on data received.

The DD-Me-Telephonic-HC group received the adaptive feedback and goal setting during weekly phone calls from a Health Coach (HC) instead of via automated messaging. The HC training program is described in detail in Philis-Tsimikas et al. [32]. This study was served primarily by one dedicated, bilingual Hispanic HC, who had diabetes herself, and conducted the calls in English or Spanish depending on the participant’s preference. Two additional bilingual HCs were trained and able to serve this role, as needed. To inform feedback and goal-setting calls, the HC utilized an automated Health Coach Report that provided real-time summaries of participants’ progress based on EMA response, and objective glucose and medication adherence data transmitted. The HC was expected to discuss medication adherence and blood glucose monitoring at every weekly call; often, at least one core content domain was also discussed. mHealth intervention delivery, technology/device integration, and Health Coach Report production was achieved through CYCORE (CYberinfrastructure to support COmparative effectiveness REsearch) [36].

Current study: process evaluation

Data sources and analyses

All data were stored using REDCap [37, 38] databases and CYCORE (see Philis-Tsimikas et al. [32]) and analyses were conducted using R v.4.0.3 [39]. Specific metrics and analytic approaches are described below.

The RE-AIM framework

Reach

Screening, recruitment, eligibility, and enrollment data were descriptively analyzed. To determine whether study participants were representative of the target population, chi-square tests and independent sample t-tests compared eligible individuals who elected to enroll versus not enroll on age, sex, preferred language, and recent clinical values (HbA1c, LDL, and SBP).

Adoption

A semistructured, poststudy interview was conducted with the HC to assess their willingness and ability to deliver the personalized feedback and goal-setting components to the DD-Me-Telephonic-HC group (only). Interview questions addressed their experience conducting personalized feedback calls, and their use and perceived utility of the Health Coach Report. Responses to interview questions were qualitatively summarized upon review of transcripts for perceived facilitators and barriers to adoption.

Implementation

Intervention fidelity was evaluated by comparing actual versus intended delivery of intervention components and the consistency of delivery of core/common elements across study groups. Fidelity statistics were calculated only for actionable alerts that warranted follow-up. To avoid outreach fatigue/burden, criteria were established to define an alert as actionable or not. For multiple, consecutive alerts of the same type, study staff performed outreach for the first 2 consecutive alerts, but tapered outreach to once/month and then twice/month. If consecutive alerts continued, alert outreach was discontinued. For the DD-Me-Telephonic-HC group, protocol adherence was descriptively summarized for call completion rates and call content. Participant engagement was assessed via the following metrics: EMA completion rates and frequency of “no data” alerts for the medication adherence box and glucose meter. Participants’ perceptions of implementation were assessed via a 12-item survey developed by the study team (Supplementary Table S2) at either month 6 (n = 127) or month 12 (n = 54). The total survey score was calculated as a sum of responses to questions that were asked to all participants, with higher scores indicating a more positive response (max score = 32). Participants’ perceptions of implementation were also captured qualitatively through key informant interviews following study completion for a random convenience sample of participants who recently completed the study proximal to the timing of the interviews (n = 18).

Intervention content delivery and response rates were compared between the three groups by chi-square tests or one-way ANOVA. If significant main effects were observed, pairwise post hoc tests were conducted with Holm correction to account for multiple comparisons. The frequencies of “no data” alerts were skewed and were analyzed by Kruskal–Wallis tests, with post hoc adjusted Mann–Whitney tests applied where appropriate.

RESULTS

Reach

Between October 2017 and March 2020, N = 571 patients at Neighborhood Healthcare were identified, screened, and deemed eligible for enrollment into the study. Of these, N = 310 (54%) enrolled. Among the N = 261 (46%) who were eligible but did not participate, common reasons included time conflicts and unsuccessful rescheduling of their baseline visit (Fig. 1 and Supplementary Table S1).

Fig 1.

CONSORT diagram depicting recruitment, screening, and enrollment.

Individuals who enrolled (N = 310) did not differ significantly from those who did not in primary language or baseline HbA1c, LDL, or SBP (ps > .10); however, patients who enrolled were more likely to be female (p < .001) and younger (p < .05; Table 1).

Table 1.

Reach: baseline demographics and clinical characteristics by enrollment status

	Not enrolled	Enrolled
	N = 261	N = 310
	n (%)	n (%)	p
Sex (male)	118 (45.2%)	96 (31.0%)	.001
Language preference (Spanish)	229 (87.7%)	282 (91.0%)	.264

	Mean (SD)	Mean (SD)	p
Age (years)	53.9 (12.5)	52 (10.2)	.045
A1c (%) (N: 257, 309)	9.8 (1.8)	9.7 (1.9)	.458
SBP (mm Hg) (N: 235, 277)	127.5 (18.9)	127.2 (20.3)	.871
LDL-C (mg/dL) (N: 101, 141)	95 (40)	102.6 (42.4)	.155

A1c glycated hemoglobin A1c; LDL-C low-density lipoprotein cholesterol; SBP systolic blood pressure.

Regarding retention, 92% (284/310 enrolled) remained engaged at 6 months by completing either follow-up surveys or study laboratories, and 90% (280/310) remained engaged at 12 months.

To examine the representativeness of the participants of our current study within our local population, we examined the demographics of individuals in publicly available data on chronic diseases (via Health & Human Services 2019 Public Health Services Data [40]) in our county and observed that while the age of eligible individuals for the present study (M = 52) was within the range of those most commonly hospitalized in our county for diabetes (approx. 39% are age 45–64), the San Diego population hospitalized with diabetes is more frequently male (roughly 59%), whereas both eligible and enrolled participants for this study were mostly (55% and 69%, respectively) female.

Adoption

The HC shared that she felt providing education and support during phone contact with patients was the most important facilitator of intervention adoption. The HC felt the weekly frequency of the personalized feedback facilitated the delivery of the intervention, explaining, “sometimes they have questions, and they don’t know what to do or where to go, especially newly diagnosed patients.” The weekly coaching calls offered an opportunity for patients to have these questions answered, allowing for swift adjustment of self-care behavior. She thought the content covered during the calls was comprehensive, and that the Health Coach Report was helpful in preparing her personalized feedback and noted the helpfulness of the forms that she filled out prior to each call. She noted that often the Health Coach Report would reveal problematic areas and found it helpful to start calls by asking the patient which of the identified areas they would like to focus on for the call, thereby following a motivational interviewing technique. She felt patients would benefit from an intervention that also included formal psychosocial/emotional support elements, as stress and depression were common barriers. Other barriers identified during remote monitoring calls included patients’ resistance to making behavioral changes.

Implementation

Intervention fidelity analyses were conducted with N = 302 participants who completed the 6-month active intervention period. Participants received an average of 243.8 (SD = 16.1), or 96% of the intended 254 total core content messages over the entire study. As intended, there were no group differences in the number of core content messages participants received overall, or by content domain (ps > 0.08). On average, participants received 75.4 (SD = 11.6), or 105% of the intended 72 total EMA prompts over 24 weeks; ­consistent with protocol, no differences were observed between groups (p > .8; Table 2).

Table 2.

Implementation: intervention content delivery and receipt

	Overall (N = 302)	DD (N = 103)	DD-Me-Automated (N = 103)	DD-Me-Telephonic-HC (N = 96)
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	p
EMA responses
Total responses	38 (23.6)	36.3 (25.5)	34.7 (22.6)	43.5 (21.8)	.020a
Total responses/week	1.6 (1.0)	1.5 (1.1)	1.4 (0.9)	1.8 (0.9)	.020a
Healthy eating/week	0.6 (0.3)	0.5 (0.4)	0.5 (0.3)	0.6 (0.3)	.013a
Physical activity/week	0.5 (0.3)	0.5 (0.4)	0.5 (0.3)	0.6 (0.3)	.035a
Well-being/week	0.5 (0.3)	0.5 (0.4)	0.5 (0.3)	0.6 (0.3)	.024a
Core content delivered
Total messages	243.8 (16.1)	245.1 (4.4)	241.6 (24.4)	244.9 (12.2)	.224
Total messages/week	10.2 (0.7)	10.2 (0.2)	10.1 (1.0)	10.2 (0.5)	.224
Healthy eating/week	2.1 (0.2)	2.1 (0.1)	2.1 (0.2)	2.1 (0.1)	.071
Physical activity/week	2 (0.1)	2 (0.1)	2 (0.2)	2 (0.1)	.272
Well-being/week	1.8 (0.1)	1.8 (0.0)	1.7 (0.2)	1.8 (0.1)	.300
Clinical indicators/week	2.1 (0.2)	2.1 (0.1)	2.1 (0.2)	2.1 (0.1)	.244
Medications/week	2.2 (0.1)	2.2 (0.0)	2.2 (0.2)	2.2 (0.1)	.427
EMA questions delivered
Total messages	75.4 (11.6)	75.5 (9.1)	75.0 (15.7)	75.8 (8.3)	.892
Total messages/week	3.1 (0.5)	3.1 (0.4)	3.1 (0.7)	3.2 (0.3)	.892
Healthy eating/week	1.1 (0.2)	1.1 (0.1)	1 (0.2)	1.1 (0.1)	.810
Physical activity/week	1 (0.1)	1 (0.1)	1 (0.2)	1 (0.1)	.710
Well-being/week	1.1 (0.2)	1.1 (0.2)	1.1 (0.3)	1.1 (0.2)	.857
EMA response rate
Responses/delivered	50.7% (31.9%)	48.1% (34.4%)	46.9% (31.0%)	57.4% (29.1%)	.040

DD Dulce Digital; DD-Me-Automated Dulce Digital-Me Automated; DD-Me-Telephonic-HC Dulce Digital-Me Telephonic Health Coach; EMA ecological momentary assessment.

^aSignificant post hoc differences between DD-Me-Telephonic-HC and DD-Me-Automated.

Alert outreach was also designed to be delivered equally across groups. Study outreach attempts occurred for 89% of all triggered alerts, or a median of 100% of all triggered alerts (IQR 83.3%–100%) and were consistent across groups (p = .7). Among attempted calls, 52% made successful contact with the participant; there were no differences between groups in the successful contacts rate relative to the number of alerts they received (Med: 60%, IQR: 33%–100%, p = .5; Table 3).

Table 3.

Implementation: triggered alerts and outreach calls per participant over the total 24-week active intervention period

		Per participant
		Overall (N = 302)	DD (N = 103)	DD-Me-Automated (N = 103)	DD-Me-Telephonic-HC (N = 96)
Alert: engagement		Median (IQR)	Median (IQR)	Median (IQR)	Median (IQR)	p
Total # of alerts triggered		9 (3–20.75)	15 (6–22)	8 (2.5–20)	6.5 (2–13.5)	.002a
Alert type
# of no data transmitted		6 (2–17.75)	13 (4–21)	6 (2–19)	3 (1–11)	<.001b
# of no blood glucose transmitted		3 (0–10)	6 (1–13)	2 (1–10.5)	1 (0–4)	<.001c
# of no EMA responses transmitted		1 (0–5)	1 (0–7)	1 (0–5)	0 (0–2)	.062
# of no pill box openings transmitted		1 (0–3)	2 (0–6)	0 (0–3)	0 (0–2)	.010b

Alertd: fidelity	Overall %	Median (IQR)	Median (IQR)	Median (IQR)	Median (IQR)	p
% of alerts where contact was attempted	89%	100 (83.3–100)	100 (83.3–100)	100 (84.5–100)	100 (83.3–100)	.712
% of alerts where patient was reached	52%	60 (33.3–100)	63.3 (33.3–100)	50 (31.0–90)	60 (20–87.5)	.496

DD Dulce Digital; DD-Me-Automated Dulce Digital-Me Automated; DD-Me-Telephonic-HC Dulce Digital-Me Telephonic Health Coach; EMA ecological momentary assessment; IQR interquartile range.

^aSignificant post hoc differences between DD and both other groups.

^bSignificant post hoc differences between DD and DD-Me-Telephonic-HC.

^cSignificant post hoc differences between DD-Me-Telephonic-HC and both other groups.

^dAlert fidelity was calculated for actionable alerts only.

In the DD-Me-Telephonic-HC group, the HC attempted 96% of expected feedback calls total, or a median of 24 (IQR 22–24) feedback calls per participant over the study period, confirming intervention fidelity given the intended weekly frequency over the 24-week period. The HC was successful in reaching participants in 81% of attempted calls and provided feedback frequently on each: medication adherence (98%), blood glucose checking (99%), and blood glucose results (95%), and less frequently on healthy eating, physical activity, and well-being (70%, 64%, and 66%, respectively; Table 4).

Table 4.

Implementation: Health Coach feedback call completion and content coverage (DD-Me-Telephonic-HC group only, N = 96)

	Overall %	Per participant median (IQR)
Call completion rates
Total contact attempts/participant	96	24 (22–24)
% successful attempts/participant	81	92% (71%–100%)
Call content coverage
% calls discussed medication adherence	98	100% (100%–100%)
% calls discussed blood glucose checks	99	100% (100%–100%)
% calls discussed blood glucose results	95	100% (93%–100%)
% calls discussed healthy eating	70	78% (50%–92%)
% calls discussed physical activity	64	69% (43%–86%)
% calls discussed well-being	66	69% (41%–86%)

DD-Me-Telephonic-HC Dulce Digital-Me Telephonic Health Coach; IQR interquartile range.

In terms of participant engagement, total responses to EMA prompts and percent of prompt responses were more frequent in DD-Me-Telephonic-HC than DD-Me-Automated (p = .02 and p = .04, respectively; Table 2). No differences were found between DD and either DD-Me groups.

Participants in the DD group had increased total alerts compared with both other groups (p = .002). Alerts ­triggered for no data transmission (p < .001) and for no pill box openings (p = .010) were higher among DD compared with DD-Me-Telephonic-HC participants. “No data” alerts for blood glucose value transmission were lowest among DD-Me-Telephonic-HC (p < .001; Table 3).

Participants who completed the survey assessing perceptions of implementation had high satisfaction scores (M = 28.4, SD = 3.8), with no group differences (p = .3). Most participants reported that they read the text messages (81%) and liked receiving the calls/text messages (86%). A majority (93%) reported they thought the intervention helped them manage their diabetes, and 99% of participants said they would recommend the intervention to friends or family with diabetes. Participants in the DD-Me-Telephonic-HC group reported more consistently carrying their cell phone (p = .015) but also more frequent confusion about messages (p = .019). Additional responses are summarized for each group in Supplementary Table S2.

Key informant interviews conducted with n = 18 study participants revealed that all felt their expectations were met, they learned something new about diabetes management, and they would enroll in the program again or continue if given the option. Most (83%) who said they would continue elaborated that they would choose to do so for the help, encouragement, and/or motivation it provided. Participants all had positive perceptions of the text message content, learned something new about their diabetes from the text messages, were all able to use their blood glucose monitors, and did not find any aspect of the calls about blood glucose values to be burdensome or unhelpful. Among the n = 7 participants interviewed in the DD-Me-Telephonic-HC group, all had positive perception of the calls from the HC, did not find any aspect of the calls to be burdensome or unhelpful, and learned to better care for their health and/or diabetes because of the calls. Most (89%) participants disclosed no aspects that they found unhelpful; however, one participant reported they never learned how to respond to the text messages properly. Most participants reported no areas for improvement for the program, aspects they liked the least, or suggestions to better the program in the future (78%, 83%, and 89%, respectively). Additional responses and themes are summarized in Supplementary Table S3.

DISCUSSION

This report sought to examine the study processes of the DD-Me trial through the lens of the RE-AIM framework. The aims were to determine whether the trial reached the desired study population, whether adoption of the intervention was acceptable to the HC, and whether the implementation was successful from a protocol fidelity and patient engagement standpoint. Taken together, these components assess the feasibility of the parent study to inform the potential to adopt and maintain this program moving forward. In this underrepresented study population, ensuring adequate reach and engagement, as well as intervention fidelity and acceptability, are paramount to understanding the potential impact of the trial—independent of clinical findings. While clinical effectiveness is the desired primary outcome of this trial, the lessons learned regarding the underlying processes of this intervention are critical for understanding the context of findings and informing future efforts to evaluate, implement, and disseminate digital health interventions for DSME/S within the Hispanic community.

This study included Hispanic individuals at an FQHC at high risk for poor diabetes outcomes, including existing poor glycemic, blood pressure, and/or lipid level management. While San Diego County is comprised of over 30% Hispanic individuals, these individuals accounted for over 40% of all diabetes-related hospitalizations in 2019 [40]. Using publicly available data on patients hospitalized with diabetes in our region to assess representativeness of our target population [40], we observed that mean age of eligible patients in this study was similar to the county records for those hospitalized with diabetes; however females were most commonly eligible (and enrolled) in the present study, while the county saw more males hospitalized for diabetes. While our study is in an outpatient setting, this is an important observation when considering generalizability of our sample and more work is needed to increase recruitment rates of male participants in diabetes research. However, higher rates of study participation among women are consistent with reported discrepancies in research participation with higher participation in preventive interventions by women [41]. Those who enrolled were about 2 years younger than those who did not, which is also consistent with the known barriers for older adults engaging mHealth-based interventions [42]; however while older, nonenrollers were generally not elderly (mean age = 52). Education and socioeconomic status, which may be important factors influencing enrollment (especially given the minimum literacy requirements for reading and responding to text messages) were not examined in the present study and should be important considerations regarding generalizability.

During recruitment in this study, 67% of those screened were eligible for the study and over half of those eligible were successfully enrolled. An enrollment rate of 54% is within the expected range for pragmatic trials [35]. The key reasons for nonenrollment of eligible participants were time conflicts and unsuccessful reschedules, which aligns with the known barriers for engaging high-risk populations in diabetes self-management [17, 18].

The interview conducted with the HC assessed her experience with facilitators and barriers of implementing DD-Me. The HC felt comfortable and confident providing personalized calls as part of the DD-Me intervention. Importantly, the five “core content” domains targeted in the feedback calls and educational text messages were viewed by the HC as comprehensive, relevant, and helpful. The HC identified several facilitators to successful implementation, including the use of motivational interviewing techniques early in the calls to identify the highest priority domains from the patient’s perspective. The HC used these techniques in combination with the Health Coach Report to guide the specific recommendations and feedback. A prior study using a HC emphasized the importance of finding a coach who is a good fit for the role [43]. In our study, the primary HC was not only highly open to learning new techniques and collaborated well with other professionals, but she was also of the same cultural/ethnic group as the participants, spoke the same language, and had diabetes herself. This allowed her to share a connection with patients and provide them with support based on her own experiences with diabetes while immersed in Hispanic culture. The HC’s patient-centered approach highlights the potential benefits lost from fully automating this feedback and omitting the personalized feedback calls. While results from the key informant interview revealed the HC role was well accepted, insights were offered into potential areas for improvement including incorporating some elements to address remaining practical barriers to participation as well as psychosocial concerns, perhaps by including a meeting with or access to a social worker for assistance in these areas. Further the HC noted that some participants found it challenging to engage in behavioral changes, indicating the need to investigate whether these difficulties could be mitigated by addressing underlying psychosocial concerns including frequent reports of anxiety and depression.

Intervention fidelity, represented by actual versus planned core content and EMA messaging, was consistent across study groups and within domains. The slight reduction in message delivery was likely related to participants’ cellular coverage. This is further supported by the lack of significant differences in delivery between study groups. Nearly all actionable study alerts prompted an outreach call from a study HC. The number of alert actions, number of call attempts, and successful participant contact were consistent across study arms. Together, these findings suggest high fidelity of implementation of the interventions.

The DD-Me-Telephonic-HC group was most engaged with responding to EMA prompts and had higher responses overall and across each domain compared with the DD-Me-Automated group. The DD-Me-Automated group had the lowest EMA response rates, perhaps due to message fatigue in this all-technology-based intervention arm. Overall, alerts were most triggered in the DD group and were mostly due to lack of data transmission. No data transmission was less prevalent in the DD-Me-Telephonic-HC group, particularly for blood glucose value transmission and pill box openings. Together, these findings corroborate prior evidence that telephonic coaching can enhance self-management support [44–46], and also provide novel evidence for the utility of a HC in an integrated mHealth intervention. Perhaps with a predominantly technology-based intervention, the human connection of contact with an HC provided encouragement and accountability leading to higher engagement.

Participant perceptions of implementation reflected their opinions of the intervention and adherence to the study goals. The overall reception to the programs was positive—most participants reported that they liked receiving calls/text messages, that messages were not a hassle, and that they would recommend the programs to friends or family with diabetes. Participants in the DD-Me-Automated group reported carrying their cell phone less, while participants in the DD-Me-Telephonic-HC group more frequently reported finding messages “confusing.” These findings support the observations that there may have been technology fatigue among those in the DD-Me-Automated group. Given the reported confusion with text messages, additional support from the HC may have ultimately facilitated better understanding and engagement since no other satisfaction metrics differed between groups. The overall positive impressions in the key informant interviews supported the survey findings with participants describing the interventions as “useful,” “helpful,” and “motivational.” Collectively, participants had positive perceptions of the study, reporting that they were comfortable using the technology provided and felt they learned how to better manage their diabetes.

The detailed findings highlight the trial’s success in reaching and engaging an often-underrepresented population in digital health intervention research—Hispanic individuals with T2D receiving care at an FQHC. This intervention was implemented with high fidelity and mitigated many barriers to accessing diabetes self-management education through the successful use of mHealth technology including core content text messaging for educational and motivational reminders, EMA via text message, cellular-enabled blood glucose ­monitoring and medication adherence tracking. The enhanced engagement of the individuals receiving supplemental support by a HC who was Hispanic, bilingual, and able to connect personally with our participants given her own diabetes diagnosis, highlights the relevance and impact cultural competency can have in augmenting a digital intervention approach. Together, these findings can inform the potential for dissemination and future implementation of the interventions in the DD-Me trial.

While our reach was aligned with our target population, a limitation of the current study was that it recruited participants who were diagnosed with diabetes and already engaged with the FQHC. Participants were identified in reports based on recent lab draws, so participants who had not engaged with the FQHC or with no recent blood work were not included. This population may be at potentially higher risk than those included in the study.

Since this study was ongoing through March 2020, study operations were impacted by the COVID-19 pandemic. As mentioned in the methods, study recruitment was halted as a precaution for the high-risk patients included in our study and to comply with the California COVID-19 stay-at-home orders. For already-enrolled participants, follow-up survey data collection was completed over phone calls with study staff rather than in-person while blood draws for laboratories were still completed at the clinic, and all enrolled participants had the opportunity to complete the intervention. Additionally, COVID-19 wellness surveys were conducted by telephone with nearly all participants still enrolled to gauge the impact of the pandemic and offer referrals/resources as needed. While we have not observed trends toward reduced engagement among individuals completing follow-up visits after the onset of the pandemic (data not shown), self-management behaviors and perception of the trial may have been directly or indirectly impacted and warrants further study.

CONCLUSIONS

The DD-Me adaptive mHealth intervention was successful in recruiting and enrolling at-risk Hispanic individuals at an FQHC who were deemed likely to benefit from improved diabetes self-management. Enrolled participants had similar diabetes risk profiles to those who were eligible but did not enroll in the study, suggesting the population was representative of the eligible population of interest. The interventions went according to the study protocols, with no differences in intervention delivery frequency between groups. Participant engagement was highest among those who received the personalized health coaching delivered by telephone, supporting the utility of this role integrating with an mHealth intervention. The HC interview supported these findings, reporting key facilitators and limited barriers. Participants reported positive perceptions of the implementation of the study through both a satisfaction survey and key informant interview. If the evaluation of effectiveness of this study shows improved outcomes, this program could be widely adopted and maintained to improve diabetes self-management and long-term outcomes.

Funding

Research reported in this publication was supported by the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under Award Number R01DK112322 (A.P.-T. and L.C.G.). Additional support was obtained from the National Center for Advancing Translational Sciences of the NIH (5 U54 TR002550 [A.P.-T. and L.C.G.]), and the National Institute of Diabetes and Digestive and Kidney Disorders of the NIH (5 P30 DK111022 [A.L.F. and L.C.G.]).

Compliance with Ethical Standards

Conflict of Interest: None declared.

Ethical Approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent: Informed consent was obtained from all individual participants included in the study

Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.

Transparency Statements: (i) Study registration: this study was registered with ClinicalTrials.Gov as # NCT03130699. (ii) Analytic plan preregistration: the analysis plan was published in Philis-Tsimikas et al. at https://pubmed.ncbi.nlm.nih.gov/35090520/. (iii) Analytic code availability: analytic code used to conduct the analyses presented in this study are not available in a public archive. They may be available by emailing the corresponding author. (iv) Materials availability: materials used to conduct the study are described in Philis-Tsimikas et al. at https://pubmed.ncbi.nlm.nih.gov/35090520/.

Data Availability

Deidentified data from this study are not available in an a public archive. Deidentified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author.