Abstract
Objective: This article examines engagement with a mobile application (“mCare”) for wounded Service Members rehabilitating in their communities. Many had behavioral health problems, Traumatic Brain Injury (TBI), and/or post-traumatic stress disorder (PTS). The article also examines associations between Service Members’ background characteristics and their engagement with mCare.
Materials and Methods: This analysis included participants who received mCare ( n = 95) in a randomized controlled trial. mCare participants received status questionnaires daily for up to 36 weeks. Participant engagement encompasses exposure to mCare, percentage of questionnaires responded to, and response time. Participants were grouped by health status—that is, presence/absence of behavioral health problems, PTS, and/or TBI. Histograms and regression analyses examined engagement by participants’ health status and background characteristics.
Results: Exposure to mCare did not differ by health status. Participants usually responded to ≥60% of the questionnaires weekly, generally in ≤10 h; however, participants with behavioral health problems had several weeks with <50% response and the longest response times. Total questionnaires responded to and response time did not differ statistically by health status. Older age and higher General Well-Being Schedule scores were associated with greater and faster response.
Discussion: The sustained response to the questionnaires suggests engagement. Overall level of response surpassed trends reported for American’s usage of mobile applications. With a few exceptions, Service Members engaged with mCare irrespective of health status.
Conclusion: Mobile health has the potential to increase the quantity and quality of patient-provider communications in a community-based, rehabilitation care setting, above that of standard care.
Keywords: case management, traumatic brain injury, post-traumatic stress disorder, mobile health, patient engagement
BACKGROUND AND SIGNIFICANCE
In 2003, the US Army established specialized units, and at that time they were called Community Based Warrior in Transition Units (CBTWUs), to provide outpatient case management services for wounded Active Duty, National Guard, and Reserve Service Members who were eligible for rehabilitation in their home communities. Each CBWTU was staffed with a care team comprising a physician, physician’s assistant or nurse practitioner, a social worker, nurse case managers, Platoon Sergeants, and administrative personnel who managed and coordinated the Service Members’ care. Upon assignment to the CBWTU, Service Members would come in for one face-to-face onsite visit for processing and evaluation by the physician. While onsite, the Service Members would meet with all of the care team members and be assigned to their home community to rehabilitate and receive care within a 50-mile radius. Once home, as part of standard care, the CBWTU case managers contacted rehabilitating Service Members weekly, via telephone, to schedule appointments with medical specialists, monitor appointment attendance, assess progress toward therapy goals, and provide emotional support. Also as part of standard care, Platoon Sergeants contacted the Service Members daily, by telephone, to monitor and address any logistical obstacles the Service Member might have encountered and consulted with other Community Based Warrior in Transition Unit (CBWTU) staff members as needed.
Although this standard care approach worked well to support rehabilitation, weekly telephone contacts with a case manager and daily telephone contact with a Platoon Sergeant may not have been adequate for some of Service Members assigned to the CBWTUS. In some cases, Platoon Sergeants managed caseloads upward of 50 Service Members and case managers managed caseloads of up to 30 Service Members. Moreover, increasing numbers of Service Members were in need of rehabilitation for unseen injuries, namely traumatic brain injury (TBI), post-traumatic stress disorder (PTS), and/or behavioral health problems. These problems may be short-term, long-term, or permanent. Furthermore, they can result in significant functional impairments throughout their duration that can limit a person’s ability to perform regular daily activities and can impede the initiation of self-care. Given the CBWTU contacts were telephonic, quantity of contacts from CBWTU staff may have been too little (i.e., fragmented), and the quality of the information exchange too challenging to cover all the issues telephonically for these Service Members faced, and the Service Members were not reaching rehabilitation goals.
In response to the needs of Service Members in the CBTWUs, particularly those with behavioral health problems, PTS, or TBI, the Telemedicine and Advanced Technology Research Center (TATRC), of the US Army Medical Research and Material Command, proposed to develop a secure mobile phone application, called “mCare,” to augment the standard care provided through the CBTWUs by sending secure mobile messages between Service Members and their CBWTU care team. Congruent with several key elements of Lequerica and colleagues’ (2010) 1 comprehensive Model for Therapeutic Engagement in Rehabilitation , mCare was based on the ideas that a) patient engagement is “active” (as opposed to passive), “effortful” (i.e., requiring effort), directed participation (as opposed to compliance) in the rehabilitation process and b) the treatment provider and his/her collaboration with the patient is essential to increasing patients’ engagement. These ideas were largely supported by previous research, which had shown that patient participation (if not “patient engagement” per the aforementioned model) was related to improvements in patient outcomes, 2–6 and enhanced self-care behaviors. 7 Moreover, mobile phones were beginning to be seen as a means to increase patient participation in their healthcare, 8,9 and interventions using mobile phones had been shown to improve a variety of clinical outcomes for people with chronic health problems 10–14 and facilitate preventive healthcare in a variety of populations. 15,16
In 2011–2012, TATRC developed and tested mCare, a bidirectional system that includes text messages to send health tips, appointment reminders, and notices of new content within the application, as well as daily questionnaires to the Service Member about his/her status in a variety of areas important to their rehabilitation. Case Managers, Social Workers, or the Platoon Sergeant (as appropriate) responded to the results of the questionnaires if the Service Member’s answers indicated an intervention was needed. mCare was intended to increase the quantity of contacts Service Member–provider interaction by a minimum of six above standard care, improve the quality of contacts by targeting issues important to rehabilitation, and expand the modality of contacts from scheduled telephone conversations only to convenient, asynchronous exchanges.
OBJECTIVE
The clinical efficacy of mCare vs standard care was tested in a randomized controlled trial (RCT) at four CBWTUs for Service Members (also called “study participants” in this article), many of whom had behavioral health problems, TBI, and/or PTS. As part of that RCT, the study team tracked participant usage of all aspects of the application. This article examines study participants’ engagement with mCare, operationalized as their exposure and response to the most intensive, effort-requiring part of it, the daily questionnaires. Additionally, this article addresses whether exposure and response to mCare differed for study participants with behavioral health problems, PTS, and/or TBI in comparison to those who did not have these problems, and reports associations between response to the application and study participants’ demographic and social characteristics.
MATERIALS AND METHODS
mCare Technology
The mCare application is a secure, bi-directional, mobile health system that facilitates contact between Service Members and their care team members, particularly their Case Managers and Platoon Sergeants. The mCare application employs two-factor authentication and leverages the National Institutes of Standard and Technology Federal Information Processing Standard 140-2 certified encryption for data at rest and in-transit. mCare is remotely activated on a mobile device with a one-time authentication code and is then routinely accessed by the user/patient using a six-digit personal identification number of their choosing. The entire system functions in both synchronously and asynchronously. The system’s secure website tracks data from the mobile devices and generates escalation triggers (as needed). It also provides longitudinal tracking of questionnaire responses received from the user/patient. The system uses text message prompts to notify users when there is new information to view and respond to within the mCare application.
mCare Study Protocol
To conduct the mCare prospective RCT, a convenience sample of Service Members assigned to four CBWTUs were recruited to participate voluntarily. Case Managers informed eligible, potential participants when they first presented to the CBWTU, and provided them with an informational brochure. Each CBWTU had an assigned registered nurse (a project liaison) who was not a member of the CBWTU and who went through the consent process with potential participants who were interested in enrolling.
Once enrolled, participants were randomized to “standard care plus mCare” or “standard care alone” groups. For group assignment, a sequential randomized block design was used based on whether the participant had confirmed TBI, suspected TBI, or no TBI (three blocks) at the time of study enrollment. The study used this approach to balance TBI symptoms between the two main treatment groups. One hundred and eighty-two patients enrolled in the evaluation, 87 in the standard care group, and 95 in the group that received the mCare intervention in addition to standard care. The treatment groups did not differ statistically on any of the background characteristics assessed ( Supplementary Appendix 1 ).
Once enrolled, the study participants were followed for up to 36 weeks. However, participation in the study could be shorter. The reason for this is that the purpose of the care management was to assist in rehabilitation and determine if participants would return to duty, retire, or discharge from active service. Each of these scenarios (i.e., returned to duty, retired, etc.) meant the participant ‘out processed’ from the CBWTU and they were no longer eligible to receive care, including mCare. All others continued to receive mCare throughout the intervention.
In addition to the standard care described in the background section above, the mCare group received and responded to messages, mCare tips, appointment reminders, administrative notifications, and mCare questionnaires. Case Managers, Platoon Sergeants, other members of the CBWTU staff, and Project Liaisons utilized a website dashboard interface to send messages and review participants’ responses. The mCare system recorded and time-stamped each instance a communication was sent to a participant, when the communication synced with the participant’s phone, and if/when the participant responded.
This analysis evaluated study participants’ engagement with them mCare questionnaires, the most interactive part of the mCare application. Each day, a study questionnaire was sent to mCare study participants at 10 a.m. local time, utilizing the mCare mobile application on the participants’ mobile phones. Participants’ responses to the questionnaires were stored and organized on a secure website, giving the Case Managers information regarding their patient’s self-perceptions of his/her functioning. There were seven questionnaires, each sent once per week: General Status, Pain Status, Energy and Sleep Status, Anger Management, Relationship Status, Transition Goal Status, and Mood Status. Also, mCare sent a Weight Status questionnaire once per month. Each questionnaire was composed of a logic tree of 2–5 brief questions with multiple choice answers. If a participant’s response to a questionnaire indicated they were having an issue, mCare automatically sent an alert—via email—to the members of the care team, which then led to a follow-up contact. The possibility that participants would provide rote answers to these questionnaires was mitigated by rotating the days on which a given status questionnaire was sent and by using at least two versions of each questionnaire.
Both treatment groups completed surveys related to the outcomes of the main study at regular intervals over 36 weeks. Study staff administered the surveys through Survey Monkey®. The results for these surveys are beyond the scope of the present analysis.
Study Participants
This analysis focuses on the 95 study participants randomized to receive the mCare intervention.
Measures
Outcome variables
The outcome of the present analysis is participant engagement, defined as exposure and response to the most intensive part of the protocol, the mCare questionnaires. As noted above, participants might have left the study if they out processed from the CBWTU. Therefore, we considered exposure to the mCare system as an outcome variable, representing one dimension of participant engagement with it. Exposure refers to the number of weeks a participant was enrolled in the study (making them eligible for the intervention), during which mCare content was successfully sent to and synced with the participant’s phone (range = 1–36 weeks).
Because the mCare system automatically collects the date and time of when a questionnaire was sent, when it synced with a participant’s mobile phone, and when the participant responded, the study was able to construct two measures of response. The first measure is a percentage, obtained by first dividing the number of questionnaires each participant responded to each week by the number of questionnaires sent and successfully synced with the participant’s mobile phone. The result was multiplied by 100 (range = 0–100%). If a questionnaire did not sync with the participant’s phone, it was not counted in the denominator. The second measure is the number of hours between when a questionnaire synced with a participant’s phone and when he/she sent a response.
Independent variable
The key independent variable is the presence or absence of a behavioral health problem, PTS, and/or TBI. The study created the following mutually exclusive groups: a) no behavioral health problem (or PTS or TBI) ( n = 40; 42.1%); b) behavioral health problem(s) only (unspecified) ( n = 22; 23.2%); c) behavioral health problem(s) (unspecified) plus PTS and/or TBI ( n = 24; 25.3%); and d) PTS and/or TBI only ( n = 9; 9.5%). TBI is not a behavioral health diagnosis per se , and it shares similar symptomatology with PTS, so we included PTS and TBI together.
Potential Correlates
The analysis also included age (years, ≥18), gender, CBWTU site [in Alabama (AL), Virginia (VA), Illinois (IL), and Florida (FL)], living arrangement (alone, with a spouse or partner, or “other,” meaning with friends or family), total number of combat deployments in the participant’s military career, length of deployments (combat and noncombat days), mobile phone carrier (AT&T, Sprint Verizon, T-Mobile), operating system of phone (Android, iPhone, other), and general well-being. The study measured general well-being using the General Well-being Schedule (GWS), an 18-item self-report questionnaire developed by the National Center for Health Statistics designed to measure an individual’s sense of well-being (affect, self-control, health, vitality). 17 Total scores range from 0 to 110. 17 The instrument has strong internal consistency for the total score (α = 0.91) 18 and acceptable test-retest reliability (coefficients ranging from 0.68 to 0.85). 19
Statistical Analyses
The statistical analyses first calculated means and standard deviations or percentages for the mCare study participants’ background characteristics, stratified by the independent variable, which groups participants according to whether they had a) no behavioral health diagnoses, b) behavioral health problem(s) only, c) behavioral health problem(s) plus PTS and/or TBI, or d) PTS and/or TBI only. The analyses then tested for differences across the groups using t -tests or Fisher’s Exact tests. Next, the analyses constructed a series of histograms showing the outcome measures for each week of the intervention. Each group, as defined by the independent variable, is shown separately. These graphs display potential heterogeneity among groups that could be lost in analyses of totals (e.g., percentage of questionnaires responded to for the full 36 weeks, etc.). Last, using Ordinary Least Squares regression and Poisson regression, the analyses tested associations between totals for the outcome variables and the independent variable as well as between totals for the outcome variables and participants’ background characteristics.
RESULTS
At baseline, the average age of mCare participants was between 34.7 and 39.7 years ( Table 1 ). The majority of participants were male, married, and living with a spouse or partner. A higher proportion of participants with PTS and/or TBI were divorced (33.3%), but marital status was not statistically different across the groups defined by the independent variable ( P = .141). Most participants (50.5%) had Android phones or iPhones (35.8%) and AT&T (37.9%) or Verizon (33.3%) for service.
Table 1:
Baseline characteristics of the mCare study participants, by presence or absence of behavioral health problem, PTS, or TBI ( n = 95).
| Variable | No Behavioral Health roblem, n = 40 | Behavioral Health Problem(s) Only, n = 22 | Behavioral Health Problem Plus PTS and/or TBI, n = 24 | PTS and/or TBI Only, n = 9 | P -value |
|---|---|---|---|---|---|
| Age at in-processing, years, mean (SD) | 39.7 (10.4) | 34.7 (10.3) | 37.7 (10.2) | 38.7 (12.3) | .359 |
| Gender, n (%) | |||||
| Male | 33 (82.5) | 16 (72.7) | 22 (91.7) | 9 (100.0) | .204 |
| Female | 7 (17.5) | 6 (27.3) | 2 (8.3) | 0 (0) | |
| Marital Status, n (%) | |||||
| Single | 11 (27.5) | 9 (40.9) | 10 (41.7) | 1 (11.1) | .141 |
| Married | 27 (67.5) | 12 (54.6) | 13 (54.2) | 5 (55.6) | |
| Divorced | 2 (5.0) | 1 (4.6) | 1 (4.2) | 3 (33.3) | |
| Living arrangement, n (%) | |||||
| Alone | 4 (10.0) | 4 (18.2) | 4 (16.7) | 1 (11.1) | .787 |
| With spouse/partner | 26 (65.0) | 11 (50.0) | 16 (66.7) | 5 (55.6) | |
| Other | 10 (25.0) | 7 (31.8) | 4 (16.7) | 3 (33.3) | |
| CBTWU Site, n (%) | |||||
| AL | 10 (25.0) | 5 (22.7) | 6 (25.0) | 6 (66.7) | .100 |
| IL | 9 (22.5) | 9 (40.9) | 9 (37.5) | 1 (11.1) | |
| FL | 18 (45.0) | 4 (18.2) | 5 (20.8) | 2 (22.2) | |
| VA | 3 (7.5) | 4 (18.2) | 4 (16.7) | 0 (0.0) | |
| Number of combat deployments, n (%) | |||||
| 0 | 4 (10.0) | 1 (4.6) | 0 (0.0) | 0 (0.0) | .037 |
| 1 | 21 (52.5) | 8 (36.4) | 6 (25.0) | 5 (55.6) | |
| 2 | 10 (25.0) | 11 (50.0) | 9 (37.5) | 1 (11.1) | |
| 3+ | 5 (12.5) | 2 (9.1) | 9 (37.5) | 3 (33.3) | |
| Days deployed, mean (SD) | 255.1 (133.1) | 271.3 (110.0) | 329.3 (98.3) | 308.0 (40.7) | .079 |
| General Well-being Schedule, mean (SD) | 83.1 (17.1) | 63.7 (20.5) | 50.4 (15.7) | 81.8 (15.8) | <.0001 |
| Mobile phone operating system, n (%) | |||||
| Android | 21 (52.5) | 12 (54.6) | 10 (41.7) | 5 (55.6) | .963 |
| iPhone | 13 (32.5) | 7 (31.8) | 11 (45.8) | 3 (33.3) | |
| Other | 6 (15.0) | 3 (13.6) | 3 (12.5) | 1 (11.1) | |
| Mobile phone service provider, n (%) | |||||
| AT&T | 14 (35.0) | 9 (40.9) | 10 (41.7) | 3 (33.3) | .937 |
| Sprint | 11 (27.5) | 6 (27.3) | 4 (16.7) | 1 (11.1) | |
| T-Mobile | 2 (5.0) | 1 (4.6) | 2 (8.3) | 0 (0.0) | |
| Verizon | 13 (32.5) | 6 (27.3) | 8 (33.3) | 5 (55.6) |
Notes: SD refers to standard deviation. % refers to column percentages. Column percentages might not sum to 100% due to rounding. P -values are from F -tests produced by ANOVA (means) or Fisher's Exact Tests (percentages). P values that are underlined are statistically significant.
With respect to group differences, participants who did not have any behavioral health problems, PTS, or TBI had fewer combat deployments ( P = .037). Average scores for the GWS were higher—indicating higher self-reported general well-being—for participants with no behavioral health problems and participants with PTS and/or TBI only ( P < .0001).
Figure 1 illustrates mCare participants’ exposure to the intervention. The sub-title of each sub-panel of Figure 1 shows the number of participants in each group at baseline. The bars of the histograms represent the number of participants enrolled and who successfully synced the questionnaires on their cell phones each week. Syncing problems occurred only in the first weeks of the study for three participants who did not have any behavioral health problems and at weeks 5 and 6 for one participant who had a behavioral health problem only. Due to outprocessing, the number of mCare participants eligible to receive the intervention declined over time. For example, 11/40 participants with no behavioral health diagnoses received mCare as of 36 weeks (top left sub-panel). Twelve of 22 participants with a behavioral health problem only (bottom left sub-panel), 8/24 with a behavioral health problem plus PTS and/or TBI (top right sub-panel), and 4/9 with PTS and/or TBI (bottom right sub-panel) were eligible for mCare as of 36 weeks, respectively.
Figure 1:
Number of participants who received questionnaires, by week and presence or absence of behavioral of health problem, PTS, and/or TBI.
Figure 2 shows the percentage of questionnaires responded to each week among participants eligible to receive them (note: only questionnaires that synced were included in the denominator). Participants with no behavioral health problem (top left panel) and participants with PTS and/or TBI only (bottom right panel) responded to ≥60% of the questionnaires 28/36 and 30/36 weeks, respectively. Participants with a behavioral health problem only (bottom left panel) and those with a behavioral health problem plus PTS and/or TBI (top right panel) responded to at least 60% of the questionnaires 25/36 and 15/36 week, respectively. Weeks 16, 17, 28, 29, and 30, participants who had a behavioral health problem plus PTS and/or TBI responded to <50% of the questionnaires. Three of the 36 weeks, participants who had a behavioral health problem only responded to 51% of the questionnaires. For participants in all groups who remained in the study at week 36, there was an increase in the percentage of questionnaires responded to.
Figure 2:
Mean percentage of questionnaires responded to, by week and presence or absence of behavioral of health problem, PTS, and/or TBI.
Participants who did not have a behavioral health diagnosis and participants with PTS and/or Traumatic Brain Injury (TBI) only had the shortest response times ( Figure 3 ). For example, people with PTS and/or TBI often responded in <1 h (Weeks 2, 5, 10, 11, 17, 19–21, etc.), and participants with no behavioral health diagnosis always responded in <12 h. However, participants with a behavioral health problem (with or without PTS and/or TBI) had multiple weeks in which their mean response time exceeded 12 h, and at week 36, the mean response time increased to 24 h.
Figure 3:
Mean hours to respond to questionnaires syncing, by week and presence or absence of behavioral of health problem, PTS, and/or TBI.
The total number of weeks participants were enrolled/eligible and received questionnaires, the total number of questionnaires responded to, and the mean response time for the entire intervention did not differ statistically across the groups defined by the independent variable ( Table 2 ).
Table 2:
Coefficients from regression analyses of the measures of response to and weeks enrolled in mCare, by individual study participant characteristics ( n = 95).
| Characteristic/Variable | Number of Weeks Enrolled/Eligible and Received Questionnaires* | Percentage of Total Messages Responded To | Hours to Respond after Syncing with Participants’ Phones | |||
|---|---|---|---|---|---|---|
| Coefficient | P -value | Coefficient | P -value | Coefficient | P -value | |
| Independent variable | ||||||
| No behavioral health problem | −0.06 | 0.42 | 2.30 | 0.84 | 0.65 | 0.78 |
| Behavioral health problem(s) only | 0.12 | 0.10 | −0.50 | 0.97 | 3.05 | 0.23 |
| Behavioral health problem plus PTS and/or TBI | 0.05 | 0.49 | 2.09 | 0.86 | 3.05 | 0.23 |
| PTS and/or TBI | Reference | Reference | Reference | |||
| Background variables | ||||||
| Age at in-processing | −0.00 | 0.77 | 0.66 | 0.02 | −0.05 | 0.40 |
| Male | −0.03 | 0.59 | 0.73 | 0.93 | −3.97 | 0.02 |
| Marital status | ||||||
| Single | Reference | Reference | Reference | |||
| Married | 0.03 | 0.43 | 9.84 | 0.14 | 0.84 | 0.55 |
| Divorced | 0.13 | 0.15 | 13.19 | 0.28 | 0.20 | 0.94 |
| Living arrangement | ||||||
| With spouse/partner | Reference | Reference | Reference | |||
| Alone | −0.05 | 0.42 | −9.70 | 0.29 | −1.90 | 0.31 |
| Other | −0.07 | 0.12 | −5.27 | 0.46 | −2.40 | 0.11 |
| CBTWU Site | ||||||
| AL | Reference | Reference | Reference | |||
| FL | −0.05 | 0.36 | 0.92 | 0.91 | −0.47 | 0.78 |
| IL | 0.22 | <0.0001 | 7.94 | 0.32 | 1.14 | 0.49 |
| VA | 0.19 | 0.005 | −4.46 | 0.67 | 3.07 | 0.17 |
| Number of combat deployments | ||||||
| 0 | Reference | Reference | Reference | |||
| 1 | 0.05 | 0.60 | 3.76 | 0.79 | −1.77 | 0.55 |
| 2 | 0.10 | 0.28 | 9.72 | 0.49 | −1.07 | 0.72 |
| 3+ | 0.09 | 0.35 | −4.75 | 0.75 | −1.65 | 0.60 |
| Days deployed | 0.00 | 0.58 | −0.02 | 0.41 | 0.00 | 0.71 |
| General Well-Being Schedule | −0.002 | 0.03 | 0.33 | 0.01 | −0.06 | 0.03 |
| Mobile phone operating system | ||||||
| Android | Reference | Reference | Reference | |||
| iPhone | 0.00 | 0.92 | 7.37 | 0.27 | −1.24 | 0.37 |
| Other | −0.25 | 0.0001 | −2.82 | 0.77 | −1.45 | 0.47 |
| Mobile phone service provider | ||||||
| AT&T | Reference | Reference | Reference | |||
| Sprint | −0.02 | 0.66 | −18.39 | 0.02 | 1.80 | 0.29 |
| T-Mobile | 0.06 | 0.52 | −2.97 | 0.83 | 0.56 | 0.85 |
| Verizon | −0.20 | <0.0001 | 0.24 | 0.97 | −0.88 | 0.56 |
Notes: The P -values are from OLS regression analyses for all measures of response percentage and response time.
The P -values for Number of Weeks Enrolled and Received Questionnaires are from Poisson regression analyses.
P -values that are underlined are statistically significant.
Each outcome was regressed on the individual study characteristics one at a time.
Several background variables were associated with the outcome totals. Specifically, the percentage of questionnaires responded to increased with age ( P = .02) and with higher GWS scores ( P = .01) ( Table 2 ). Compared with participants at the Alabama (AL) site, those at the Illinois (IL) ( P < .0001) and Virginia (VA) ( P = .005) sites were in the study slightly longer. Participants with higher GWS scores ( P = .03), mobile phone types “other” than Android or iPhone ( P = .0001), and with Verizon service ( P < .0001) tended to be enrolled in the study/eligible to receive questionnaires for less time. The number of hours to respond to questionnaires after receiving them was statistically related to gender and GWS score, with males responding significantly faster ( P = .02) and participants with higher GWS scores responding slightly faster ( P = .03).
DISCUSSION
mCare was developed to augment standard care. The goal was to increase the quantity of interactions between rehabilitating Service Members and their various providers at CBTWUs, and improve the quality of the information exchanged about health, psychosocial status, and progress toward treatment goals. In alignment with the Model for Therapeutic Engagement in Rehabilitation , 1 mCare was designed to require action and effort on the part of the Service Member, and to foster collaboration between him/her and staff at the CBTWU. All Service Members receiving care through the CBTWUs had some type of illness or injury, and therefore required long-term management as they progressed through the rehabilitation process and eventually transitioned into retirement, discharge or returned to duty. However, engagement with mCare among people with behavioral health problems, PTS, and/or TBI were particularly of interest in this analysis because these conditions can include debilitating cognitive (e.g., deficits in attention, concentration, memory, speed of processing, new learning, planning, reasoning, judgment, etc.) and emotional signs and symptoms (e.g., depression, anxiety, agitation, irritability, impulsivity, aggression) that make the monitoring and support provided by CBTWU staff simultaneously more difficult and more necessary. Little is known about whether people with such problems will adopt and use a mobile application in a sustained way.
Observation of the raw data by week showed that participants’ exposure to mCare declined systematically as they out processed from rehabilitation, irrespective of presence or absence of a behavioral health problem, PTS, or TBI. Also, this examination showed that, over the course of the study, the percentage of questionnaires that study participants responded to was at least 60%, except for participants with behavioral health problems plus PTS and/or TBI. Time to respond was generally ≤10 h, and some weeks, the average response time was <1 h. Again, mean response time was longer for participants with participants with behavioral health problems, with or without PTS and/or TBI. The regression analyses found that presence/absence of a behavioral health problem, PTS, and/or TBI was not statistically related to total exposure, total percentage of questionnaires responded to, or total mean response time. Altogether, these findings suggest that mCare can and will be adopted and used by Service Members in a community setting, even those with cognitive and emotional difficulties.
Observation of the relationship between the measures of engagement and participants’ background characteristics showed that the total number of questionnaires responded to over the study were statistically associated with higher age. That is, as age increased, so too did the percentage of questionnaires responded to. Higher GWS score, a measure of general well-being, was associated with higher and faster response.
To put the results about engagement into context, it is useful to compare mCare participants’ response to how people have responded to or used other mobile applications. On one hand, mCare participants’ engagement with the application was as expected, given recent market analyses. According to Localytics data (2013) the time people spend with mobile applications now exceeds the time they spend on traditional desktop web. 20 The Makovsky/Kelton fifth annual “Pulse of Online Health” survey found that 66% of Americans would like to use a mobile health application for their health-related problems. 21 On the other hand, mCare study participants’ level of engagement with the mCare application surpassed market trends. Localytics reported in 2013 that users typically open an application on their phones 11.5 times per month and use it for an average of 5.7 min. 20 Another Localytics analysis found that the percentage of people who opened their mobile applications only once in the first 6 months of having it was 20%. 22 Eleven percent opened their applications twice, 27% opened their applications 3–10 times, and 39% opened their mobile applications 11 or more times in the first 6 months of having it. 25 Even at a low of 60% response to the daily questionnaires, mCare study participants opened their application and used it 4.2 times per week for up to 36 weeks.
An unanticipated finding of this analysis is the notable increase in the percentage of questionnaires responded to at week 36. The increase occurred for all groups, suggesting a systematic, artificial inflation of engagement. The increase did not occur in response times, however; rather, the two groups with behavioral health problems had an increase in the length of time to respond and the PTS/TBI group responded faster. These patterns are consistent with the overall picture for the groups. Because the goal of the analyses was to show the actual pattern of response, permitting observation of heterogeneity that is often missed in typical analyses of central tendency, the figures included the last week of data despite the potential for inflation of engagement. But supplementary analyses in which week 36 was removed found that conclusions made with and without the week-36 data were the same.
Without additional data (not collected in the main mCare RCT), the reasons for this comparatively high level of response are unknown. The lack of information about causation is a limitation of this study. One reason might be that mCare participants had high perceived need for treatment and reasonable outcome expectancies, two additional elements of the Model of Therapeutic Engagement . 1 In the vein of perceived need for treatment, Simmond and Fleming note that, for people with brain injury, one’s awareness of their deficits is important for increasing motivation and improving functional outcomes. 23 Another reason for the high response level, which was sustained throughout the study, might be that participants received feedback and help if they answered the questionnaires in certain ways. This explanation is consistent with behavioral economics research, which points to timely feedback or reward as a powerful motivator. 24,25 With standard care, participants would have had to wait up to a week for feedback or help.
Also, without additional data, it is unknown why participants with higher GWS scores had greater engagement. Possibly because they seemed to have a lower burden of disease, as noted in the comparison of background characteristics across the four main groups in this study, this made it easier to engage. Or perhaps because they had a higher sense of optimism, which is has been shown to be related to greater engagement in the rehabilitation process. 26,27 When determining whether a patient is a candidate for a mobile health solution, it might behoove clinicians to evaluate the patient’s current well-being as a proxy for their ability to engage with it.
Another limitation of this study is that it recruited Service Members, who tend to be seen as “different” from the general American population and/or as a “captive audience,” making generalization difficult. Although mCare was developed for Service Members, the problems it was designed to mitigate are relevant to American civilians as well. Behavioral health problems, PTS, and TBI are major public health problems for the American general population. The Centers for Disease Control and Prevention reports that in 2010, about 2.5 million emergency department visits, hospitalizations, or deaths in the United States were associated with TBI. 28 Likewise, the National Comorbidity Survey has found that the lifetime prevalence of PTS among adult Americans is 7.8%. 29 Major depression, just one behavioral health problem, affected 6.9% of American adults in 2012. 30 Additionally, this cohort was being cared for in community settings, as much of the general American population is.
This analysis addressed the questionnaires, for reasons previously described, but mCare also provided education (health tips), administrative announcements, and appointment reminders to connect with study participants and keep them on track with their treatment plan. Future analyses of the mCare RCT will address participant response to these other components of mCare, as well as treatment group differences in health outcomes and traditional contacts with care team members. Moreover, follow-on studies are scheduled to evaluate specific elements of mCare where frequent provider contact and patient engagement could aid in the rehabilitation process. One study is in the areas of pain and goal awareness in wounded Service Members. Another is an evaluation of the mCare system for chronic care management of people with diabetes. The latter study will expand mCare functionality to include biosensors and patient health record, allowing for a more informed picture of the patient’s treatment compliance in the home environment.
CONCLUSION
The standard of care for provider to patient communications at the CBWTUs prior to mCare were comprised of weekly telephone contact between nurse Case Managers and rehabilitating Service Members, and once daily telephone contact between Platoon Sergeants and Service Members. Because mCare solicited detailed information about patients’ daily status, it had the potential to increase the quantity of provider-patient communications, to improve the quality of information providers received, and to shorten the waiting time for providers’ responses to problems. This analysis showed that rehabilitating Service Members, many of whom had behavioral health problems, PTS, and/or TBI, actively used mCare on a daily basis, responding to most status questionnaires and doing so in a timely way. Some weeks, Service Members with a higher level of burden were less responsive, but overall they responded in a manner comparable to their counterparts. If engagement is the key marker in determining the success of a rehabilitation program, mCare is a successful addition to standard care.
COMPETING INTERESTS
The authors have no competing interests to disclose.
FUNDING AND ALL OTHER REQUIRED STATEMENTS
This study was funded by the Office of Force Health Protection, and managed by the Mobile Health Innovation Center, an intramural laboratory of the Telemedicine and Advanced Technology Research Center, United States Army Medical Research and Material Command.
H.P. and S.J.F. did the analysis and drafted the manuscript. J.R.L. reviewed the analysis and manuscript. The remaining authors participated in study design and data collection.
Supplementary Material
ACKNOWLEDGEMENTS
The authors would like to thank the following individuals for their contributions towards conducting the mCare clinical research study:
Mike Bairas, Joyce Bennett, Mabel Cooper, Eric Cutlip, Lois Goldstein, Kurt Huttar, Cindy Gilley, Amanda Martin, April Pradier, Celicia Thomas, and Rick Wise.
SUPPLEMENTARY MATERIAL
Supplementary material is available online at http://jamia.oxfordjournals.org/ .
REFERENCES
- 1. Lequerica AH, Kortte K . Therapeautic engagement: a proposed model of engagement in medical rehabilitation . Am J Phys Med Rehabil. 2010. ; 89 : 415 – 422 . [DOI] [PubMed] [Google Scholar]
- 2. Hadgkiss EJ, Jelinek GA, Taylor KL, et al. . Engagement in a program promoting lifestyle modification is associated with better patient-reported outcomes for people with MS . Neurol Sci. 2015. ; 36 ( 6 ): 845 – 852 . doi: 10.1007/s10072-015-2089-1 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Fox BI, Felkey BG . Engaging patients with digital tools: what we think we know . Hosp Pharm. 2014. ; 49 : 992 – 993 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Laurance J, Henderson S, Howitt PJ, et al. . Patient engagement: four case studies that highlight the potential for improved health outcomes and reduced costs . Health Aff. 2014. ; 33 : 1627 – 1634 . [DOI] [PubMed] [Google Scholar]
- 5. Peña-Purcell NC, Boggess MM, Jimenez N . An empowerment-based diabetes self-management education program for Hispanic/Latinos: a quasi-experimental pilot study . Diabetes Educ. 2011. ; 37 : 770 – 779 . [DOI] [PubMed] [Google Scholar]
- 6. Musacchio N, Lovagnini Sher A, Giancaterini A, et al. . Impact of a chronic care model on patient empowerment on the management of Type 2 diabetes: effects of the SINERGIA programme . Diabet Med. 2011. ; 28 : 724 – 730 . [DOI] [PubMed] [Google Scholar]
- 7. Tang TS, Funnell MM, Brown MBB, et al. . Self-management support in “real world” settings: an empowerment-based intervention . Patient EducCouns. 2010. ; 79 : 178 – 184 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Agboola S, Havasy R, Myint UK, et al. . The impact of using mobile-enabled devices on patient engagement in remote monitoring programs . J Diabetes SciTechnol. 2013. ; 7 : 623 – 629 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Boland P . The emerging role of cell phone technology in ambulatory care . J Ambul Care Manage. 2007. ; 30 : 126 – 133 . [DOI] [PubMed] [Google Scholar]
- 10. Irvine AB, Russell H, Manocchia M, et al. . Mobile-Web app to self-manage low back pain: randomized controlled trial . J Med Internet Res 2015. ; 17 : e1 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Mbuagbaw L, Mursleen S, Lytvyn L, et al. . Mobile phone text messaging interventions for HIV and other chronic diseases: an overview of systematic reviews and framework for evidence transfer . BMC Health Serv Res. 2015. ; 15 : 33 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Verwey R, van der Weegen S, Spreeuwenberg M, et al. . A monitoring and feedback tool embedded in a counselling protocol to increase physical activity of patients with COPD or type 2 diabetes in primary care: study protocol of a three-arm cluster randomised controlled trial . BMC FamPract. 2014. ; 15 : 93 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Jones KR, Lekhak N, Kaewluang N . Using mobile phones and short message service to deliver self-management interventions for chronicconditions: a meta-review . Worldviews Evid Based Nurs. 2014. ; 11 : 81 – 88 . [DOI] [PubMed] [Google Scholar]
- 14. Holtz B, Lauckner C . Diabetes management via mobile phones: a systematic review . Telemed J E Health. 2012. ; 18 : 175 – 184 . [DOI] [PubMed] [Google Scholar]
- 15. Bailey JV, Webster R, Hunter R, et al. . The Men's Safer Sex (MenSS) trial: protocol for a pilot randomised controlled trial of an interactive digital intervention to increase condom use in men . BMJ Open. 2015. ; 5 : e007552 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Ajay VS, Praveen PA, Millett C, et al. . Role of mobile phone technology in tobacco cessation interventions . Glob Heart. 2012. ; 7 : 167 – 174 . [DOI] [PubMed] [Google Scholar]
- 17. Fazio AF . A concurrent validational study of the NCHS General Well-Being Schedule . Vital Health Stat 2. 1977. ; 73 : 1 – 53 . [PubMed] [Google Scholar]
- 18. McDowell I, Newell C . Measuring Health: A Guide to Rating Scales and Questionnaires . New York: : Oxford University Press; ; 1987 : 125 – 133 . [Google Scholar]
- 19. Poston WS, 2nd, Olvera NE, Yanez C, et al. . Evaluation of the factor structure and psychometric characteristics of the General well-Being Schedule (GWB) with American women . Women Health. 1998. ; 27 : 51 – 64 . [DOI] [PubMed] [Google Scholar]
- 20. Localytics . Time in App Increases by 21% Across All Apps September 16, 2014 at 9:00 AM, http://info.localytics.com/blog/time-in-app-increases-by-21-across-all-apps . Accessed 25 February 2015 . [Google Scholar]
- 21. Makovsky Integrated Communications . Fifth Annual “Pulse of Online Health” Survey Finds 66% of Americans Eager To Leverage Digital Tools To Manage Personal Health, February 24, 2015. http://www.makovsky.com/insights/articles/733 . Accessed March 3, 2015. [Google Scholar]
- 22. Localytics . Number of times mobile apps were used during the first six months of ownership among mobile app users worldwide, March 2011 - March 2014. http://www.emarketer.com/images/chart_gifs/175001-176000/175014.gif . Accessed 27 February 2015 . [Google Scholar]
- 23. Simmond M, Fleming J . Reliability of the self-awareness of deficits interview for adults with traumatic brain injury . Brain Inj. 2003. ; 17 : 325 – 337 . [DOI] [PubMed] [Google Scholar]
- 24. Volpp KG . Paying people to lose weight and stop smoking . LDI Issue Brief. 2009. ; 14 : 1 – 4 . [PubMed] [Google Scholar]
- 25. Volpp KG, Asch DA, Galvin R, Loewenstein G . Redesigning employee health incentives-lessons from behavioral economics . N Engl J Med. 2011. ; 365 : 388 – 390 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Riley GA, Hough A, Meader LM, et al. . The course and impact of family optimism in the post-acute period after acquired brain injury . Brain Inj. 2015. ; 1 – 9 . [DOI] [PubMed] [Google Scholar]
- 27. Feng J, Li S, Chen H . Impacts of stress, self-efficacy, and optimism on suicide ideation among rehabilitation patients with acute pesticide poisoning . PLoS ONE. 2015. ; 10 ( 2 ): e0118011 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Centers for Disease Control and Prevention . Injury Prevention and Control: Traumatic Brain Injury . www.cdc.gov/traumaticbraininjury/get_the_facts.html . Accessed 27 January 2015 . [Google Scholar]
- 29. National Center for PTS . Facts about PTS . psychcentral.com/lib/facts-about-ptsd/000662 . Accessed 28 January 2015 . [Google Scholar]
- 30. National Institutes of Mental Health . Major depression among adults . www.nimh.nih.gov/health/statistics/prevalence/major-depression-among-adults.shtml . Accessed 28 January 2015 . [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.