Linking Parent Confidence and Hospitalization through mHealth: A Multisite Pilot Study

Abstract

Objective:

To evaluate the associations between parent confidence in avoiding hospitalization and subsequent hospitalization in children with medical complexity (CMC); and feasibility/acceptability of a texting platform, ACTIV, to collect repeated measures of parent confidence.

Study design:

This prospective cohort study purposively sampled parent-child dyads (n=75) in one of three complex care programs for demographic diversity to pilot test ACTIV for 3 months. At random days/times every 2 weeks, parents received text messages asking them to rate confidence in their child avoiding hospitalization in the next month, from 1 (not confident) to 10 (fully confident). Unadjusted and adjusted generalized estimating equations with repeated measures evaluated associations between confidence and hospitalization in the next 14 days. Post-study questionnaires and focus groups assessed ACTIV’s feasibility/acceptability.

Results:

Parents were 77.3% mothers and 20% Spanish-speaking. Texting response rate was 95.6%. Eighteen hospitalizations occurred within 14 days after texting, median (IQR) 8 (2-10) days. When confidence was <5 vs. ≥5, adjusted odds (95% CI) of hospitalization within 2 weeks were 4.02 (1.20-13.51) times higher. Almost all (96.8%) reported no burden texting, one-third desired more frequent texts, and 93.7% were very likely to continue texting. Focus groups explored the meaning of responses and suggested ACTIV improvements.

Conclusions:

In this demographically diverse multicenter pilot, low parent confidence predicted impending CMC hospitalization. Text messaging was feasible and acceptable. Future work will test efficacy of real-time interventions triggered by parent-reported low confidence.

A hallmark of medical complexity is unpredictable and severe illness exacerbations often necessitating hospital care.(1–5) Health crises for children with medical complexity (CMC) typically emerge at home, where families are first in line to manage nuanced, frightening, and technically challenging episodes.(6, 7) At times of acute illness, CMC care routines can be especially difficult if parents perceive having limited knowledge or skills to manage them.(8, 9) Although clinical programs can react to parent-expressed concerns, they rarely have systems by which parents can easily express needs in real-time. Clinicians typically rely on families to initiate communication during such crises, making interactions inherently reactive. Proactively supporting families of CMC to face health exacerbations early requires tools and skills clinical programs typically lack.

Parents of CMC have described the period leading to hospitalization as a dynamic balance among the child’s health, health system supports and family capacity.(5) Using the conceptual model from this prior work, we hypothesize that parent-reported confidence to avoid hospitalization expresses the state of this balance simply, holistically, and in a potentially modifiable way. Decreased confidence can arise from any combination of health instability, unmet support needs or strained family capacity, therefore signaling increased risk for hospitalization. Prospective studies suggest that similar measures may predict hospitalization, having effect sizes as large as or larger than demographic, clinical, and primary care variables.(10, 11) Tools that assess this construct would leverage parent expertise by rapidly identifying when their comfort with keeping the child at home is approaching its limits and hospitalization might be imminent. Such tools could also allow clinical and research teams to test whether real-time clinical responses might reduce hospitalization risk.

Mobile (mHealth) technologies, such as text messaging, support simple real-time monitoring and early detection of health problems. Research on mHealth symptom monitoring to predict pediatric chronic disease exacerbations, eg, asthma, is early but promising. (12) Meta-analysis of mHealth interventions suggests they can improve chronic disease control in children, particularly when directed at caregivers.(13)

Our objective was to conduct a multisite pilot study of an mHealth platform with CMC caregivers (Assessing Confidence at Times of Increased Vulnerability, “ACTIV”). ACTIV uses longitudinal text messaging to prospectively monitor parent confidence for their child to avoid hospitalization over the subsequent month. Our aim was to identify associations between ACTIV’s repeated measures and CMC hospitalization, and to evaluate ACTIV’s feasibility/acceptability when implemented within a complex care program.

METHODS

This multisite prospective cohort study took place at pediatric complex care programs at University of Wisconsin American Family Children’s Hospital, UCLA Mattel Children’s Hospital and Boston Children’s Hospital, between August 2018 and May 2019. Each program delivers comprehensive interdisciplinary care and care coordination. UCLA’s program is distinguished from the others by delivering primary care. Programs use similar enrollment criteria based on numbers of affected organ systems, subspecialists, and past health services use. All programs aim to coordinate services, reduce unmet needs, achieve family-identified goals, and lower health services use. Program models have each been described previously in detail.(2, 14, 15) This study was approved by the Institutional Review Boards at all sites.

Participants

Caregiver inclusion criteria included being ≥18 years of age, having a child < 18 years enrolled in the site’s complex care program and having a personal phone capable of sending/receiving text messages. We purposively sampled n=75 caregivers (n=25 per site) to ensure participant diversity by sex, age, race/ethnicity, and English/Spanish primary language. Caregivers were consented and enrolled by phone, in clinic, or prior to hospital discharge. Spanish-speaking caregivers were recruited at a single site (UCLA), and all study activities were conducted in Spanish with native Spanish-speaking team members.

Longitudinal Measurement of Parent Confidence

Caregivers received and responded to a single text message prompt, “How confident are you that your child can avoid an unplanned hospitalization over the next month? Please respond on a scale from 1 (‘not confident’) to 10 (‘fully confident’).” If the child was hospitalized at the time of the text message, caregivers were instructed to respond with the letter “H” We modeled this approach after our previous observational study(10) and recommendations for constructing self-efficacy scales.(16)

We programmed text message delivery to occur for three months, at random days and times, averaging once every two weeks with at least seven days between messages. All texts were sent between 8AM and 9PM local time, with up to two reminders after 2 hours of non-response. Responses that were not a number from 1 through 10 or the letter “H” were classified as invalid and triggered an auto-reply asking the respondent to clarify their response. All text responses were collected in a single database. Responses were not monitored in real-time; however, study personnel were alerted to invalid responses for signs of technical difficulties or other concerns and logged any needed corrective actions.

Outcomes

Hospitalization.

The primary outcome was hospitalization within 14 days after text message, which aligned with text messaging frequency. Hospitalization was defined as an unscheduled hospital admission taking place across at least one midnight. Scheduled hospitalizations and day procedures were not included. We also conducted a pre-planned sensitivity analysis evaluating hospitalizations within 30 days, the timeframe about which parents were asked.

Feasibility and Acceptability.

We designed a mixed-methods evaluation to assess protocol uptake, feasibility, acceptability, and intention to use. Each domain was evaluated quantitatively through post-study participant questionnaires we developed for this study, and qualitatively through post-study focus groups at each site.

Data Collection

Quantitative Measures.

Hospitalizations were identified through a structured chart review protocol by trained research team members at the study conclusion. Protocol uptake, feasibility, acceptability and intention to use data were collected with a caregiver questionnaire conducted by trained research team members during a post-study phone call. Study data from all sites were entered into a REDCap database (Vanderbilt, https://www.project-redcap.org/) at UW.

Focus Groups.

Upon completion of the post-study questionnaire, a convenience sample of participants was invited to take part in a focus group, conducted by each site in-person or via a video conferencing platform (Zoom). Using a structured guide, a trained facilitator prompted participants to discuss their experiences with the study protocol, text-messaging, and the potential impact of texting. Focus groups were audio-recorded and transcribed; Spanish transcripts were subsequently translated to English. Two study team members reviewed the transcripts and developed a codebook through consensus-seeking iterative discussion. The study team, as a whole, discussed, refined, and finalized the codebook. The two coders then independently coded transcripts, and kappa statistics estimated inter-rater reliability between coders. Using an iterative, consensus-seeking inductive content analysis approach,(17) we identified themes related to caregivers’ experiences with ACTIV. Qualitative analyses were conducted in Dedoose (Manhattan Beach, California).

Statistical Analyses

Generalized estimating equations with repeated measures evaluated associations between confidence and 14-day hospitalization after text response. Prior to analyses, we planned to model confidence as a dichotomous variable, because we assumed that associations between confidence and hospitalization were likely not linear. We chose to use rating threshold of <5 vs ≥5 based on parent focus groups indicating that <5 typically represented concerning health instability. Receiving operating characteristic (ROC) analyses then compared areas under the curve, sensitivity and specificity of using each possible confidence threshold (e.g., <2 vs ≥2, <3 vs ≥3, <4 vs ≥4, etc.). Multivariable models used variables selected a priori, including caregiver age, study language, caregiver education, child technology assistance and duration of complex care clinic enrollment. Our pre-planned sensitivity analyses modeled confidence as a continuous variable and modeled 30-day hospitalization as the outcome. Results were presented as odds ratios (OR) with two-sided 95% confidence intervals (95% CI), and two-sided P <.05 was considered statistically significant. Analyses were conducted in STATA (version 16.0, College Station, TX).

RESULTS

We enrolled 25 caregivers per site (75 total), representing 92.6% of those approached. Table I summarizes enrolled caregiver and child characteristics. Caregivers were primarily mothers (77.3%), median (IQR) aged 39.5 (33-52) years, over 90% primary publicly insured, and racially/ethnically diverse (20% participating in Spanish). Almost half were from single-parent or split households (45.3%) and with GED or less education (40.6%). Among children in the cohort, median (IQR) of clinical measures included: 7 (3-11) medications, 6 (4-8) subspecialists, and 1 (0-2) hospitalizations in 12 months prior to study enrollment. Over two-thirds (68.9%) were assisted by medical technology. Text message response rates were 95.6% overall, with 83% occurring without any reminders. Duration of time in the study was not associated with increased need for reminders.

Table 1.

Demographic and Clinical Characteristics of the Study Participants

Caregiver Characteristics, n=75	n (%)
Relation to Child
Mother	58 (77.3)
Father	8 (10.7)
Foster Parent, Guardian, or Other	9 (12.0)
Age, years, median (IQR)	39.5 (33-52)
Gender
Female	65 (89.0)
Study Language
English	60 (80.0)
Spanish	15 (20.0)
Race/Ethnicity
White, non-Hispanic	29 (38.7)
Black, non-Hispanic	7 (9.3)
Hispanic	34 (45.3)
Other	5 (6.7)
Highest Education
Graduate degree	9 (12.2)
College degree	18 (24.3)
Some college or associate’s degree	17 (23.0)
General Education Diploma (GED)	21 (28.4)
Did not graduate high school	9 (12.2)
Household Income (pre-tax)
>$80,000	14 (18.7)
$40,000-79,999	17 (22.7)
< $40,000	35 (46.7)
Did not answer	9 (12.0)
Household Status
Married or Domestic Partnership	41 (54.7)
Single, Never-Married	15 (20.0)
Widowed	4 (5.3)
Divorced or separated	15 (20.0)
Child Characteristics, n=75
Complex Care Clinic Enrollment, months, median (IQR)	32.5 (12-58.5)
Affected Organ Systems, mean (SD)	5.4 (1.9)
Medications upon enrollment, median (IQR)	7 (3-11)
Subspecialists 12 months prior to enrollment, median (IQR)	6 (4-8)
Technology Assistance	51 (68.9)
Hospitalizations 12 months prior to enrollment, median (IQR)	1 (0-2)

Parent Confidence and Associations with Hospitalization

The distribution of text message responses (Figure) illustrates use of the entire confidence scale. About half of responses were 8 or higher and 20.8% were 5 or lower. We observed eighteen hospitalizations, occurring median (IQR) 8 (2-10) days after text message. In unadjusted models (Table 2), compared with a response ≥5, those <5 had significant associations, with OR (95% CI) of 3.38 (1.29-8.86) for hospitalization. After adjustment, effect sizes were larger, with AOR (95% CI) of 4.02 (1.20-13.51). Other variables included in the models did not have statistically significant associations with hospitalization (Table 2). Predicted probability of hospitalization was 12.8% when confidence was <5 compared with 3.7% when confidence was ≥5. ROC analyses (Table 3; available at www.jpeds.com) confirmed that the best odds ratio, sensitivity, specificity, and area under the curve occurred using the <5 vs ≥5 confidence threshold. When confidence was a continuous variable in the exploratory model, each additional confidence point had non-significant lower hospitalization odds; OR (95% CI) of 0.88 (0.74-1.06).

Figure 1. Parent Confidence Ratings for their Child to Avoid a Hospitalization in the Next Month.

Shown is the distribution of ratings of confidence to avoid hospitalization in the next month from 1 (not confident) to 10 (fully confident), measured by text messaging every two weeks among n=75 parents of children with medical complexity at three sites during the study period.

Table 2.

Associations between Parent Confidence and Future Child Hospitalization

	Hospitalization in 14 days
	OR (95% CI)	AOR (95% CI)a
Confidence Rating, (ref: ≥5)
<5	3.38 (1.29-8.86)	4.02 (1.20-13.51)
Caregiver Age, years	0.92 (0.86-0.99)	0.97 (0.90-1.03)
Study Language (ref: English)
Spanish	0.58 (0.12-2.78)	0.30 (0.02-4.00)
Highest Education (ref: Graduate degree)
College degree	0.44 (0.06-3.11)	0.27 (0.01-5.06)
Some college or associate’s degree	1.37 (0.27-6.87)	0.90 (0.06-13.31)
General Education Diploma (GED)	1.26 (0.26-6.11)	0.57 (0.04-7.62)
Technology Assistance	2.07 (0.55-7.72)	1.62 (0.41-6.42)
Complex Care Clinic Enrollment	0.99 (0.98-1.01)	0.99 (0.98-1.01)

^aModels adjusted for caregiver age, study language, caregiver education, child technology assistance and duration of complex care clinic enrollment

AOR: Adjusted Odds Ratio

Table 3.

Generalized Estimating Equation Model Characteristics with Different Confidence Thresholds

	Hospitalization in 14 days
Confidence Threshold	AORa	p-value	Sensitivity, %	Specificity, %	AUC
<2 vs ≥2	3.52	0.26	5.6	97.2	0.74
<3 vs ≥3	2.09	0.44	5.6	95.1	0.73
<4 vs ≥4	4.02	0.048	16.7	93.3	0.73
<5 vs ≥5	4.02	0.024	27.8	89.5	0.75
<6 vs ≥6	2.04	0.25	38.9	80.0	0.75
<7 vs ≥7	1.62	0.42	44.4	72.6	0.73
<8 vs ≥8	1.34	0.60	50.0	63.8	0.72
<9 vs ≥9	1.63	0.39	66.7	49.5	0.72
<10 vs 10	1.62	0.44	77.8	34.1	0.72

^aModels adjusted for caregiver age, study language, caregiver education, child technology assistance and duration of complex care clinic enrollment

AOR: Adjusted Odds Ratio

AUC: Area under the curve

In the sensitivity analysis using 30-day hospitalization as the outcome, results were similar. Among twenty-eight 30-day hospitalizations, compared with a confidence response ≥5, those <5 were significantly associated with AOR (95% CI) of 4.50 (1.51-13.39) for 30-day hospitalization.

ACTIV Feasibility and Acceptability

Complete post-study questionnaire feasibility and acceptability data were available for 63 (84%) of enrolled caregivers (Table 4; available at www.jpeds.com). Over 95% reported ACTIV to be very easy to respond to and not a burden at all to send/receive texts. Few (7.9%) reported technical difficulties, all within the first few weeks of the study. When asked their opinion about future texting frequency, about half (55.6%) reported we should send texts every 2 weeks; however, almost one-third (31.2%) reported that we should send them daily or weekly. Nearly all (>90%) were interested in continuing to use ACTIV or participating in a similar study in the future.

Table 4.

Feasibility and Acceptability of Measuring Parent Confidence by Text Message

	n (%)a
Feasibility Measures
How easy was it to respond to text messages during the study?
Very easy	60 (95.2)
Somewhat easy	2 (3.2)
Neutral	1 (1.6)
Somewhat difficult	0
Very difficult	0
Did you experience any technical difficulties sending or receiving texts?
No	58 (92.1)
Acceptability Measures
During the study, the total number of texts you received was…
About right	56 (88.9)
Too few	6 (9.5)
Too many	1 (1.6)
You received a text message asking you to tell us about your child’s health every two weeks. How often do you think we should send parents a text message?
Daily	2 (3.2)
Weekly	17 (28.0)
Every 2 weeks	35 (55.6)
Monthly	8 (12.7)
Every 2 months	1 (1.6)
How much of a burden was it for you to send and receive the study text messages?
Not a burden at all	61 (96.8)
Minor burden	1 (1.6)
Major burden	1 (1.6)
Very likely to continue texting	59 (93.7)
Would be interested in participating in another study with similar goals	58 (92.1)

^aComplete post-study data was available for n=63 enrolled caregivers.

In caregiver focus groups, parents’ experiences with ACTIV were summarized with four main themes: 1) facilitated communication with medical team, 2) global health assessment, 3) dynamic mode of care, and 4) increased support builds parent confidence (Table 5). Kappa >0.76 was achieved for all codes related to the major themes and subthemes.

Table 5.

Caregiver Perspectives about ACTIV - Focus Group Themes and Representative Quotes

Themes - Subthemes	Selected Illustrative Quotes
Facilitated Communication with Medical Team	“Then you have to wait until the provider is in the office to call back. Or we, even with MyChart, there’s times there might take a day or so to get back to us, things that they’re not real emergencies…so we’re not having to worry about, oh, it’s going to take a while for us to get a response from someone.” “The last 364 days have sucked. And then they say, he seems to be doing good right now. But if I saw or communicated with that person maybe on a monthly basis, I could relay the messages on his condition, and the doctor could give us a little bit more feedback and get a better understanding of how he was doing.”
Global health assessment	“My decision was based on how I felt my daughter was doing at that time. So, I was taking into consideration her wellbeing at that time. So, if she had any type of cold or anything and it had been a few days and it wasn’t improving, that’s what I would base my response on. “Si es que mi niño ha estado enfermo en eso días y tenía síntomas que ya se va enfermar que no está como normalmente jugando o comiendo bien. Si ya lo miro como todo tranquilito y eso entonces me iba a pensar que ya le voy a comenzar con el uno y el dos.” (If my son has been sick those days and he has symptoms that he will soon be getting sick, that he’s not playing or eating the way that he normally does. If I then see him back to normal/peaceful and in that moment I would have thought that I would start him with a one or two.)
Dynamic Mode of Care - Unpredictability	“I think the only problem was with our daughter, things change for her so quickly that I consistently answered five because she could be fine one day, and then be in the hospital the next.” “Para mí era un diez o un nueve, digo porque yo no sé el día de mañana si él va estar bien o no.” (For me it was a ten or a nine, I’d say because I don’t know if tomorrow he will be fine or not.)
Dynamic Mode of Care - Interventions	“I’m texting 10, 10, 9, 9, and that just goes into somebody’s folder and database and they, okay, he’s doing great. But if I start trending down, all of a sudden, I was nine, and then the next week I was 8, and then 7, and then 6, is someone reaching out saying, hey, how come you feel this way? What’s going on?” “I think its slowly reasonable to say that if a person were responding with negative results, or if you know a response of 1,2, or a 3 on a daily basis, then there obviously needs to be some type of a follow-up. And if a simple phone call from a doctor reach out to say hey what’s up, what’s going on, you know how’s your child doing, what’s the problem would be great.”
Dynamic Mode of Care - Individualized Care	“Mas o menos en las necesidades de cada niño. Y los ninos que están un poco más delicados, mandarles más seguido para que ustedes se den cuenta del niño más o menos. Es una forma de apoyo para los papas, especialmente porque a veces uno si siente. Cuando tú tienes un niño especial aprendes a junto a todo momento. A todo momento estas aprendiendo.” (More or less on the necessities of each child and the children that are a bit more delicate, send them more frequently so that you all can know about the child more or less. It’s a form of support for the parents, especially because sometimes you feel it when you have a child with special needs you learn at every moment.) “Hay niños que, aunque ya son grandes tienen sus defensas muy bajas y también tendría que ser más frecuente porque por ejemplo los de asma hay un tiempo que se enferman mucho.” (There are children that even though they are older have low defenses and will also need to be more frequent because for example, those with asthma, there’s a time when they get sick often.)
Increased Support Builds Parent Confidence	“Si, porque yo antes todo me daba pena preguntar al doctor. Mi hermano siempre me dice, ‘tienes que apuntar a preguntar esto…’ Esto de ver eso de hospitalizaciones, evitar cosas así, o cómo prevenir me hizo ser más preguntona. Fue como dice una motivación.” (Yes, because before I was too shy to ask the doctor any questions. My brother would always tell me, ‘You have to take note of asking this question… ‘ Having seen this about hospitalizations, avoiding it, or how to prevent made me ask more questions. It was what you call a motivation.) “Por ejemplo, la otra vez el niño cuando me entro ese texto tenía mucha fiebre y yo pues e otro niño tenía vómito. Tengo cuatro niños y estaban los tres malitos. Y a ver que ago. Y me entro el texto y yo estaba frustrada y cuando lolee, tengo que evitar la hospitalización y como que me levante y me puse a bajarle la fiebre y yo lo tome como una motivación para mí. Como si fuera alguna terapia.” (For example, the other time that I received a text, my child had a lot of fever and I, well, another child was vomiting. I have four children and three were sick. And I didn’t know what to do. And I received a text and I was frustrated, but when I read it, I have to avoid the hospitalization and it was as if I was encouraged and I started to lower their fever and I took it as a motivation for me. As if it were some sort of therapy.)

Facilitated Communication with Medical Team.

Caregivers perceived that receiving frequent communication through text messaging could improve care delivery by helping medical providers have more awareness of what parents go through between clinical visits. Caregivers referenced the ease of using text messaging, and identified limitations in current forms of communication with their child’s medical team, such as delayed response time.

Global Health Assessment.

Parents shared that when they received a text message, they would consider their child’s health, and use the 1-10 scale to convey how well their child was doing. When discussing whether / how to change the text prompt (i.e., “How confident are you that your child can avoid an unplanned hospitalization over the next month?”) participants discussed clarifying the response scale with more narrative and anchoring language as opposed to editing the prompt itself.

Dynamic Mode of Care.

Given the complexity of their child’s health, caregivers expressed the need to have a more fluid model of health care delivery to address issues that constantly arise. Within this theme, three subthemes were identified: unpredictability, interventions, and individualized care. Unpredictability: Caregivers expressed caution in rating their confidence because of the uncertainty of their child’s future health, noting how things can change in a matter of hours or days. Interventions: Caregivers identified a need for intervention, e.g., a follow-up phone call or text, based on their ratings or rating trend. Individualized Care: Caregivers perceived that the content and frequency of texts should depend on their child’s condition, age, and family circumstances.

Increased Support Builds Parent Confidence.

Caregivers described that receiving text messages served as a caregiving motivator, reminder, or stress relief. Some caregivers referenced that ACTIV helped them gain a new perspective or become more willing to ask questions that they might not have otherwise, thus strengthening their relationship with their clinical team. Of note, this theme, in comparison with the other themes, was coded more frequently in the Spanish versus English focus groups.

DISCUSSION

In this study, a simple text messaging platform to longitudinally monitor caregiver confidence for children to avoid a hospitalization identified significant associations between low ratings and hospitalization. When confidence was low (<5), which occurred about 10% of the time, CMC had four times higher adjusted odds of being hospitalized in the next 2 weeks compared with when confidence was higher. Moreover, mixed-methods data suggested texting was feasible and acceptable, implying the potential to integrate refined versions of the platform into real-time monitoring and clinical response systems. In addition to its clinical relevance, research could use this platform as a tool to study what drives real-time positive and negative family caregiver perceptions about their child’s health.

The notion that caregiver confidence measured longitudinally could predict hospitalizations builds off our prior conceptual model depicting hospitalization drivers for CMC,(5) in which the balance of the child’s health, health system supports, and family resources explains hospitalization risk. We hypothesize that ACTIV’s confidence rating is a summary measure reflecting the state of balance of these inputs, i.e., it reflects both parent impression of the child’s health and critical contextual realities (health system support and available family resources). By allowing context to affect responses, these ratings may offer richer information than general questions evaluating the child’s current state of health.

Findings from our focus group appear consistent with this conceptual model. In addition, mixed-methods research to understand key drivers to lower CMC hospital use (5, 18, 19) suggests that caregiver confidence may play an important role. Moreover, when measured using a question paralleling the text prompt, caregiver confidence at discharge predicted readmissions with effect sizes larger than clinical and demographic factors.(10, 20) Overall confidence with CMC home management plans is how caregivers have expressed self-efficacy to fulfill postdischarge care duties.(21, 22) Although self-efficacy has been defined as “having the confidence in one’s ability to deal with a situation without being overwhelmed,”(23, 24) because it is often ability-specific(25) and multi-dimensional, it can be difficult to measure. Confidence, on the other hand, may be a pragmatic metric for summarizing one’s various levels of self-efficacy.(26, 27) In fact, among adult patients with heart failure, caregiver confidence explained most of the variance in their contributions to patients’ self-care.(28) Measure development research illustrates promising psychometric properties of measuring caregiver confidence to care for children with tracheostomies in the home. (29)

Mobile health technology, despite its widespread availability, remains a relatively unexplored tool to monitor CMC health longitudinally. To date, mHealth has not been used to predict CMC hospitalizations, and descriptions of mHealth for CMC have focused predominantly on shared care plans and self-management support.(30, 31) Nkoy et al reported the perspectives of 13 caregivers for designing an mHealth home monitoring system for CMC.(32) Although somewhat broader in scope from our platform, a number of similarities to our work were notable. For example, caregivers in the study by Nkoy et al sought direct access to health care providers via text messaging and wanted to track symptoms that drive emergency department and hospital use. In particular, parents in this study were interested in tracking a “general feeling of uneasiness about their child’s condition”, which may align with our measure of confidence to avoid a hospitalization. Parents opposed monitoring their own (caregiver) symptoms or stress related to caregiving, which will be an important area for future exploration because many CMC caregiving interventions prioritize evaluating caregiving experiences.

Our platform, ACTIV, was feasible and acceptable according to our measures, even in a population with a relatively high proportion of caregivers with limited educational attainment, low household income and limited English proficiency. ACTIV appears to have a clear prompt, is user friendly, and takes parents only a few seconds. Caregivers did propose ideas for improvements. Families appreciated the proactive nature of the prompt; however, they thought that a dynamic texting frequency might be more useful, e.g., texting more frequently when confidence is lower and less frequently when confidence is higher. We suspect this perspective might explain why quantitative responses to ideal texting frequency were spread across the answer categories. Future iterations of ACTIV could be automated to alter the timing of subsequent text messages based on the current response or response trajectory, or even allow families to request a specific timeframe for the next text. Despite probing families to suggest changes to the text prompt, families instead prioritized clarifying the response choices as most important. Caregivers suggested giving families guidance on what 1, 5, 10 or any other response should mean to allow more consistency over time and uniformity across different families. Linking response numbers to recognizable circumstances was a related suggestion. We speculate that these changes might improve the performance of text-based communication to predict hospitalizations or other meaningful health outcomes.

Pairing an appropriate clinical response to low confidence is an important next step in this line of research. Because proactive crisis planning, enhanced access, and increased caregiver knowledge and technical skill have been expert-identified keys to lowering CMC hospital use,(18) approaches like ACTIV tied to a response may be particularly efficient strategies to reduce hospitalizations in this population. Complex care program infrastructure, which typically includes personnel already responsive to phone or electronic communication with families, may be relatively well positioned to handle a response burden. proactively identifying and then addressing problems earlier may ultimately prove to consume less staff time. Future work should evaluate provider perceptions of this communication strategy. During focus groups, families proposed relatively simple responses, including more frequent text messaging, automated yet directive replies, or a follow-up phone call or electronic message. In its current state, a cutoff text response of 5 to trigger intervention conveyed high specificity and relatively low sensitivity, suggesting some patients will be missed. We do not expect this strategy to capture all admissions given the variety of root causes for CMC hospitalization. We do suspect it identifies the important subset fitting our conceptual model of parent-perceived risks and resources related to their acute circumstances. These hospitalizations might be more avoidable than others, with well-designed interventions.

Our study has important limitations. The sample may be less representative of our populations than we assume. Caregivers of children in complex care programs may have unique associations between confidence and hospitalizations compared with caregivers of CMC not in complex care programs. Broadening our ACTIV platform to CMC outside complex care programs would be a valuable corollary to understand how complex care programs influence parent confidence and the ability to predict health exacerbations. We did enroll patients with diverse sociodemographic characteristics from eastern, western and midwestern United States; however, o our findings may not generalize to all CMC or those from backgrounds with different culture or primary language. In a larger study over a longer period, we may be able to construct more sophisticated confidence trajectories and adjust for important confounders using latent modeling or machine learning. Measuring daily stress and potential stress triggers simultaneously with confidence may shed light on what drives ratings and hospitalization risk. These models may perform better than our current approach.

Finally, our question was intentionally general to maximize its feasibility and acceptability. We deliberately did not ask probing questions about the sources for lower or higher confidence, though we recognize additional detail might guide the best type of follow-up intervention. Gathering more detail about reasons for low confidence might generate more accurate predictions. Cascading additional questions only when initial responses indicate low or changing confidence may be one strategy to avoid sacrificing feasibility and acceptability.

Despite these limitations, our multisite study represents an important first step towards establishing mHealth platforms to longitudinally monitor and identify periods of health stability and instability. Families offered several insightful improvements for future iterations of the text messaging platform. Given the observed associations between caregiver confidence and future hospitalization, this platform may prove effective at triggering just-in-time adaptive interventions to support vulnerable periods and improve key child and family health outcomes, including hospital utilization.

Abbreviations:

AUC

area under the curve

CMC

Children with Medical Complexity

ED

Emergency Department

mHealth

Mobile Health

ROC

receiver operating characteristic