Iterative Development of the Caregiver Wellness after Traumatic Brain Injury Program (CG-Well)

Abstract

Objectives:

1) To iteratively design a web/phone-based intervention to support caregivers of adults acutely following traumatic brain injury (TBI), Caregiver Wellness (CG-Well) and 2) To obtain qualitative and quantitative feedback on CG-Well from experts and caregivers to refine the intervention.

Setting:

A level-I trauma and tertiary medical center.

Participants:

Convenience sample of a total of 19 caregivers and 25 experts.

Design:

Multi-step prospective study with iterative changes to CG-Well: 1) Developed intervention content based on qualitative feedback from a prior study and literature review; 2) obtained qualitative feedback from 10 experts; 3) refined content using a modified Delphi approach involving 4 caregivers and 6 experts followed by qualitative interviews with 9 caregivers; 4) designed CG-Well website and videos; 5) obtained feedback on program acceptability, appropriateness and feasibility from 6 caregivers and 9 experts.

Interventions:

CG-Well included content on TBI, self-care and support, and skill-building strategies delivered through a website and telephone calls.

Main Outcome Measure(s):

Qualitative data were analyzed using content analysis. Caregivers and experts completed Likert-type scales to rate module relevance, clarity, accuracy, and utility and website acceptability, appropriateness, and feasibility (1 = strongly disagree to 5 = strongly agree). Means and standard deviations (SD) characterized ratings.

Results:

Qualitative findings were instrumental in designing and refining CG-Well. Ratings were positive for modules [means and SD for relevant (4.9, 0.33), clear (4.6, 0.53), accurate (4.9, 0.33) useful (5, 0)] and the website [means and SD for acceptable (4.8, 0.36), appropriate (4.8, 0.35), and feasible (4.8, 0.36)].

Conclusions:

The iterative design process for CG-Well resulted in a highly acceptable program. An early stage randomized controlled trial is underway to estimate treatment effects for a future well-powered clinical trial.

Introduction:

Traumatic brain injury (TBI) is the leading cause of death and disability in young adults, leads to high morbidity and mortality in older adults, and is projected to remain the highest cause of disability from all neurological diseases worldwide until 2030.^1–3 During and after the critical illness of a moderate to severe TBI(msTBI), untrained friends and family members of TBI survivors often assume acute caregiving responsibilities.⁴ These caregivers may be poorly equipped emotionally and physically to provide care; subsequently, as many as three quarters of them suffer from anxiety or depression.^5–12 Improving a caregiver’s ability to cope with stress and manage caregiver burden may improve the quality of support that they provide to the survivor, which may, in turn, promote the msTBI survivor’s neurologic recovery.^13–17

Caregivers of msTBI survivors report the greatest amount of life change, as well as greatest need for education, support, and preparation shortly after their loved one’s injury, often while they are still admitted to the intensive care unit (ICU). ^4,18 Prior work focusing on dyad interventions during inpatient rehabilitation have shown positive results, however, many patients with msTBI are not discharged to rehabilitation facilities, or are never approved by insurance for inpatient rehabilitation.¹⁹ Proactive help for caregivers could anticipate common problems such as survivor emotions and behaviors, navigate early phases of injury, and prevent acute stressors from becoming chronic.²⁰

In-person interventions are expensive, cumbersome, and require local expertise.¹⁸ Online interventions may have advantages compared to phone-only or in-person interventions. These advantages include the ability for caregivers to access information privately, conveniently on demand at any time of day or night, even at a distance, with less stigma to seek help, which may improve their feasibility and acceptability to caregivers.^23–28 A recent published review of trials of telebhehavioral interventions for family caregivers of individuals with TBI described that the majority (2/3) were aimed at children or adolescents, so target outcomes were in families, rather than primary caregivers. Of the remaining four interventions directed toward adults, three were primarily phone-based.²⁹

Our goal was to develop a feasible caregiver intervention that began while the adult survivor was in the ICU, Caregiver Wellness after TBI (CG-Well). We obtained qualitative and quantitative outcomes to better understand acceptability, feasibility, and appropriateness. When interventions are developed with these implementation outcomes in mind, uptake by health systems or individual participants is improved.²²

The overarching goal of CG-Well is to support family caregivers during the initial days and weeks post msTBI by equipping them with education, information about support systems, and skills to manage common challenges. The purpose of this study was to develop and refine the content for CG-Well, which will be subsequently tested in a randomized controlled trial (RCT). The development entailed a multi-phase process that integrated qualitative and quantitative methods. This article describes the iterative approach to the development and reports the early implementation ratings of CG-Well.

Methods:

Overview of the Design Process, Participants, and Setting:

We incorporated feedback from a multidisciplinary group of expert and caregiver participants throughout all development stages of CG-Well, which occurred from 2019–2021. Caregivers of patients with an msTBI were recruited using convenience sampling at our institution, a Level 1 trauma center that serves as a quaternary referral center for neurotrauma. Recruitment of caregivers was via phone or in-person. Caregiver participants were untrained family or friend caregivers of patients with a moderate to severe TBI (GCS < 13) who were admitted to the neuro ICU or surgical ICU. We excluded caregivers of patients not expected to survive their injuries, since those caregivers may require different information regarding support for grief, rather than how to care for a person with TBI. Experts included clinicians or researchers with over five years’ experience treating patients and families with TBI. Experts were recruited from a variety of practice settings across the United States using purposeful sampling by email. Specialties of experts included physicians in neurocritical care, neurosurgery, physical medicine and rehabilitation; as well as experts in nursing, social work, physical therapy, occupational therapy, and psychology, A total of four separate cohorts were enrolled in the studies to provide qualitative feedback and quantitative ratings: 1) Ten experts who reviewed modules in the form of a Word document; 2) A combined group of four caregivers and six experts who formed a modified Delphi consensus group and reviewed the Word document modules; 3) An independent cohort of nine caregivers who provided qualitative feedback on the Word document; and 4) A group of nine experts and six caregivers who reviewed the website once it was developed.

Approval was obtained from the institutional review board before the initiation of each component of this research. All participants provided informed consent. All interviews were conducted by phone, due to Covid. All participants received a gift card as a token of appreciation.

Preliminary Work and Step 1:

CG-Well was conceptualized and designed in several stages (Figure 1). Prior to these stages, we conducted a qualitative study of serial in-depth interviews with caregivers. This study described the unmet needs of caregivers of patients with a TBI beginning at 72 hours post injury, characterized the changing timeline of caregiver needs in the first six months after TBI, and informed the initial organizational structure of CG-Well.^4,24 These data, as well as successful components of other previously published interventions were utilized to build the initial iteration of CG-Well.^6–9,20,30 This draft of modules was written in a Microsoft Word document as Step 1.

Figure 1.

*In step 3 the same module document was reviewed by two separate cohorts.

Step 2:

These modules were reviewed by a multidisciplinary group of experts, and changes made based on their suggestions (Step 2, Figure 1). This cohort were clinicians or researchers with expertise in TBI, consisting of three critical care and rehabilitation physicians, three critical care or home healthcare nurses, two physical therapists, and two social workers.

Step 3:

The content was then reviewed by a combined group of experts and caregivers to reach modified Delphi consensus (Step 3, Figure 1).³¹ This modified Delphi consensus panel consisted of three healthcare providers, identified as experts by having five years of clinical experience with patients with a TBI and their caregivers; three researchers, identified as having a medical or doctoral degree and record of publication in TBI, and; four long-term (over 6 months post-injury) caregivers of patients with a TBI.⁴

Quantitative ratings were obtained to determine the content was relevant, clear, accurate, and useful using Likert scales (1 = strongly disagree to 5 = strongly agree).^32,33 These ratings included four questions adapted from published forms developed by one of the authors (TB) for use in her published intervention, Telephone Assessment and Skill-Building Intervention for Stroke Caregivers.³⁴ We established a priori that 75% of the modified Delphi consensus needed to provide a score of 4 or 5 for each of the ratings.

Since mode of delivery is important to intervention development and implementation, we also collected information about how caregivers would prefer to communicate and receive information from study staff in future studies. Participants ranked several options of delivery including in-person at home, in-person at clinic, telephone, videoconferencing, website / blog / web chat, smart based mobile app, Facebook or social media, email, or texts (1 = lowest preference to 10 = highest preference).

We conducted qualitative interviews with the Delphi consensus group, as well as a new group of caregivers who had been enrolled while their loved one was in the ICU (Step 3, Figure 1). The new caregiver group was enrolled after injury, and their interviews were conducted six months later to allow them time to reflect on their experiences as a caregiver transitioning from the ICU, post-hospital care, and home. We obtained qualitative data from these cohorts (total n = 16) using a semi-structured interview guide designed by the investigators that differed slightly depending on whether the participant was an expert or caregiver (Supplemental Table 1). All qualitative interviews were conducted by phone by a trained coordinator (ST) who was not involved in the development of the modules. The coordinator’s qualitative training consisted of practice interviews, role play, and feedback. Interviews were recorded and transcribed, and lasted an average of 17 minutes.

Step 4:

In step 4, we developed a website using the module content that had been reviewed by experts and caregivers in Step 3 (Step 4 in Figure 1). The website modules were then revised several times by the core team of researchers (NK, TB, BK, SW, OA, LG) to simplify content and to accommodate a wide range of health literacy (Step 4, Figure 1). Members of this core team had extensive experience in web-based interventions or psychosocial interventions.^{24,25,27,28,35,36}

Step 5:

The website was then reviewed by a multidisciplinary group of nine experts and six caregivers who were between 1- and 3- months post injury. Qualitative feedback was obtained using a semi-structured interview guide designed by the investigators (Supplemental Table 1). Interviews were recorded, and transcribed, and lasted an average of 13 minutes.

Quantitative data were obtained using three four-item measures, the Acceptability of Intervention Measure (AIM), the Feasibility of Intervention Measure (FIM), and the Intervention Appropriateness Measure (IAM), which are Likert scales from 1 (strongly disagree) to 5 (strongly agree) (Step 5, Figure 1).³⁰ Within each of the three categories the means and SDs were calculated.³⁰ The measures were adapted for CG-Well by substituting, “this EBP” with “CG-Well” for each question, as recommended by the authors of the measures. ³⁰ When these measures were combined with questions from the semi structured interview guide (Supplemental Table 1), they provided important implementation information. We used feedback from Step 5 to improve clarity, appearance and usability of the website.

Analysis:

Quantitative sample characteristics and module ratings were analyzed using descriptive statistics in Excel. Continuous variables were summarized using mean, standard deviation, and range. Discrete variables were described using frequency and percent. Quantitative data and demographic data were stored in RedCap.

Qualitative findings were analyzed using content analysis in Excel with standard procedures after interviews had been transcribed verbatim.³⁷ Two authors (LG and NK) read the qualitative responses several times to gain familiarity with the content. Themes were determined in advance using strengths and weaknesses pertinent to each Likert category. Two code books were developed for both analyses, edited, revised several times, and agreed upon (Supplemental Tables 2 and 3). The code book included definition, when to include, when not to include, and examples of each code. The two coders independently coded the statements, then met to discuss findings. Unresolved discrepancies were resolved by a third coder. Qualitative data were stored in Excel. The COREQ framework was used for reporting, conducting, and analyzing the qualitative analysis.³⁸ The recurrence of themes that emerged from the data implied saturation.

Results:

Participants, Qualitative Data, and Ratings in Each Step:

Sample characteristics for total participants are shown in Table 1. During recruitment, 15 caregivers refused enrollment. An additional 31 caregivers were identified and not enrolled because they either did not meet inclusion criteria, it was unclear who the caregiver was, or the caregiver could not be reached. There were no refusals from experts.

Table 1:

Demographics and Participant Characteristics

	Step 2 Experts (N=10) M(range) or N (%)	Step 3 Delphi Consensus Group of Experts and Caregivers (N=10) M(range) or N (%)	Step 3 Caregivers at 6 months post injury (N=9) M(range) or N (%)	Step 5 Experts (N=9) M(range) or N (%)	Step 5 Caregivers at 1–3 months post injury (N=9) M(range) or N (%)
Age, years (range)	46.7 (35–70)	43.7 (34–65)	50 (37–60)	41 (35–51)	52.4 (23–82)
Sex, male	1 (10%)	6 (60%)	1 (11%)	4 (44%)	1 (11%)
Race
White	9 (90%)	8 (80%)	9 (100%)	9 (100%)	9 (100%)
Black	1 (10%)	2 (20%)	0 (0%)	0 (0%)	0 (0%)
Hispanic	0 (0%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)
Number of caregivers	0 (0%)	4 (40%)	9 (100%)	0 (0%)	9 (100%)
Number of experts	10 (100%)	6 (60%)	0 (0%)	9 (100%)	0 (0 %)
Specialty (if expert)
Social Work	2	0	N/A	1	N/A
Nursing	3	0	N/A	1	N/A
PMR Physicians	1	0	N/A	1	N/A
Neurocritical are Physician	1	5	N/A	4	N/A
Physical Therapy	2	0	N/A	0	N/A
Psychology (PhD)	1	0	N/A	1	N/A
Neurosurgery	0	1	N/A	0	N/A
Occupational Therapy	0	0	N/A	1	N/A
Relationship to Patient (if caregiver)
Parent	N/A	2	3	N/A	2
Spouse or SO	N/A	2	5	N/A	4
Grandchild	N/A	0	0	N/A	1
Sibling	N/A	0	1	N/A	2

M: Mean; SO: Significant other; N/A: not applicable

Step 2:

Examples of feedback in step 2 are depicted in Table 2 and included: developing additional topics, information, or explanation in certain areas, creating additional images, or changing the order of topics. Experts provided information specific to their field. Using these data, changes were made to the module content, mostly in terms of adding content to the modules.

Table 2:

Qualitative Feedback

	Theme	Code	Brief Description of Code	Number of statements coded (Positive, Negative)	Example
Caregivers, Step 3	Relevant	Heterogenous	Applicable to wide variety of CGs	2, 0	“Any caregiver should get this information.”
		Timing	Discussion of when to receive modules post injury	14, 1	“Caregivers should receive this right at the beginning.”
	Clear	Health Literacy	Readability of modules	1, 1	“Those who do not have any language barriers will benefit.”
	Accurate	General	Content is factually correct	1, 2	“When I went through this it wasn’t explained well.”
	Useful	Need for this	An intervention like this is necessary for CGs	23, 1	“They aren’t going into it blind (with this). I went into it blind.”
		Recommended	Would offer this intervention to others in their field	3, 0	“(Would recommend) to ones literally caring for the patient.”
		Comprehensive	Modules contain all information that is likely needed	4, 6	“You can’t go through them (modules) all at one time.”
Experts, Steps 2 and 3	Relevant	Heterogenous	Applicable to wide variety of CGs	2, 1	“I wonder if you’ve considered the impact of a high level of anxiety on the family’s capacity to absorb information.”
		Timing	Discussion of when to receive modules post injury	3, 5	“My concern is offering too much information in the beginning.”
	Clear	Health Literacy	Readability of modules	6, 7	“All of the modules are well written - but I wonder if they could be written even more directed at the lay person.”
	Accurate	General	Content is factually correct	6, 0	“A lot of great content!”
	Useful	Need for this	An intervention like this is necessary for CGs	22, 3	“Conceptually, this program is definitely needed.”
		Recommended	Would offer this intervention to others in their field	6, 0	“I would recommend this tip sheet.”
		Comprehensive	Modules contain all information that is likely needed	1, 2	“Not everyone survives. There is no content about this being a lifelong decision.”
Caregivers, Step 5	Acceptable	Technology	User-friendliness of the website	6, 2	“I really liked that there was a video with someone talking.”
		Digestible	Modules are broken down	2, 0	“I liked that they (modules) were broken down.”
	Appropriate	CG centered	Modules are meant for CGs rather than clinicians or experts	3, 0	“Very helpful that the caregiver is the focus.”
		Necessary modules	A specific module is needed	4, 7	“Would add grief module if loved one does not survive.”
		Thorough	Modules have everything that is needed in them	4, 0	“All are must see (modules).”
		Timing	Time after injury when CGs should receive modules	6, 0	“I wish I had this when he was in the ICU.”
		Recommended	CG/expert recommends website	7, 0	“Would recommend to all possible caregivers.”
	Accurate	General	Information was factually correct.	1, 0	“The suggestions applied to me.”
Experts, Step 5	Acceptable	Technology	User-friendliness of the website	12, 1	“The website was easy to navigate.”
		Digestible	Modules are broken down	6, 3	“Perhaps shorten the length to < 8 minutes, I tended to lose attention after that.”
	Appropriate	CG centered	Modules are meant for CGs rather than clinicians or experts	10, 0	“Easy language and sensitive images.”
		Necessary modules	A specific module is needed	12, 9	“There is fairly little discussion of palliative approaches to care for severe TBI and decisions regarding withdrawal of care.”
		Thorough	Modules have everything that is needed in them	7, 1	“I feel like the website answers a lot of questions we get from families all of the time.”
		Timing	Time after injury when CGs should receive modules	4, 0	“I like how the content is divided up by time period since injury.”
		Recommended	CG/expert recommends website	9, 1	“I also would probably mostly focus on younger patients who are more likely to survive long term from their ICU stay. “
	Accurate	General		0
Examples of Specific Suggestions for Changes to Make to CG-Well Based on Feedback
Examples of Specific Suggestions (Steps 2 and 3)		Information to Add/Remove		“Maybe add more real life / hypothetical cases for some of the tips.”
		Other ideas mentioned		“Support groups. Absolutely. And also social media.”
Specific Suggestions (Step 5)		Content to add/remove		“It would be helpful to add a module on neuropsych visits would be helpful. Research has shown that a lot of patients have issues.”
		Appearance/ Spelling/ Clarifications		“Have a variety of people do the videos.”
		Organization / Links		“Could add music to the website.”

CG - caregiver

Step 3:

Among the ten participants who completed the modified Delphi panel, the average ratings for modules were 4.89 for “relevant,” 4.56 for “clear,” 4.89 for “accurate,” and 5 for “useful.” Since over 75% of the group provided ratings of 4 or greater in the first round, only one round of the modified Delphi consensus was conducted. Qualitative data were obtained, with examples in Table 2, and strengths or weaknesses within the codes of “relevant,” “clear,” “accurate,” and “useful” were documented.

The average age of the group of nine caregivers in step 3 was 50 years old (range of 37 to 60 years). All were white, and none were Hispanic. The most common relationship to the survivor was spouse or significant other. Of these nine caregivers, six provided qualitative interviews when their loved one was six months post injury. One caregiver was lost to follow up. In two other cases, the patient with TBI died. Examples of qualitative data are included in Table 2. The most commonly coded statements were in the theme of “Useful,” under the code, “Need for this,” which meant that the participant felt an intervention such as this was necessary. The most commonly cited weakness in Step 3 was in the theme of “Clear,” under the code, “Health Literacy,” which meant the readability of the modules needed to be more accessible to a broader audience.

Qualitative feedback from both cohorts in Step 3 was utilized to make the following edits to the modules: adding colorful images to accompany text, ensuring the terminology was the appropriate level of health literacy, adding content to topics related to TBI rehabilitation practices, and adding information in the early ICU modules about legal and financial burdens.

Step 4:

As the modules were adapted into a website, extensive revisions were made by the research team (NK, TB, SW, BK, OA, LG). Major changes to the video components of the modules included simplification of slides and reduction of length of videos. An interactive PDF version of each module was developed and added to accompany each video, closed captioning added, and reference websites available in the public domain were added.

Step 5:

As part of step 5, the website was reviewed by six caregivers of patients with TBI and nine experts (Table 1). The six caregivers were recruited during their loved one’s ICU stay and interviewed 1–3 months later. We approached a total of 17 caregivers for enrollment. Of the nine who consented to the study, one caregiver was lost to follow up and two patients died of their injuries. Six caregivers provided feedback at 1–3 months post injury. Of the nine enrolled, one was male, eight were female. The average age of the caregivers was 52.4. The nine experts at this stage consisted of four neurocritical care physicians, one social worker, one occupational therapist, one PM&R physician, one psychologist with experience in TBI rehabilitation, and one nurse with extensive experience in a neurointensive care unit. Ratings across a Likert scale were obtained to determine if the modules were acceptable, appropriate, and accurate (1 = strongly disagree to 5 = strongly agree). Mean ratings were 4.8 (SD 0.36) for acceptability, 4.87 (SD 0.35) for appropriateness, and 4.82 (SD 0.36) for feasibility (Table 3).

Table 3:

Module Ratings and Preferred Method of Delivery of Modules

Ratings of Modules in Word and Website Format
	Mean ratings of all modules	Standard Deviation	Range
Ratings of the Word Document Modules
Relevant	4.89	0.31	4–5
Clear	4.56	0.50	4–5
Accurate	4.89	0.31	4–5
Useful	5	0	5
Ratings of the Website
Acceptability	4.80	0.36	4–5
Appropriateness	4.87	0.35	4–5
Feasibility	4.82	0.36	4–5
Preferred method of delivery of the intervention
Method of Delivery	Average Rating	Standard Deviation	Range
Smart-device based mobile app	8.75	1.29	6–10
Website	8.50	1.68	5–10
Hard Copies, In-person	7.78	2.99	1–10
PDF, iBook, or other digital format	6.13	2.59	5–10
Hard Copies, by Regular Mail	3.63	2.43	1–7
USB Flash Drive by Regular Mail	2.88	2.07	1–9
Methods to communicate results and feedback of the modules back to program staff
Telephone calls	7	2.12	3–10
Facetime, skype, or other video conferencing	7	1.41	4–8
Smart device based mobile app	7	2.12	3–8
Email exchanges	6.75	1.85	7–9
In person meetings in a clinic	6.56	2.75	1–10
In person meetings in a clinic	6.56	2.75	1–10
Text messages	6	1.66	4–8
Website, blog, or web chat	6	1.87	3–8
In person meetings at home	5.67	3.13	1–10
Facebook or other social media	4.50	2.29	1–6

Ratings of the word document files and ratings of the website consisted of Likert scales where 1=strongly disagree to 5=strongly agree. Ratings of the preferred method of delivery and preferred methods to communicate results and feedback back to program staff consisted of Likert scales where 1=least desirable to 10=most desirable.

Similar to step 3, strengths and weaknesses within the themes of “acceptable,” “appropriate,” and “accurate” were documented (Table 2). Experts had more comments than caregivers acknowledging the website was intended for patients likely to survive their TBI, with little information acknowledging withdrawal of care decisions or end of life care. Compared to caregivers, experts had more comments to improve diversity and accessibility of the website. The most commonly coded positive statements were in the theme, “Acceptable,” under the code, “Technology,” meaning participants found the website to be user-friendly. Minor changes were made to the website in Step 5: pictures to reflect more caregiver diversity, titles to be user friendly, adding a search function, and adding a frequently asked questions section.

Quantitative responses at all stages were generally positive. Because of this, attention was placed on the qualitative ratings which highlighted key areas that could be targeted to optimize CG-Well. Combined, the ratings across all measures were at or above 4.56.

Preference for Delivery

We queried participants on multiple possible forms of delivery of CG-Well using a Likert scale of several approaches, (1=least desirable, 10=most desirable, Step 3, Figure 1 and Table 3). Participants responded with a variety of preferences in how they would like to receive the modules. Although smart based app, website, and in-person hard copies received the highest scores as an option for module delivery, the scores were clustered closely together, and the range in ratings for each item was wide, indicating that preference for delivery was an individualized decision.

Qualitative responses indicated that a variety of delivery methods should be offered to participants. Examples of comments were: “Would like a website with small bits at a time,” and “You will probably need multiple options (of delivery).” In addition to feedback about preference for delivery, we took future dissemination of CG-Well into account when determining the platform. For example, in-person delivery poses challenges related to feasibility, cost, and intervention fidelity.¹⁸

We chose a web-based platform for CG-Well. Because of weaknesses identified in an online only platform, a phone protocol is also in place for the ongoing RCT. This phone protocol is standardized, and adapted from prior work, in which extensive feedback was previously obtained..³⁹ The phone protocol will serve as a way to maintain participant engagement, provide tailored assistance to participants, and assist participants who have challenges with website literacy.

Description of the final product:

The final product was a website that contained all modules developed in this process (Table 4). The website contained four time points, consisting of “Early ICU,” “Later ICU,” “After the ICU,” and “Later Time Periods.” Each time point contained 8 to 12 separate modules, for a total of 43 modules (Table 4). Each module contained a 5 to 12-minute-long video, an interactive PDF workbook, questions to help the caregiver reflect on the material, and links to other relevant websites. These links included a sleep log, headache diary, information available through the Brain Injury Association of America, and other resources. Videos were designed using Microsoft Powerpoint and Jumpshare.

Table 4:

Module and Protocol Content

Time Period		Module
Early ICU	The ICU	Understanding Initial Steps
		ICU Schedule and How the ICU Works
		Understand the Team Caring for my Loved One
		How to Communicate with the ICU Teams
	Medical Information	Devices, Tubes, and Machines
		Prognosis and Likelihood of Improvement
		Early Financial / Legal Issues
	Your Emotions	Managing Emotions in the Fast-Paced ICU
		Balancing Life and Time in the ICU
		Feeling Angry After the TBI
Later ICU	The ICU	How to Help my Loved One in the ICU
		Transitioning out of the ICU
	Medical Information	Sharing the Injury with Others
		Finding the Right Skilled Nursing Facility
	Your Life and Emotions	What is Caregiver Burden
		Building Support Network Early
		Early Conversations with Children
		Staying Organized
		Problem Solving and Decision Making
After ICU	Medical Information	Additional Survivor Emotions
		Communication with the Survivor
		Handling Survivor Anger and Emotions
	Your Life and Emotions	Caregiver Coping Strategies
		Managing Your Anxiety and Stress
		Managing Sadness in the TBI Survivor
		Balancing Caregiving with your Job
		Balancing Caring for Children
		Becoming an Advocate
		Managing Finances
Later Time Points	Medical Information	Arranging Follow-Up Appointments
		Managing Medications at Home
		Assisting with Survivor Mobility
		Cognitive Problems after TBI
		Medical Issues and Complications at Home
	Your Life and Emotions	Taking Care of Your Health
		Caregiver Sadness and Depression
		Dealing with Isolation
	Logistical Issues	Caregiving Basics if you are Coming Home
		Preparing Your Home
		Activities of Daily Living
		Transportation Needs

The website was accessible to a variety of caregivers. All audio contained close captioning. Images depicted drawings or pictures of caregivers and survivors with a range of ages, physical abilities, races, and ethnicities. Because the results on delivery preference were mixed between in-person, paper copies, phone, and web, we included several options for delivery in the final product. For example, even though the platform is web-based, caregivers receive phone calls from a trained coordinator using a protocol to work through modules or any technology related issues. This is similar to other published interventions, where it was felt that a web-based platform alone was insufficient.^40,41

Discussion:

Development and early implementation testing of CG-Well represents an important step in meeting the unmet needs of family caregivers soon after a loved one has sustained an msTBI. CG-Well offers caregivers tailored resources, which aim to improve their quality of life and to reduce distress. Consistent with best practices in user-centered design, CG-Well was iteratively developed and informed by feedback from caregivers of TBI survivors as well as a wide array of healthcare professionals who interact with TBI survivors and their families.⁴² By testing acceptability, feasibility, and appropriateness early in the development process of our intervention with qualitative and quantitative measures, we kept in mind issues related to implementation that are critical prior to testing for efficacy outcomes.^21,22 As an example, we focused on developing and revising website content in the form of short videos that were appealing, yet digestible.

This iterative process of combining experts and caregivers has been previously used to successfully develop interventions in many other disease states.^32,43 Specifically, collaboration with experts has been recommended to validate the content and layout of materials intended for interventions targeted and tailored to specific populations.^44,45 In addition, it allowed us to consider how the intervention may work within a healthcare system in future dissemination steps.⁴⁶ We did note that experts and caregivers brought up different issues (Table 2). For example, caregivers requested additional modules about grief and palliative care. Experts commented that the modules were best for patients with TBI who were more likely to survive past their ICU stay. These comments, although contrasting, highlighted the fact that CG-Well corresponded to the specific patient population for whom it was designed (e.g., caregivers of patients with an msTBI likely to survive).

Given the high quantitative ratings of intervention content, qualitative feedback proved essential in refining and improving the module content. For example, the highest number of negative comments focused on health literacy, which describes issues of acceptability and appropriateness for caregivers. In response, the team revised the modules numerous times to simplify language and improve readability to a 5^th grade level. Although iterative changes were made throughout the development process, not all changes or recommendations could be incorporated, since some suggestions were based on personal preference. For example, suggestions that significantly hampered the feasibility of the project, such as adding music to the website, were not included.

Although we studied several implementation outcomes such as appropriateness, acceptability, and feasibility as part of this study, we have ongoing work to continue iterative modifications to CG-Well to improve implementation. We recently began an RCT to compare CG-Well to an attention control group of caregivers of patients with msTBI who are admitted to the ICU. As part of this study, we will obtain additional implementation outcomes such as fidelity and cost, as well as satisfaction ratings. ^21,22 We are actively recruiting a more diverse population, as this has been a weakness of most previous TBI behavioral telehealth interventions.²⁹ We are collecting data on educational, healthcare-related, and digital literacy to better understand how CG-Well can meet the needs of a more diverse population.

Limitations:

Despite the rigorous processes used in this study, limitations exist. First, experts were selected purposefully, to assure that we included multiple specialties that interact with caregivers of patients with a TBI at different points in the first six months after injury. Notably, we had a small sample size at each stage of development. Although previous studies describing expert ratings for content validity are few, recommendations for the number of experts required to establish validity range from 5 to 10 participants.⁴⁷ We do acknowledge our sample of experts and caregivers lack diversity and heterogeneity, which has been a weakness of prior similar work. ²⁹ Our sample was predominately white, with no Hispanic participants, so generalization to minoritized populations is limited. We are purposefully enrolling a more diverse group of caregivers in the ongoing RCT to further refine the program. Lastly, research activities took place during the Covid-19 pandemic. This necessitated that enrollments and follow-up interviews were conducted virtually or by phone, which may have influenced our results. In particular, due to hospital visiting restrictions, we met many of the caregiver participants for the first time by phone. This may have decreased rapport, and led to some caregivers being less willing to participate in the study. Although enrollment and responses may have differed if conducted in-person, CG-Well is a web and phone-based intervention, so these collection methods mirrored our ultimate delivery platforms. Our study is not generalizable outside of the United States.

Conclusions:

CG-Well was developed to address a critical gap in accessible interventions that are tailored for the ICU admission. This intervention was developed with a multidisciplinary team of experts and caregivers. This development process resulted in a tailored, relevant, and meaningful web and phone-based program with exciting potential to improve outcomes for this overlooked population. Future research will establish feasibility and efficacy.

Supplementary Material

Supplemental Image 1

Website Home Screen

Supplemental Image 2

Website page depicting where caregivers can choose modules based on time since injury