Digital health interventions to support family caregivers: An updated systematic review

Abstract

Objective

Chronic diseases are the leading causes of death and disability in the U.S., and disease management largely falls onto patients’ family caregivers. The long-term burden and stress of caregiving negatively impact caregivers’ well-being and ability to provide care. Digital health interventions have the potential to support caregivers. This article aims to provide an updated review of interventions using digital health tools to support family caregivers and the scope of the Human-Centered Design (HCD) approaches.

Methods

We conducted a systematic search on July 2019 and January 2021 in PubMed, CINAHL, Embase, Cochrane Library, PsycINFO, ERIC, and ACM Digital Library, limiting to 2014–2021 to identify family caregiver interventions assisted by modern technologies. The Mixed Methods Appraisal Tool and the Grading of Recommendations Assessment, Development and Evaluation were used to evaluate the articles. Data were abstracted and evaluated using Rayyan and Research Electronic Data Capture.

Results

We identified and reviewed 40 studies from 34 journals, 10 fields, and 19 countries. Findings included patients’ conditions and relationships with family caregivers, how the technology is used to deliver the intervention, HCD methods, theoretical frameworks, components of the interventions, and family caregiver health outcomes.

Conclusion

This updated and expanded review revealed that digitally enhanced health interventions were robust at providing high-quality assistance and support to caregivers by improving caregiver psychological health, self-efficacy, caregiving skills, quality of life, social support, and problem-coping abilities. Health professionals need to include informal caregivers as an essential component when providing care to patients. Future research should include more marginalized caregivers from diverse backgrounds, improve the accessibility and usability of the technology tools, and tailor the intervention to be more culturally and linguistically sensitive.

Introduction

Chronic diseases such as diabetes and cancer are the leading causes of disability and death, accounting for more than three-quarters of U.S. healthcare spending. ¹ Chronic disease management largely falls on patients and their families. Over 50 million family caregivers provide an estimated $470 billion in unpaid care in the U.S. ² Long-term burden and stress of caregiving negatively impact caregivers’ physical and mental health and ability to provide care. Caregiving support is highly fragmented and scarce, exacerbating health impacts associated with caregiving, especially for families residing in marginalized communities. ³ Cost-effective tools such as digital health allow caregivers to defer and mitigate public budgets such as hospitalization and long-term services and support. ⁴ With the rapid development of new technologies in the past decade, such as ubiquitous smart devices and advances in artificial intelligence, it is essential to review the current evidence of how these tools affect caregivers. Digital health refers to “the use of information and communications technologies in medicine and other health professions to manage illnesses and health risks and to promote wellness.” ⁵ It compasses mobile health, telehealth, wearable devices, health information system, and telemedicine. ⁵ So far, several existing systematic literature reviews have mainly focused on caregivers of adults or older adults with dementia, cancer, or late-life diseases.^5,9 Two systematic reviews focused only on web-based interventions.^6,7 All these systematic reviews found evidence that digital health tools had an overall positive effect on supporting caregivers and their well-being. The existing reviews also indicated that the accessibility, acceptability, and sustainability of digital health interventions for end-users need to be explored further, and more studies about integrating different health tool interventions need to be conducted. To develop digital health interventions, many intervention developers use Human-Centered Design (HCD) with a focus on designing technologies to serve the stakeholder (e.g., end-user).^8,9 The stakeholder's needs dominate the interface of the technology, which then impacts the design of the rest of the system. In HCD, designers consult with representative stakeholders from project inception to completion. The goal is to engage and enable stakeholders to understand and manage the technology in multiple situations and contexts. ¹⁰

In 2014, Chi and Demiris ¹¹ conducted a systematic review of the effectiveness of telehealth interventions to support family and informal caregivers. They reviewed technologies including video, web-based, telephone-based, and telemetry and they also categorized interventions as education, consultation, psychosocial behavioral therapy, social support, data collection, and care delivery. They found that more than 95% of the studies reported significant improvements in the caregivers’ outcomes and that caregivers were satisfied and comfortable with telehealth. Building on Chi and Demiris's work, we aimed to update their systematic review by expanding the search terms, broadening the search databases, and repositioning the categories of digital health tools to include more updated digital health interventions from 2014 to 2021. In addition, we also described the breadth and scope of the HCD design approaches used with these digital health interventions.

Methods

Design

Following the 2020 Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) for Systematic Reviews guidelines, ¹² we conducted a systematic literature search in bibliographic databases to identify caregiver interventions supported by various digital health tools. For the population of interest caregivers, we searched for “unpaid care provided by family, close relatives, friends, and neighbors.” ¹³

Inclusion and exclusion

We included modern technologies such as interactive web resources, web resources with nonexpert intervention, learning management system (LMS), telemonitoring system, online support group, videoconference, telehealth, mobile health, text message, and wearable technology (Table 1). Eligible studies had at least one caregiver-specific outcome or finding related to caregivers’ well-being and quality of life (QoL), such as their mental and physical health, skill, and self-care ability. We also limited the studies to Clinical Trials to avoid non-intervention studies. The study design of randomized controlled trials (RCTs), quasi-experimental, and mix-methods were included. Studies focused on paid professionals, professionally trained caregivers, and targeted care receivers were excluded. Studies were excluded if the intervention was not published in English, contained no human subjects, was only about HCD, had no caregiver self-care outcomes, were telephone-only interventions, were pilot studies with a sample size less than 30, or were protocol only.

Table 1.

Characteristics of included studies.

1a. Randomized Controlled Trial (RCT)

Title lead author/year/country	Caregiver identity and sample size	Patients’ age and conditions	Technology type and study settings	Interventions and duration	Caregiver outcomes and follow-up timeline	*Technology intervention delivery evaluation outcomes
Effectiveness of web-based versus folder support interventions for young informal carers of persons with mental illness: A randomized controlled trial Ali/2014/Sweden	Sibling and friend N  =  241	Adult (19–64 years) Mental Illness	Interactive web resource and Web resource with nonexpert interaction Home	A regularly updated website provided access to asynchronous information on advice and tips about taking care of themselves, where to turn to for help, how to know when it is time to seek help for patients; and information about mental illness (www.molnhopp.nu), a forum where they could have real-time discussions about optional topics and the opportunity to send questions to the support team, and a frequently updated blog where they could access the latest news. 12 weeks	Stress levels, caring situation, general self-efficacy, well-being, and quality of life. 8 months	Satisfaction and feasibility
Effectiveness of an Internet Intervention for Family Caregivers of People with Dementia: Results of a Randomized Controlled Trial Blom/2015/Netherlands	Spouse and children N  =  251	Older adults (65+ years) Dementia	Static web resource, Web resource with expert human interaction (e.g., learning management system), and Telemonitoring Home	Internet course Mastery over Dementia (MoD). MoD consists of 8 lessons and a booster session with the guidance of a coach monitoring the progress of participants and evaluating the homework. Each lesson has the same structure and consists of information (text material and videos), exercises, and homework, with an evaluation at the start and end of each session. The elements of the course are presented in the following order: coping with behavioral problems (problem-solving); relaxation; arranging help from others; changing non-helping thoughts into helping thoughts (cognitive restructuring); and communication with others (assertiveness training). The booster session is provided a month after participants finish the eight lessons, and it provides a summary of what has been learned. 8 weeks	Caregivers’ symptoms of depression and anxiety 6 months	NA
An intervention that reduces stress in people who combine work with informal care: randomized controlled trial results Boezeman/2018/Netherlands	Spouse, parents, children, siblings, and grandparents N  =  128	Children to older people	Static web resource and Web resource with nonexpert interaction (e.g., online support group) Home	The intervention was a role-focused self-help course (i.e., E-book plus a nonobligatory internet support module) developed for people who combine paid work with informal care. It contained exercises, texts, and practical suggestions. 8 weeks	Caregiver stress and distress 2 months	NA
Access to an online video enhances the consent process, increases knowledge, and decreases anxiety of caregivers with children scheduled for inguinal hernia repair: A randomized controlled study Book/2020/Germany	Parents N  =  50	Child (birth-18 years) Children undergoing inguinal hernia repair	Interactive web resource Academic pediatric surgery department	A standardized face-to-face consent dialogue was entertained in clinic, including technical aspects of the surgery, benefits, risks, possible complications, and expected outcome. After the clinic visit, the intervention group had unlimited access to the consent video. The 6-min video contained only information that was already given during the medical consultation. (6 min)	Anxiety, knowledge, and consent process. No follow-up assessment	Satisfaction
Effectiveness of a Blended Care Self-Management Program for Caregivers of People with Early-Stage Dementia (Partner in Balance): Randomized Controlled Trial Boots/2018/Netherlands	Spouse N  =  81	Older adults (65+ years) Dementia	Static web resource, Interactive web resource, Web resource with expert human interaction (e.g., learning management system), Web resource with nonexpert interaction (e.g., online support group), and Videoconferencing/telehealth Home	Blended care self-management Partner in Balance program (1) a face-to-face intake session with a personal coach to familiarize participants with the program, set goals, and select preferred module themes; (2) tailored online thematic modules, including psychoeducation, behavioral modeling, reflective assignments, change plans, and email feedback from the coach over 8 weeks; and (3) a face-to-face evaluation session with the coach evaluating previously set goals. 8 weeks	Self-efficacy (care management, service use), mastery, quality of life, depressive symptoms, anxiety, and perceived stress. No follow-up assessment	NA
Fulfilling the psychological and information need of the family members of critically ill patients using interactive mobile technology: A randomized controlled trial Chiang/2017/China	Spouse, children, and sibling N  =  74	Older adults (65+ years) critically ill patients	Interactive web resource, and Mobile health Adult intensive care unit	Education of families by tab (EF-T): was delivered to participants using interactive tabs. EF-T includes two parts, 1) the general information about the care patient in ICU, which would be provided during the initial time of ICU admission; and 2) the episodic information which would be disseminated and explained to caregivers according to different conditions of the patients. Duration: NA	Depression and anxiety No follow-up assessment	NA
Effect of MyTeen SMS-Based Mobile Intervention for Parents of Adolescents A Randomized Clinical Trial Chu/2019/New Zealand	Parents N  =  221	Adolescent (13–18 years) Depression and anxiety	Text messaging Home (community based)	MyTeen is a tailored program of text messages with evidence-based parenting tips for establishing and maintaining positive relationships with adolescents; strategies to increase adolescent autonomy; advice about adolescent development, family functioning, and parental self-care; information to help parents recognize depressive symptoms and understand treatment options; and links to evidence-based support and informational resources. 3 months	Parental competence, knowledge of help-seeking, parent–adolescent communication, and parental distress. 3 months	NA
Remotely Delivering Real-Time Parent Training to the Home: An Initial Randomized Trial of Internet-Delivered Parent-Child Interaction Therapy (I-PCIT) Comer/2017/United States	Parents N  =  40	Child (birth-18 years) Disruptive behavior disorders	Wearable technology and Videoconferencing Home	Internet-delivered parent-child interaction therapy (I-PCIT) follows traditional clinic-based PCIT but uses a VTC platform to enable therapists to remotely deliver treatment to families in their homes. Instead of interacting in front of a one-way mirror at a clinic, families use a webcam to broadcast home-based interactions to their therapists, who remotely provide live coaching through a parent-worn Bluetooth earpiece. 60 weeks	Children's symptoms, burden to parents, engagement, and treatment retention. 6 months	Satisfaction and barriers in using the technology
The impact of assistive technology on burden and psychological well-being in informal caregivers of people with dementia (ATTILA Study) Davies/2020/England UK	Relatives N  =  354	Older adults (65+ years) Dementia	Web resource with nonexpert interaction (e.g., online support group) Home	A semistructured needs assessment for ATT by a health or social care professional, followed by installation of ATT devices and response services as indicated by the assessment. 24 weeks	Caregivers’ burden, anxiety, and depression. 6 months
CHESS Improves Cancer Caregivers’ Burden and Mood: Results of an eHealth RCT DuBenske/2014/United States	Spouse N  =  285	Adult (19–64 years) Advanced non-small cell lung cancer	Interactive web resource, Web resource with expert human interaction (e.g., learning management system), and Web resource with non-expert interaction (e.g., online support group) Home	CHESS (Comprehensive Health Enhancement Support System) integrates services to facilitate coping by: (1) providing ready and organized access to information; (2) serving as a channel for communication and support with peers, experts, and users’ social networks; and (3) acting as an interactive coach by gathering information from the user, applying algorithms or decision rules, and providing feedback specifically relevant to the user. 24 months	Burden, negative mood, and disruptiveness. 6 months	Effectiveness
Web-Based Intervention for Family Carers of Persons with Dementia and Multiple Chronic Conditions (My Tools 4 Care): Pragmatic Randomized Controlled Trial Duggleby/2018/Canada	Spouse N  =  199	Older adults (65+ years) Dementia and other chronic conditions	Interactive web resource Home	My Tools 4 Care (MT4C) consists of six main sections: (1) about me, (2) common changes to expect, (3) frequently asked questions, (4) resources, (5) important health information, and (6) calendar. All data entered by participants into the site remained confidential, even from the study team. Participants also received an electronic copy of the Alzheimer's Society's The Progression of Alzheimer's Disease booklet, a copy of the study questionnaires, and the MT4C toolkit checklist intended for participants to record their use of the MT4C site. 3 months	Mental health, and hope 6 months	Effectiveness and feasibility
Feasibility Study of an Online Intervention to Support Male Spouses of Women with Breast Cancer Duggleby/2017/Canada	Spouse N  =  40	Adult (19–64 years) Breast Cancer	Interactive web resource Home	Male Transition Toolkit (MaTT) is a web-based intervention consisted of six sections (a) about me, (b) common changes to expect, (c) frequently asked questions, (d) resources, (e) calendar, and (f) important health information. 4 weeks	Quality-of-life, and guilt 4 weeks	Efficiency, feasibility, acceptability, and learnability
Efficacy of a Smartphone App Intervention for Reducing Caregiver Stress: Randomized Controlled Trial Fuller-Tyszkiewicz/2020/Australia	Children, spouse, and friend N  =  183	Children, adolescents, adults, and older adults Physical and/or mental disability	Mobile health Home (community based)	The intervention provides psychoeducation (through text, video, audio, and graphics) and a series of interactive exercises or activities. The intervention comprises 5 modules: (1) introduction on psychoeducation about stress reduction and third-wave CBT; (2) values clarification and goal setting; (3) mindfulness skills; (4) well-being enhancement; and (5) behavioral activation to increase engagement and enjoyment. A Troubleshooting tab was also available, which contained a series of activities to help with stress (e.g., destress with a body scan and breathing to diffuse negative thoughts). 5 weeks	Depression. emotional well-being, optimism, self-esteem, support from family, support from significant others, and subjective well-being. 3 months	Satisfaction, engagement, functionality, aesthetics, and information
Web-Based STAR E-Learning Course Increases Empathy and Understanding in Dementia Caregivers: Results from a Randomized Controlled Trial in the Netherlands and the United Kingdom Hattink/2015/Netherlands and the United Kingdom	Spouse, children, and sibling N  =  142	Older adults (65+ years) Dementia	Static web resource, interactive web resource, Web resource with expert human interaction, Web resource with nonexpert interaction Home	Empathy and Understanding (EU) Skills Training and Reskilling (STAR) project include: 1, a collection of 8 modules on different topics in dementia care. 2, a Learning Path Advisor through an online tool integrated in STAR that assesses baseline knowledge and confidence to help people decide at which point to start the course; and 3. Facebook and LinkedIn communities to promote peer support and provide opportunities to contact other dementia care professionals. 2–4 months	Positive attitudes to dementia; distress, empathic concern, and taking the perspective of the person with dementia. 4 months	Satisfaction and learnability
Enhancing Interactions during Daily Routines: A Randomized Controlled Trial of a Web-Based Tutorial for Parents of Young Children with ASD Ibañez/2018/United States	Parents N  =  104	Children (birth-18 years) Autism spectrum disorder (ASD)	Interactive web resource, Web resource with expert human interaction, and Web resource with non-expert interaction (ex. online support group) Home	The tutorial comprises three main sections: (1) introductory material (e.g., definition of home routines; tips for establishing routines); (2) description and illustration of four daily routines-bath time, snack time, play time, and bed time-which includes both general information and individualized content; and (3) “toolbox” modules describing specific, evidence-based behavioral strategies for enhancing children's cooperation and participation with routines. 6 hours	Use of evidence-based instructional strategies, child engagement parenting stress. parenting efficacy, and child social communication 2 months	Satisfaction
Online support for employed informal caregivers: Psychosocial outcomes Klemm/2014/ United States	Spouse, parents, and children N  =  86	Adult (19–64 years) Dementia, diabetes, stroke, cancer, and heart disease.	Web resource with expert human interaction and Web resource with nonexpert interaction Home	The professionally facilitated/psychoeducational online support groups (OSGs) were led by a member of the research team in a semistructured format for a period of 12 weeks using asynchronous communication. Each week, the facilitator introduced a different discussion topic that was pertinent to caregiving. Topics (e.g., caregiving tips, depression, fatigue, financial concerns) were chosen based on research that focused on family caregivers of people with chronic illness. Moderated/peer-directed groups were run in a self-help format over 12 weeks with no set agenda or weekly topics (peer directed) and were monitored by the principal investigator (PI).12 weeks	Depressive symptoms and quality of life 12 weeks	NA
Health-Related Quality of Life Outcomes of a Telehealth Care Coordination Intervention for Children with Medical Complexity: A Randomized Controlled Trial Looman/2018/United States	Parents N  =  163	Child (birth-18 years) Complex chronic conditions (CCCs)	Videoconferencing/telehealth Ambulatory care clinic	The intervention groups included telephone group and telephone plus video group. Families in the telephone plus video group were provided with a netbook with Webcam and high-speed Internet access, which they could use in addition to telephone contact. The care coordinator responded to calls from families of children in the study for acute and chronic condition management and provided proactive care coordination and case management services for these families. 30 months	Child quality of life and family impact 24 months	NA
Acceptance and treatment effects of an internet-delivered cognitive-behavioral intervention for family caregivers of people with dementia: A randomized controlled trial Meichsner/2019/Germany	Spouse and children N  =  37	Older adults (65+ years) Dementia	Web resource with expert human interaction Home	The intervention was delivered via a secure internet platform, the Tele.TAnDem. online blog where they could read messages from their therapist and post replies. The manual follows cognitive-behavioral principles and consists of 10 therapy modules: basic elements, problem analysis, psychoeducation, Strengthening problem-solving abilities, Changing dysfunctional cognitions, Increasing the use of informal and/or professional support, Self-care, creating value-based activities, Stress management, and emotion regulation strategies and evaluation. 2 months	Caregivers’ well-being, able to cope with the anticipated death, and utilized psychosocial resources. 3 months	NA
Randomized Controlled Trial of a Facilitated Online Positive Emotion Regulation Intervention for Dementia Caregivers Moskowitz/2019/United States	Spouse, children, and friend N  =  155	Older adults (65+ years) Dementia	Videoconferencing/telehealth Home	The intervention condition consisted of six sessions in which a facilitator taught participants a set of eight emotion regulation skills intended to increase positive emotion. Prior to beginning sessions, the participants were sent a package that contained study materials and an 8-inch tablet computer. Session 1 noticing positive events, capitalizing on them, and gratitude; session 2 mindfulness; session 3 positive reappraisal; session 4 personal strengths and attainable goal setting; session 5 acts of kindness; and session 6 practice and suggestions. 6 sessions	Depression and quality of life, anxiety, physical health, positive emotion, and positive aspects of caregiving 6 months	NA
Effect of a Videoconference-Based Online Group Intervention for Traumatic Stress in Parents of Children with Life-threatening Illness A Randomized Clinical Trial Muscara/2020/Australia	Parents N  =  81	Child (birth-18 years) Life-threatening illness or injury	Web resource with expert human interaction The oncology or cardiology departments or the pediatric intensive care unit (PICU)	It is based on acceptance and commitment therapy (ACT). The program is a 6-session, parent-mediated, psychological intervention based on ACT It consists of five 90-min consecutive weekly sessions. Parents participated from their homes using the Google Hangouts videoconferencing application (Alphabet) on a study-provided iPad (16 GB with Retina Display Wi-Fi + 3G; Apple). 2 months	Posttraumatic Stress No follow-up	NA
Distance-Delivered Parent Training for Childhood Disruptive Behavior (Strongest Families™): a Randomized Controlled Trial and Economic Analysis Olthuis/2018/Canada	Parents N  =  172	Child (birth-18 years) Disruptive behavior disorders	Static web resource Home	Caregivers in the intervention group received the Strongest Families™ (SF™) behavioral intervention for disruptive behavior that includes 12 skill-based sessions delivered by written information and videos sent by mail and by telephone coaching. Parents were mailed a handbook (Parenting the Active Child; for intervention content), a skill demonstration DVD, and other program materials (e.g., behavioral chart, school daily report card). Each family was assigned a coach (based on a match between the availability of parents and coaches) who completed 12 weekly 30–40 min telephone sessions with the parent. 12 weeks	Distress. 22 months	NA
Older adult caregivers of their spouses with acquired late-life disability: examining the effectiveness of an internet-based meditation program in mitigating stress and promoting well-being Pandya/2020/South Asian	Spouse N  =  192	Older adults (65+ years) Late-life disability	Web resource with expert human interaction Home	Internet-based meditation program group (IMP) contended: Prayer or sitting in simple silence (5 min), Instant Relaxation Technique (IRT) in supine position with isometric contraction of the muscles (5 min), Tree posture (i.e., standing still) and centering in tree posture (5 min), Deep breathing and focusing on flow of breath and the rhythm, Deep relaxation technique (DRT) in supine position (5 min). Online leisure program cohort (OLP) session was taught by social worker with online sessions with cellular data Session content (any one of the following in a single session): Playing the following games/solving puzzles - Sudoku, Crossword, Scrabble with online partners, watching chat show, watching any entertainment TV program, Listening to music, Watching movie clips/documentaries 50 weeks	Caregiving burden, responses to care challenges, distress, and well-being. No follow-up	NA
Long-Term Caregiver Mental Health Outcomes Following a Predominately Online Intervention for Adolescents With Complicated Mild to Severe Traumatic Brain Injury Petranovich/2015/United States	Parents N  =  132	Adolescent (13–18 years) Complicated Mild to Severe Traumatic Brain Injury	Videoconferencing/telehealth Home	Families in the Counselor-Assisted Problem Solving (CAPS) condition were provided a family problem-solving intervention that focused on training in problem-solving, communication skills, self-regulation, and anger management. After the initial Skype session with the therapist, each CAPS session consisted of a didactic online module that the family completed without the psychologist's assistance. After reviewing the modules, the family and therapist participated in a Skype session, at which time they discussed the module and ways to implement the problem-solving process around the family's goal(s). 6 months	Psychological distress 18 months	NA
In-person vs. web-based administration of a problem-solving skills intervention for parents of children with cancer: Report of a randomized noninferiority trial Phipps/2020/United States	Parents N  =  621	Child (birth-18 years) Childhood cancer	Interactive web resource and web resource with expert human interaction Home	Bright IDEAS(BI): the term “Bright” signifies optimism and instills the belief that problems can be solved, which is considered an essential component for successful implementation of the intervention. The acronym “IDEAS” is used as a mnemonic for the 5 essential steps of our problem-solving approach, with each letter signifying a step: I (Identify the problem), D (Determine the options), E (Evaluate/choose the best option), A (Act), and S (See if it worked). The content of the web-based intervention included modeling videos, interactive activities, and homework/worksheet tools, included 7 videos presenting the steps of problem-solving via the interaction between a clinician and parent, and 4 fotovellas depicting parents working through typical problems encountered. The fotonovelas incorporated interactive skills practice-brief practice exercises for each step of the BI method. 2 months	Problem-solving 3 months	Efficiency
A Randomized Trial of Mobile Health Support for Heart Failure Patients and Their Informal Caregivers: Impacts on Caregiver-Reported Outcomes Piette/2015/United States	Children, sibling, and friend N  =  369	Older adults (65+ years) Heart Failure	Static web resource, Mobile health, and Videoconferencing/telehealth Home	“Mobile health  +  CarePartner” (mHealth  +  CP) patients received identical services, plus email summaries and suggestions for self-care assistance automatically sent to their CarePartners. 12 months	Caregiving strain and depressive symptoms 12 months	Effectiveness
Web-Based Parenting Skills Program for Pediatric Traumatic Brain Injury Reduces Psychological Distress Among Lower-Income Parents Raj/2015/United States	Parents and grandparent N  =  37	Child (birth-18 years) TBI	Interactive web resource and Videoconferencing/telehealth Home	Internet-based Interacting Together Everyday: Recovery After Childhood TBI [I-InTERACT] was a Web-Based Parenting Skills Program that comprised of 10 core sessions and up to 4 supplemental sessions. The first part was a self-guided Web session addressing the following topics: positive parenting skills (session 2), staying positive and managing stress (session 3), behavior management following TBI (session 4), giving good commands (session 5), dealing with anger (session 6), consequences for not following direction (session 7), cognitive problems following TBI (session 8), establishing house rules and using positive parenting skills in daily life (session 9), and planning for future transitions (session 10). The second part consisted of a videoconference call with the therapist to discuss content from the Web-based session and practice positive parenting skills while receiving “bug-inthe-ear” feedback from the therapist 4–6 months	Psychological distress, depression, parenting stress, and caregiver efficacy. 6 months	NA
Randomized controlled trial of family-based education for patients with heart failure and their carers Srisuk/2016/Thailand	Spouse, children, and sibling N  =  100	Older adults (65+ years) Heart Failure	Static web resource and Telemonitoring Hospital outpatient clinic	The family-based education program comprised one face-to-face education counseling session, provision of a heart failure manual and DVD and telephone follow-up. 6 months	Knowledge and perceived control 6 months	NA
Effectiveness and Usability of a Web-Based Mindfulness Intervention for Families Living with Mental Illness Stjernswärd/2017/Sweden	Spouse, parents, children, and sibling N  =  156	Adult (19–64 years) Mental Illness	Static web resource Home	It consists of audio/video files (960 min) accompanied by written keywords on the screen, descriptive text files, and instructions for daily mindfulness exercises, including (self) compassion exercises, a time log, and a private diary (not visible to the researchers). 10 weeks	Mindfulness self-compassion, perceived stress, and caregiver burden 3 months	Effectiveness, efficiency, acceptability, and error frequency
Teletherapy Delivery of Caregiver Behavior Training for Children with Attention-Deficit Hyperactivity Disorder Tse/2015/United States	Parents N  =  37	Child (birth-18 years) Attention-Deficit Hyperactivity Disorder (ADHD)	Videoconferencing/telehealth Home	All participants received consensus-based pharmacotherapy with a child and adolescent telepsychiatrist The second component was a manualized caregiver behavior training intervention for ADHD delivered to each caregiver individually by community therapists who were trained and supervised remotely by a clinical psychologist 6 weeks	Distress. 6 months	Feasibility, acceptability
Evaluation of DEM-DISC, customized e-advice on health and social support services for informal carers and case managers of people with dementia; a cluster randomized trial Van Mierlo/2015/Netherlands	Spouse, children N  =  73	Adult (19–64 years) Dementia	Interactive web resource and web resource with expert human interaction Home	DEM-DISC is a DEMentia-specific. Digital Social Chart that provides information on health and care services, tailored to the needs of people with dementia and carers, and is easily accessible, anywhere and anytime 12 months	Sense of competence, sense of competence, quality of life, emotional distress. 12 months	Efficiency, feasibility, and learnability
The Children's Attention-Deficit Hyperactivity Disorder Telemental Health Treatment Study: Caregiver Outcomes Vander Stoep/2017/United States	Parents N  =  88	child (birth-18 years) ADHD	Videoconferencing/telehealth Community clinics	Synchronous videoconferencing was used to provide access to child psychiatry services including pharmacotherapy. Multiple asynchronous telehealth technologies were used to enhance the skills of master's level community therapists to deliver in-person an evidence-based caregiver behavior training intervention for children with ADHD. 22 weeks	Parents’ distress, parenting stress, caregiver strain, and family empowerment. 6 months	NA
Counselor-Assisted Problem Solving Improves Caregiver Efficacy Following Adolescent Brain Injury Wade/2014/United States	Parents N  =  132	Adolescent (13–18 years) TBI	Static web resource, Videoconferencing/telehealth Home	The core sessions primarily focused on problem-solving and application to a goal or problem identified by the family. After generating a solution to the goal or problem that the family initially identified as problematic, the family's homework was to implement the agreed-upon plan and evaluate its success. Subsequent core sessions focused on developing strategies to facilitate effective family problem-solving, including basic communication skills and developing strategies to address common cognitive and behavioral consequences of TBI. 6 months	Caregiving self-efficacy and depression. 6 months	Time spent engagement

* Technology Intervention Delivery Evaluation Outcomes: 1, effectiveness (The accuracy and completeness with which users achieve specified goals); 2, efficiency (Once users have learned the design, how quickly can they perform tasks?); 3, satisfaction (After using the technology does the user have a good feeling about the technology?); 4, feasibility (How users perceive the ability to integrate the technology into their lives?); 5, acceptability (How willing are the users to use this technology?); 6, learnability (How easy is it for users to accomplish basic tasks the first time they encounter the design?); 7, memorability (Can a returning user remember how to effectively use technology?); 8, error frequency (How many errors do users make, how severe are these errors, and how easily can they recover from the errors?); and 9, understandability of content (How well users understands the content?).

Search strategy

The librarian in our team (FC) performed the comprehensive searches in July 2019 and repeated in January 2021. The search was conducted in PubMed, CINAHL, Embase, Cochrane Library, PsycINFO, ERIC, and ACM Digital Library databases with date limits of 2014–2021. A combination of controlled vocabularies and keywords was used in the search strategy. A full search strategy with filters and limits for each database can be seen in the supplementary files.

Study selection

Two authors (SZ and FC) followed two steps of data extraction. In the first step, each author independently assessed the titles and abstracts of the references identified in the search and determined whether they followed the study's objective and met the inclusion criteria. The results (include, exclude, and not sure) were compared and discussed by two reviewers until a consensus was reached, and then they read the full-text articles. In the second step, full-text articles were reviewed and assessed individually, compared, and discussed among two reviewers, and the potential articles were categorized in a list The search identified 2350 records after de-duplication with Endnote ¹⁴ and Rayyan. ¹⁵ Using Rayyan, two researchers (SZ and FC) reviewed 2350 records and excluded 2177 records. A total number of 173 full-text articles were reviewed for data abstraction.

Methodological quality appraisal

The researchers used the Mixed Methods Appraisal Tool (MMAT) ¹⁴ to screen the articles and ensure the study was of the desired quality. MMAT is a tool used as a checklist for appraising the methodological quality of various empirical research methods, including qualitative, RCTs, quantitative nonrandomized, quantitative descriptive, and mixed methods. We used MMAT to include randomized controlled and nonrandomized trials that could be prescreened by different evaluation items. Two raters (SZ and FC) independently assessed the methodological quality of the included studies through the MMAT checklist. Discrepancies were resolved through discussion until an agreement was reached. We used MMAT to evaluate the study designs, the quality of the data collection, and analytic methods.

Certainty of evidence

We applied Grading of Recommendations, Assessment, Development and Evaluations (GRADE) to assess the quality of the body of evidence separately for the prespecified outcome.^16,17 As defined by the GRADE working group, high certainty refers to the authors being very confident that the true effect lies close to that of the estimate of the effect; moderate certainty means that the authors are moderately confident and the effects are likely to be close to not only estimating of the impact but also possible that it is different; low certainty means the confidence in the effect estimate is limited; and very low certainty refers to the authors have least confidence in the effect estimate.

Data synthesis

Data were abstracted and evaluated using Research Electronic Data Capture, a secure web application for designing clinical/translational research and collecting research data.^18,19 Similar to Chi and Demiris's work, we extracted information on the relationship between the caregiver, care receiver, age group of the care recipient, patient condition, technology intervention categories, specific caregiver outcomes, and study limitations reported in this paper. Moreover, we expanded and deepened our review of detailed information about each study, including sample size, content, duration, follow-up timeline, the effectiveness of each intervention, and description of measurement tools for each outcome (Table 1). The researchers also abstracted whether the studies described the study design, evaluated their technology intervention, and assessed for HCD concepts of effectiveness, efficiency, satisfaction, feasibility, acceptability, learnability, memorability, error frequency, and understandability of content. ²⁰

Results

Following PRISMA guidelines, the search and review process identified 2551 records in 2021 and 2350 records after de-duplication with Endnote and Rayyan. Using Rayyan, two researchers (SZ and FC) reviewed all 2350 records and excluded 2177 records. Of the 173 full-text articles, 133 studies were excluded, and 40 full-text articles were included for review and quality assessment (Figure 1). The studies were published in 34 different journals from 10 fields (Figure 2). The most common field was medicine. The papers were from 19 countries, and the U.S. was the most common country (Figure 3).

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) 2020 flow diagram for new systematic reviews which included searches of databases and registers only.

Figure 2.

Journal fields.

Figure 3.

Country of studies conducted.

Study settings and design

Thirty (75%) studies were conducted online at participants’ homes or community settings, nine (23%) were conducted at a hospital or clinic, and one study (2%) in an academic center. Thirty-two (80%) studies are RCTs, and eight (20%) are quasi-experimental designs.

Sample and sample size

For the care recipients’ ages, 21 studies (53%) focused only on adults and older people, 18 articles (44%) focused on children and adolescents, and one study (3%) focused on care recipients of all ages. The care recipients had various conditions. For adults and older people, conditions included mental illness,^21,22 dementia,^7,23,31 Alzheimer's disease, ³² cancer,^31,33,35 critically ill or late-life disability,^36,38 and heart disease.^31,39,40 For children and adolescents, conditions included Anorexia Nervosa, ⁴¹ Attention-Deficit Hyperactivity Disorder,^42,44 Autism Spectrum Disorder,^45,46 life-threatening illness or injury,^47,48 disruptive behavior disorders,^49,50 undergoing repair or surgery,^51,53 depression and anxiety, ⁵⁴ and traumatic brain injury.^48,55,56 Caregivers’ relationships with the patients included relatives and nonrelatives. Specifically, 20 studies included spouses or intimate partner caregivers. Parents as caregivers were in 26 studies. Children or children-in-law as a caregiver were included in 13 studies, while nine studies had siblings as caregivers. Three studies had grandparents as caregivers, and seven studies included friends or volunteers.

Study quality appraisal and effects of interventions

Table 2 reported the quality appraisal for each included study, including screening questions for both quantitative RCT and non-RCT. Table 3 shows the results of the certainty of evidence. SPSS was used to calculate the inter-rater reliability score of 0.9 (IBM SPSS Statistics Version 27). For the screening questions, all included studies presented clear research questions and collected appropriate data to address the questions. Among the 32 studies that used RCT, most addressed randomization (n  =  28, 88%), ensured intervention adherence (n  =  27, 84%), and completed more than 90% of outcome collection compared to groups at baseline (n  =  27, 84%). Fewer studies reported blinded assessors for the outcomes (n  =  17, 53%). For eight that had a nonrandomized study component, the majority had a good completion rate (80%), administrated intervention as intended (n  =  8, 100%), and used appropriate sampling methods and measures (n  =  8, 100%). Seven studies considered confounders in design and analysis (n  =  7, 87%).

Table 2.

Risk of bias (RoB) in individual study.

Lead Author/Year	Screen		Quantitative RCT					Quantitative Non-RCT
	S1	S2	D1	D2	D3	D4	D5	D6	D7	D8	D9	D10
Ali/2014
Bartels/2019
Blom/2015
Boezeman/2018
Book/2020
Boots/2018
Chiang/2017
Chu/2019
Comer/2017
Davies/2020
DuBenske/2014
Duggleby/2018
Duggleby/2017/Canada
Fernandes/2015
Fuller-Tyszkiewicz/2020
Hattink/2015
Ibañez/2018
Klemm/2014
Kuravackel/2017
Looman/2018
Meichsner/2019
McKechnie/2015
Moskowitz/2019
Muscara/2020
Northouse/2014
Olthuis/2018
Pandya/2020
Petranovich/2015
Phipps/2020
Piette/2015
Raj/2015
Srisuk/2016
Stjernswärd/2017
Trail/2020
Tse/2015
Truttmann/2020
Van Mierlo/2015
Vander Stoep/2017
Wade/2014
Yu/2019

• Screening Questions

S1. Are there clear research questions?

S2. Do the collected data allow to address the research questions?

• Quantitative randomized controlled trials

D1: Is randomization appropriately performed?

D2: Are the groups comparable at baseline?

D3: Are there complete outcome data?

D4: Are outcome assessors blinded to the intervention provided?

D5: Did the participants adhere to the assigned intervention?

• Quantitative nonrandomized study

D6: Are the participants representative of the target population?

D7: Are measurements appropriate regarding both the outcome and intervention (or exposure)?

D8: Are there complete outcome data?

D9: Are the confounders accounted for in the design and analysis?

D10: During the study period, is the intervention administered (or exposure occurred) as intended?

*Judgement: Low risk; High risk; Unclear risk

Table 3.

GRADE evidence profile (EP) for included studies.

Study	Quality Assessment of Evidence
	Limitation	Inconsistency	Indirectness	Imprecision	Publication bias	Quality
Ali/2014	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Bartels/2019	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Blom/2015	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Boezeman/2018	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Book/2020	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ◯ ◯ Low
Boots/2018	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Chiang/2017	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Chu/2019	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Comer/2017	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Davies/2020	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
DuBenske/2014	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Duggleby/2018	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Duggleby/2017	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ◯ ◯ Low
Fernandes/2015	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Fuller-Tyszkiewicz/2020	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ◯ ◯ Low
Hattink/2015	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Ibañez/2018	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Klemm/2014	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Kuravackel/2017	No serious limitation	No serious inconsistency	No serious indirectness	serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Looman/2018	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ◯ ◯ Low
Meichsner/2019	serious limitation	No serious inconsistency	No serious indirectness	serious imprecision	Undetected	⊕ ⊕ ◯ ◯ Low
McKechnie/2015	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Moskowitz/2019	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Muscara/2020	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Northouse/2014	No serious limitation	No serious inconsistency	No serious indirectness	serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Olthuis/2018	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Pandya/2020	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Petranovich/2015	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Phipps/2020	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Piette/2015	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Raj/2015	serious limitation	No serious inconsistency	No serious indirectness	serious imprecision	Undetected	⊕ ◯ ◯ ◯ Very low
Srisuk/2016	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Stjernswärd/2017	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ◯ ◯ Low
Trail/2020	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Tse/2015	serious limitation	No serious inconsistency	No serious indirectness	serious imprecision	Undetected	⊕ ◯ ◯ ◯ Very low
Truttmann/2020	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Van Mierlo/2015	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Vander Stoep/2017	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate
Wade/2014	No serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕⊕ High
Yu/2019	serious limitation	No serious inconsistency	No serious indirectness	No serious imprecision	Undetected	⊕ ⊕ ⊕ ◯ Moderate

We have presented the GRADE evidence profile for included studies in Table 3. Among 40 included interventions, 37.5% (n  =  15) were rated high, 42.5% (n  =  17) were rated moderate, 15% (n  =  6) were rated low, and 5% (n  =  2) were rated very low.

Theoretical frameworks informing intervention content

Thirty studies (75%) had a theoretical underpinning or guiding framework for the intervention's design. For example,adult learning theory principles,^40,45 social learning theory and self-regulation theory,^25,39,49,57 cognitive behavioral therapy (CBT) framework,^28,42 acceptance and commitment theory,^22,58 transition theory,^7,34 stress and coping theory,^25,33 and information provision model. ⁵²

Technology used to deliver intervention

Three main domains of digital health technologies emerged from our review: telehealth, telemedicine, and mHealth. ⁵⁹ The following sections will describe each type of digital health tool.

Telehealth refers to delivering and facilitating medical care, provider and patient education, health information services, and self-care via remote and digital communication technologies. ⁶⁰ Telehealth includes four types of resources: (1) Static web resource: a web page delivered to the user's browser exactly as it is stored, which means all users receive the same information. The static web page was used in nine studies^{22,24,25,27,39,40,50,55,61}; (2) Interactive Web Resources, which refer to the webpage that requires interaction/engagement from users to understand, execute, or experience by themselves. Fifteen studies used interactive web resources^{7,21,25,27,30,33,36,41,45,48,51,52,62}; (3) Web resources with expert human interaction mainly refer to the LMS, which is a software application for the administration and delivery of educational programs. ⁶³ LMS was used in 11 articles^{24,25,27,28,30,31,33,37,45,58,62}; (4) Web resource with non-expert interaction, defined as “any virtual social space where people come together to get and give information or support, to learn, or to find the company,” such as an online support group. ⁶⁴ Communication with nonexpert interaction was often through bulletin/discussion boards or specific software for live interaction with other group members.^{21,25,27,31,33,45,61} The second domain that emerged from our review was telemedicine. Telemedicine refers to two-way, real-time interactive communication between educator/counselor/doctor/nurses and client/patient. ⁶⁵ Telemedicine was used in 11 studies.^{25,27,31,33,37,46,48,49,58} The third domain was mHealth or Mobile health, referred to as conducting health services, information, and data collection through mobile phones, tablet computers, or personal digital assistants (PDA). ⁶⁶ mHealth was employed in six studies.^{7,23,24,36,39,42} Text messaging was used in two studies.^53,54 Additionally, wearable technology, which detects, analyzes, and transmits information worn by the wearer via their mobile devices,^67,68 was used in one study. ⁵⁰

Digital health delivered interventions

Based on the five categories that Chi and Demiris ¹¹ developed, we summarized the purpose of the intervention and what digital health tools supported the intervention delivery (Table 1). Table 4 also summarizes the intervention purposes for each study.

Table 4.

Intervention categories.

Lead Author	Education	Real-time communication	Data collection and monitoring	Psychotherapy	Connection and support
Ali		X
Bartels			X
Blom	X			X
Boezeman	X
Book	X
Boots	X	X
Chiang	X
Chu	X
Comer	X	X
Davies			X
DuBenske	X	X			X
Duggleby	X		X
Duggleby	X		X
Fernandes	X
Fuller-Tyszkiewicz				X
Hattink	X	X			X
Ibañez	X
Klemm		X			X
Kuravackel	X	X
Looman		X
McKechnie					X
Meichsner		X		X
Moskowitz	X	X
Muscara	X			X	X
Northouse	X
Olthuis	X
Pandya	X	X
Petranovich	X			X
Phipps	X			X
Piette	X	X	X
Raj	X	X
Srisuk	X	X
Stjernswärd	X
Trail					X
Truttmann	X	X
Tse	X	X
Van Mierlo	X	X	X
Vander Stoep	X	X
Wade	X	X		X
Yu	X	X

Education. One of the primary purposes of intervention is education. Educational interventions focused on improving self-help strategies and practice, gaining knowledge of patients’ diseases, getting familiar with medical/hospital procedures, and fostering a positive relationship between caregivers and patients. The delivery formats included interactive web-based programs, multimedia applications, LMS, and videoconferencing. This category accounted for 78% of the interventions.

Real-time Communication. The interventions compassed real-time communication as the main component. Caregivers asked questions, obtained information for decision-making, and consulted on better self-care through real-time communication with the coach/therapist/coordinator. The digital health tools for real-time communication included videoconferencing, instant messaging, discussion boards, and robotic telepresence. Studies using online support groups usually adopted a real-time communication approach. For example, Hattink et al. used Facebook® and LinkedIn® communities to provide opportunities to contact other dementia care professionals and peers.

Data Collection and Monitoring. This type of intervention aims for continuous data collection, analysis, and monitoring. The most popular system is Assistive Technology and Telecare (ATT). ATT offers innovative methods of supporting people by reducing potential harmful risks, increasing independence, and improving communication and QoL. The system has sensors that detect falls, gas leaking, and movement, and alerting devices to transfer data/information to the support center.^68,69 This approach is often used among older adults with dementia who want to live independently. Davies et al. used the full version of ATT, which involves installing equipment to continuously, automatically, and remotely monitor real-time emergencies and lifestyle changes among older adults with dementia living at home. ²⁶

Psychotherapy. Psychotherapy is embodied in CBT, and aims to support caregivers in reducing stress, gaining coping strategies, and improving problem-solving skills. Coach feedback or support groups were often combined with CBT to better help caregivers.^27,28,33,55 Researchers often employ static or interactive web resources integrated with nonexpert interaction or mobile health web resources to deliver psychotherapy.

Connection and Support. Social support is an approach to exchanging information, connecting with caregivers with a similar situation, and obtaining support from a peer or an expert to increase the sense of hope, mastery, self-efficacy, and compassion. Social support mainly occurred on social media sites, and password-protected web page discussion forums could significantly connect and encourage caregivers. For example, McKechnie et al. used Talking Point (the UK Alzheimer's Society's online platform) for informal caregivers of people with dementia. This forum acted as a place for anyone affected by dementia to ask for advice, share information, and join the discussion to feel supported. ³²

HCD/UCD methods and outcomes

Of the 40 studies, 18 (45%) included some HCD methods. All 18 studies evaluated the usability and satisfaction of the technology delivering the intervention,^{7,22,27,30,33,35,42,45,51,55,62,70,75} while two studies included design and evaluation methods for their intervention.^30,62 Van Mierlo et al. ³⁰ utilized potential users for information with no direct input from these users about the design of the intervention. In contrast, Phipps et al. ⁶³ specifically stated that they used a user-centered design process that included focus groups for formative development of the intervention and representative users to review the intervention. ⁶²

In the 18 studies that included an evaluative component for their intervention, the usability concepts ranked from most frequent to least frequent were: Satisfaction (9), Feasibility (8), acceptability (6), effectiveness (6), Learnability (6), Efficiency (3), Error Frequency (2), Understandability (1), and Memorability (0). Some researchers explicitly called out concepts of Usefulness, Usability, Ease of Use, Engagement, Functionality, Aesthetics, and Content which overlap with the previously stated ideas. The specific questionnaires utilized for evaluating the technology in the intervention were satisfaction questionnaires (3), usability questionnaires (4), and two questionnaires/questions developed for the study (Table 5). Three studies measured technology use, examining time spent engaged on the website,^33,55 number of logins, ³³ number of page views, ³³ or attendance. ⁴³ One study used interviews to evaluate the intervention (7), while one used interviews and a questionnaire. ³⁰

Table 5.

Study instruments.

Caring/Parenting

Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
COPE-index	Ali/2014	The COPE index contains three sub-scales: (1) the negative impact of caring; (2) the positive value of caring; and (3) the quality of support. All questions in the COPE-index are answered on a 4-point scale ranging from “1  =  never” to “4  =  always.” The COPE-index has good reliability for the negative impact (Cronbach’s alpha  =  0.83), acceptable for positive value (0.66), and acceptable for quality of support (0.64)
Alabama Parenting Questionnaire (APQ)	Olthuis/2018	The APQ, a 42-item questionnaire for parents of 6- to 13-year-olds, was used to evaluate parenting practices. Parents indicate the frequency with which they engage in a number of parenting behaviors (e.g., You take away privileges or money from your child as a punishment^) on a 5-point Likert scale (1  =  Never to 5  =  Always). We used monitoring, positive parenting, and inconsistent discipline subscales. Items in each subscale were averaged to create mean subscale scores. The average reliability across these subscales is 0.68; in the present study, Cronbach's alphas were 0.55 for monitoring, 0.75 for positive parenting, and 0.68 for inconsistent discipline. The APQ has adequate validity in identifying parents of children with disruptive behavior and is widely used to study parenting program outcomes.
Self-Efficacy
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
General self-efficacy (GSE)	Ali/2014 Duggleby/2018	GSE measures individuals’ beliefs in their abilities to manage specific situations. The questions are all answered on a 4-point scale ranging from “1  =  not applicable” to “4  =  always true” [51]. The GSE has good internal consistency (Cronbach's alpha  =  0.87).
Caregiver Self-Efficacy Scale	Boots/2018 Petranovich/2015 Raj/2015 Wade/2014	It measures care management self-efficacy (4 items) and service use self-efficacy (5 items). Care management self-efficacy scores theoretically range from 4–40 and service use self-efficacy from 5–50. Higher scores on the CSES indicate higher levels of self-efficacy.
Parental Efficacy Scale (PES)	Ibañez/2018	Ten items measuring parenting-related efficacy (e.g., understanding what the child wants, knowing what the child enjoys) are rated on a four-point Likert scale ranging from “not good at all (1)” to “very good (4).” The PES has strong psychometric properties and has been used in many studies involving children with ASD; it demonstrated acceptable internal consistency reliability (α  =  0.78) with the current sample. Items are summed to obtain a total score that ranges from 10 to 40, with higher scores reflecting higher levels of perceived parenting efficacy.
17-item version Lewis Cancer Self-efficacy Scale	Northouse/2014	It assessed patients’ and caregivers’ confidence about managing the illness (e.g., “I am confident that I can use information and resources to cope with the demands of cancer”). Evidence of content and criterion validity of the original scale was reported by Lewis. The modified version of the scale demonstrated high internal consistency reliability in our prior studies. The alpha reliability coefficient in this study was 0.95.
Well-Being
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Well-being Index (WHO-5)	Ali/2014	The well-being index (WHO-5) was used, has 5 positively worded items related to vitality, positive mood, and general interests. Measured on a 6-point scale ranging from “0  =  not present” to “5  =  constantly present.” The scale has a good reliability (Cronbach's alpha  =  0.87). General health was measured with one self-rated health question commonly used in research, “How would you say your health is in general?” The single question is answered on a 5-point scale ranging from “1  =  excellent” to “5  =  bad.” Another single question used to measure quality of life, “Overall, how would you rate your quality of life (life situation) in the last two weeks?” is measured on a 5-point scale ranging from “1  =  very good” to “5  =  very bad.”
Global Health Scale	Moskowitz/2019	The 10 PROMIS items used in this analysis included overall self-rated health; overall quality of life; overall physical health; overall mental health; and individual items on fatigue, pain, emotional distress, and social activities and roles. Most questions asked about a person's experience “in general,” with items on fatigue, pain, and emotional problems experienced during the past 7 days. Psychometric evaluation of the PROMIS global health items was based on two global physical health (GPH) and global mental health (GMH) scales. The PROMIS GPH scale included four items that rated overall physical health (physical functioning, physical activities, pain, and fatigue). GPH and GMH total raw scores were computed by summing item scores that ranged from 1 to 5, such that higher scores reflected better functioning and are then rescaled to a mean of 50 and an SD of 10 using nationally normative data from the U.S. general population. The estimated correlation between the GPH and GMH was 0.63.
Warwick-Edinburgh Mental Well-being Scale (WEMWBS)	Pandya/2020	The WEMWBS is a 14-item scale designed to measure positive mental health or mental well-being. It comprises both hedonic elements (happiness, joy, contentment) and eudaimonic elements (psychological functioning, autonomy, positive relationships with others, and sense of purpose in life). Respondents are asked to describe their experience on each statement in the past 2 weeks, rated on a 5-point scale ranging from 1  =  none of the time to 5  =  all of the time. Total scores range from 14 to 70; higher scores indicating greater well-being.
Mental Health Inventory-38 (MHI-38)	Pandya/2020	The MHI-38 is a widely used self-rated 38-item questionnaire rating symptoms or states of mind over the past month. It consists of two global scales: psychological distress (24 items) and psychological well-being (14 items). All items except two are rated on a 6-point scale; items 9 and 28 are rated on a 5-point scale. There are six subscales: general positive affect, emotional ties, life satisfaction, anxiety, depression, and loss of behavioral/emotional control.
Depression and Anxiety
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Center for Epidemiological Studies Depression Scale (CES-D)	Bartels/2019 Blom/2015 Boots/2018 Davies/2020 Dowling/2014 Klemm/2014 Meichsner/2019 Petranovich/2015 Piette/2015 Raj/2015 Wade/2014	a 20-item inventory that provides an assessment of specific symptoms of depression. For this measure, caregivers rated the frequency of depressive symptoms over the past week, including depressed mood and social withdrawal. The CES-D has well-established psychometric properties, including high internal consistency, adequate test–retest reliability, and high degree of reliability and validity. Validations studies further suggest that the CESD has a high correspondence with clinical ratings of depressive symptoms and is useful for screening individuals at risk for developing clinical depression. A raw score of 16 or greater indicates clinically significant depressive symptoms. Cronbach's alpha across the four time points indicated that the measure was highly reliable (a  =  .92).
Hospital Anxiety and Depression Scale (HADS)	Bartels/2019 Blom/2015 Boots/2018	It is a 14-item measure designed to assess anxiety and depression symptoms in medical patients, with emphasis on reducing the impact of physical illness on the total score. The depression items tend to focus on the anhedonic symptoms of depression. Items are rated on a 4-point severity scale. The HADS produces two scales, one for anxiety (HADS–A) and one for depression (HADS–D), differentiating the two states. Scores of greater than or equal to 11 on either scale indicate a definitive case.
State-Trait-Anxiety inventory (STAI)	Blom/2015 Book/2020 Davies/2020 Fernandes/2015 Yu/2019	It comprises two categories and 40 items, with 20 items for each category. Each item was rated on a 4-point Likert scale. To ensure consistency of meaning, positive items are reversed for calculating the total score; the total score ranges from 20–80, and a higher score indicates higher anxiety. Trait anxiety was measured only once before the intervention. The Cronbach's α for the state and trait anxiety tool in this study was .93 and .93, respectively.
Depression Anxiety Stress Scale − Chinese (C − DASS)	Chiang/2017 Fuller-Tyszkiewicz/2020	It has 21 items and was originally developed to measure the emotional states of depression, anxiety, and stress with three respective subscales. The instrument has good psychometric properties and has been widely used. A validated Chinese version of C − DASS and in particular the subscales of stress and anxiety was used in this study as the indicators of emotional and psychological responses possible association with EF-T. Emotional state of each subscale in C − DASS is measured on a 4-point Likert scale from 0 (does not apply to me at all) to 3 (applied to me very much, or most of the time). The total scores for each scale ranged from 0 to 21, and higher scores indicated a higher level of depression, anxiety, and stress.
Short Version Profile of Mood States (SV-POMS)	DuBenske/2014 Phipps/2020	It was used to assess caregiver negative mood. Items were selected to be representative of the three negative mood subscales: (a) Tension-Anxiety (tense, on edge, uneasy, nervous, anxious); (b) Anger-Hostility (annoyed, angry, grouchy, furious, bitter); and (c) Depression-Dejection (discouraged, helpless, hopeless, sad, unhappy, worthless).
7-item Generalized Anxiety Disorder scale (GAD-7)	McKechnie/2014	is a widely used 7-item measure of anxiety. Scores of 15 or greater indicate severe anxiety. It has good sensitivity and specificity for GAD and is a valid and reliable measure for detecting GAD in the general population, as well as social anxiety, panic disorder, and posttraumatic stress disorder.
Emotional Distress-Depression, PROMIS Item Bank, v. 1.0	Moskowitz/2019 Northouse/2014	Used to assess depression. Participants rated 28 items (α = .95) focused on depressive symptoms over the past 7 days. The items included in the final bank specifically focus on negative mood, decreases in positive emotions, cognitive deficits, negative self-image, and negative social cognition. The items are scored on a 5-point verbal response scale (i.e., ordered categorical item responses) where respondents are asked to rate the experienced frequency of symptoms (never, rarely, sometimes, often, always).
Quality of Life in Neurological Disorders (NeuroQOL)	Moskowitz/2019	Measure contains 29 items (α = .95) to tap anxiety over the past 7 days. Neuro-QoL provided raw scores which were converted to T-Scores; with a T  =  50 indicating average function compared to the reference population and a standard deviation of 10.
Revised Memory and Behavioral Problem Checklist (RMBPC)	Blom/2015	RMBPC included 24 items of caregivers’ distress related to memory and behavior problems. A mean product score ranging from 0 to 16 was calculated by adding up per item the product score of frequency times the level of perceived stress indicator, divided by the number of behavioral problems.
Self-Perceived Pressure from Informal Care scale	Blom/2015	14-item Self-Perceived Pressure from Informal Care Scale9 with items, such as “Generally speaking I felt very pressured because of the situation of my care recipient” and “I was too tired to do anything in my free time in the period that I was providing help.” Responses were coded as 0  =  disagree and 1  =  (somewhat) agree. Item scores were added to produce a total score ranging from 0 (no burden) to 14 (heavy burden; H value  =  0.87, Cronbach’s alpha  =  .86).
5-point 16-item distress scale of the 4DSQ	Boezeman/2018	An example item is “During the past week, did you feel tense?.” After measurement, the procedure for aggregating the distress-scores was used (i.e., 3-point scale; “No”  =  0, “Sometimes”  =  1, all other responses  =  2).28 The scale ranges from 0 to 32; a high score indicates a high level of distress.
Depression, Anxiety, and Stress Scale-21（(DASS-21)	Muscara/2020 Olthuis/2018	DASS-21 total score was used to evaluate current symptoms of parental depression, anxiety, and stress (i.e., distress). The DASS-21 is a 21-item self-report measure that asks individuals to indicate the extent to which a particular emotional state (e.g., BI found myself getting agitated^) has applied to them over the past week (0  =  did not apply to me at all to 3  =  applied to me very much, or most of the time). The DASS-21 demonstrates good internal consistency (α  =  0.93), has good convergent and discriminant validity with standardized measures of anxiety and mood symptoms, and is sensitive to the effects of parenting interventions.
Global Severity Index (GSI) of the Symptom Checklist-90-R (SCL-90-R)	Petranovich/2015 Raj/2015 Truttmann/2020 Wade/2014	The SCL-90-R is a 90-item self-report inventory that assesses a range of clinical symptomatology including depression (e.g., depressed mood, social withdrawal) and anxiety (e.g., somatic symptoms of anxiety, symptoms of social anxiety). The SCL-90-R has well-documented reliability and validity and is sensitive to the presence of significant psychological distress. The SCL90-R GSI is reported as a T-score with a mean of 50 and a standard deviation of 10. Scores greater or equal to 63 are considered to be indicative of clinically significant levels of distress. Cronbach's alpha across the four time points indicated that this measure was highly reliable (a  =  .97).
Patient Health Questionnaire (PHQ)	McKechnie/2014 Phipps/2020 Trail/2020 Truttmann/2020 Tse/2015 Vander Stoep/2017	is a widely used 9-item measure of depression. Scores of 20 or more suggest severe depression. It has high sensitivity and specificity for diagnosing depression], good internal consistency. convergent and discriminant validity, robustness of factor structure, and responsiveness to change.
Sense of Competence
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Short Sense of Competence Questionnaire	Bartels/2019 Blom/2015 Hattink/2015 Van Mierlo/2015	SSCQ reflects the carer's sensation of being capable to care for the person with dementia. It consists of seven items and total scores range from 7 to 35. The construct validity of this instrument was supported by a high Person correlation (0.88) between the SSCQ and the original Sense of Competence Questionnaire as well as a high reliability (Cronbach's alpha of .89)
PSOC Scale	Chu/2019	measures parental self-esteem on 2 dimensions: satisfaction (feelings associated with parenting, as anxiety or frustration) and efficacy (perceived ability and confidence in handling parenting problems). These constructs are closely linked with both positive family interactions and positive child development. The total PSOC score is calculated as the sum of 17 items and has a possible range of 17–102. The PSOC has substantial strengths, including good content validity, internal consistency (α  =  .80), normative data, test–retest reliability (0.73–0.74), 25 and indicators of both convergent and discriminant validity.
Sense of Mastery
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Pearlin Mastery Scale	Bartels/2019 Blom/2015 Boots/2018	PM measures an individual's level of mastery, which is a psychological resource that has been defined as “the extent to which one regards one's life chances as being under one's own control in contrast to being fatalistically ruled.” The 7-item scale comprises five negatively worded items and two positively worded items, presented with the following response options: (1) Strongly Disagree (2) Disagree (3) Agree (4) Strongly Agree. The negatively worded items require reverse coding prior to scoring, resulting in a score range of 7–28, with higher scores indicating greater levels of mastery.
Quality of Life
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Investigation Choice Experiments for the Preferences of Older People	Boots/2018	This index value indicates how good or bad the average person aged 65 or older considers a given state to be, for instance attributing to “attachment” (love and friendship) and “control” (independence). The value system for the 1024 possible states uses a best-worst scaling valuation method, providing a single summary score, anchored at zero (“no capability”) and 1.0 (“full capability”).
Caregiver Quality of Life – Cancer Scale (CQOLC)	DuBenske/2014 Duggleby/2018	The CQOLC is a self-administered scale specifically designed to evaluate cancer patient caregiver QoL. This scale includes 35 items. CQOLC responses are scored from 0 (not at all) to 4 (very much). The total possible score is 140, with higher scores representing better QOL. In the original study, evidence supported four subscales: Burden, disruptiveness, positive adaptation, and financial concern. These four factors include 27 items, with 8 additional items not loading onto these factors. Adequate internal consistency has been demonstrated, with internal consistency values (Cronbach's alpha) for the four subscales being 0.89, 0.83, 0.73, and 0.81, with a value of 0.90 for total CQOLC scores.
Caregiver Quality-of-Life (CQoL-I) index	Klemm/2014	The Quality of Life Index is a general QoL index that covers five dimensions: activity, daily living, health, support of family and friends, and outlook. This is one of the earliest QoL instruments to measure activity level, social support, and mental well-being. Each item is rated on a three-point scale (0–2), with the total scores ranging from 0 to 10. Higher scores reflect better performance (2); lower scores (0) indicate poor quality of life.
Scale for the Quality of the Current Relationship in Caregiving (SQCRC)	McKechnie/2014	asks carers about their relationship with the person that they are caring for, giving equal weight to positive and negative aspects. A higher score implies the presence of warmth and affection and the absence of conflict and criticism in the relationship. The measure has high internal consistency and good face validity but has had little further psychometric investigation.
Functional Assessment of Cancer Therapy (FACT-G version 4)	Northouse/2014	a 27-item cancer-specific instrument that assesses QOL and four domains: emotional, social, functional, and physical well-being. The FACT-Ghas been used frequently in studies with cancer patients. The alpha coefficient for the total QOL score was 0.90 in this study and alpha coefficients for the subscales ranged from 0.71 to 0.83. Higher scores indicated better QOL.
Minnesota Living with Heart Failure (MLHF) questionnaire	Srisuk/2016	It consists of 21 questions focused on patients’ perceptions concerning the effects of HF on their physical functioning, such as shortness of breath, fatigue, and peripheral edema, and their emotional life such as memory loss, loss of self-control and side effects of HF treatment. The Thai version has been used in 422 HF patients and in pilot testing in a sample of 30, had good reliability (Cronbach's alpha coefficient was 0.94. The MLHF was applied only to patients with HF and not the carers. In this study reliability of the MLHF was 0.87.
Short-Form 12 health survey (SF-12)	Srisuk/2016	The 12 items include the self-assessment of health, physical functioning, physical role limitation, mental role limitation, social functioning, mental health, and pain. The summary scores provide an indication of physical and emotional functioning, with higher scores indicating better health-related quality of life (HRQL). The reliability of the SF-12 in this study was 0.86.
Communication
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Society of Critical Care Medicine's Family Needs Assessment Questionnaire (SCCMFNA)	Chiang/2017	The tool Communication and Physical Comfort Scale (CPCS) therefore included a total of seven items with a total maximum score of 28, with the minimum score of 7 regarding satisfaction after EF-T. Since the EF-R group did not receive the education via tab, the CPCS for them did not have the two questions about their satisfaction regarding the use of tabs for educating them about the patient condition. Maximum score of CPCS for the EF-R group was 20, and the lowest score was 5.
Parent Adolescent Communication Scale	Chu/2019	This scale consists of 20 items and generates a total score and 2 subscale scores (open family communication and problems in family communication). The scale has good internal reliabilities for both subscales (0.87 and 0.78, respectively) and test–retest reliabilities (0.78 and 0.77, respectively).
23-item version of the Lewis Mutuality and Interpersonal Sensitivity Scale	Northouse/2014	Assessed patients’ and caregivers’ degree of open communication and ability to share feelings about the illness (e.g., “We are comfortable sharing feelings about cancer with each other”). The original scale developed by Lewis has established validity and demonstrated high internal consistency in our prior studies with patients and caregivers. The alpha reliability of the scale in this study was 0.93.
Knowledge
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Mental Health Literacy Scale	Chu/2019	The subscale consists of 4 items, rated on a 5-point scale, ranging from strongly disagree to strongly agree. The scale has demonstrated good internal and test–retest reliability, and scores are significantly correlated with help-seeking intentions.
Approaches to Dementia Questionnaire (ADQ)	Hattink/2015	The latter questionnaire was also administered among informal caregivers with one question omitted (“It is important not to become too attached to people with dementia”) because it was deemed inappropriate. The ADQ consisted of 19 questions on attitudes toward dementia and could be answered on a 5-point scale ranging from “completely agree” to “completely disagree” (range 19–95), such as “People with dementia are like children.”
Being a Parent Scale (BPS)	Kuravackel/2017	The BPS is a 16-item questionnaire measuring parents’ views of their own competence as parents. Dimensions include satisfaction with their parenting role (reflecting the extent of frustration, anxiety, and motivation) and feelings of self-efficacy as a parent (reflecting competence, problem-solving ability, and capability in parenting role). Items that comprise the two factors, Satisfaction (9 items) and Efficacy (7 items), are scored on a 6-point Likert scale (Strongly agree to Strongly disagree). In the current sample, total score reliability estimates at pre- and post-assessment were 0.85 and 0.87, respectively.
Dutch Heart Failure Knowledge Scale (DHFKS)	Srisuk/2016	The questionnaire consists of 15 items assessing HF knowledge: four items regarding general HF information, six items regarding HF treatment (diet, fluid restrictions, and activity), and five items assessing symptoms and symptom recognition. In this study internal consistency, measured by Cronbach’s alpha, was 0.61 for patient and for the carers was 0.34.
Parents’ knowledge about postdischarge home care	Yu/2019	In this study, items related to pain management (e.g., a scheduled around-the-clock administration of pain medication rather than as needed, instruction to not tolerate pain, nonpharmacological methods, such as listening to music or watching TV, staying with the child or hugging) were added and the healing period was modified from 3–4 to 2–3 weeks by experts’ advice on the basis of their clinical experience. The tool comprises 19 true–false type questions, for example, “It is better to gargle instead of brushing for the second day after the tonsillectomy,” and “It is helpful to eat soft and cold porridge for a week after a tonsillectomy,” with each item given a score of 1 for the correct answer and a score of 0 for a wrong answer or the response “I do not know.” The total score ranged from 0–19, and a higher score indicates a higher level of knowledge. The Kuder–Richardson Formula 20 was 0.63 in the current study.
Barriers to Treatment
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Barriers to Treatment Participation Scale	Comer/2017	44-item parent-report measure of perceived barriers to treatment participation. Items are rated along 5-point scales and assess stressors and obstacles that compete with treatment (e.g., transportation, scheduling), treatment demands issues (e.g., uncomfortable treatment setting), and attitudes about treatment and the therapist (e.g., treatment is not working). Tallying the items yields a total barriers score (.88 in present sample).
Treatment Satisfaction
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Client Satisfaction Questionnaire (CSQ-8)	Comer/2017	Self-report statement of satisfaction with health and human services. For the overall score, sum item responses, range from 8–32, higher score indicates higher satisfaction. Questions range from “To what extent has our program met your needs?” to “Have the services you received helped you to deal more effectively with your problems?” and are answered on a scale of one to four. The instrument has good concurrent validity and internal consistency with alphas between .86 and .94.
Therapy Attitude Inventory	Comer/2017	The TAI is a brief measure of parent satisfaction with parent training or family therapy. The TAI includes 10 items addressing the impact of therapy on parenting skills and child behavior. Parents rate items on a scale from 1 (indicating dissatisfaction with treatment, or a worsening of problems) to 5 (indicating maximum satisfaction with treatment or improvement of problems). The item ratings are summed to yield a total score between 10 and 50.
Consultation Satisfaction Questionnaire (CSQ)	Kuravackel/2017 Meichsner/2019	Consultee satisfaction was assessed using a multiple-item satisfaction questionnaire. Ratings were based on a 4-point scale (1  =  strongly disagree, to 4  =  strongly agree) and each questionnaire was unique to the individual session, ranging from 10 to 11 questions on each form. Sample items include: “I felt involved during the session and able to express my views”; “The therapist's communication skills were effective”; “The therapist was knowledgeable about autism.” This questionnaire was completed by parents after each session and scores were averaged across each treatment group. Sample internal consistency in the current study ranged from 0.92 to 0.96 across sessions.
Group Session Rating Scale (GSRS)	Kuravackel/2017	The GSRS, adapted from the Session Rating Scale, is a 4-item visual analog scale, designed to be a brief clinical tool to measure group-therapy alliance. The GSRS was completed by each participating caregiver at the end of each group session, to determine the quality of group alliance depending on treatment condition. The items are based on a response using a ten-centimeter line. The “relationship” aspect is assessed on a continuum of “I felt understood, respected, and accepted by the leader and the group” to “I did not feel understood, respected.” The “goals and topics” aspect is assessed on a continuum of “We worked on and talked about what I wanted to work on and talk about” to “We did not work on or talk about what I wanted to work on and/or talk about.” The acceptability of the approach used in the group is assessed on a continuum of “The leader and group's approach is a good fit for me” to “The leader and/or group's approach is not a good fit for me.” A sense of overall fit is assessed on a continuum ranging from “Overall, today's groups was right for me - I felt like a part of the group” to “There was something missing in group today—I did not feel like a part of the group.” Scores are summed out of a total possible score of 40 and averaged over the 4 group sessions for each participant and averaged across each treatment group per session. The GSRS shows evidence of concurrent validity, correlating with other individual alliance measures with coefficients ranging from 0.41 to 0.61 and Cronbach’s alphas ranging from 0.86 to 0.90 over four sessions (Quirk et al. 2013). In the current study, reliability for the GSRS across the four sessions ranged from 0.82 (group session 1) to 0.96 (group session 3).
Client Satisfaction Questionnaire (CSQ) modified for ADHD	Tse/2015	Self-reported measure of caregivers’ satisfaction with services received and their perception of the clinician's understanding of their children's treatment needs. The CSQ ADHD consists of 10 items rated on a 4-point Likert scale. Satisfaction ratings range from 0 to 40, with higher scores indicating greater satisfaction with care. Attkisson CC, Zwick R. The Client Satisfaction Questionnaire: Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann 1982;5:233–237.
Caregiver Stress/burden
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Zarit Burden Interview	Davies/2020 Dowling/2014 Moskowitz/2019	The Zarit Burden Interview15 is a 22-item scale assessing burden of caregiving. Participants respond on a 5-point Likert-type scale ranging from 0 (never) to 4 (always), to generate a single score with higher scores indicating greater burden. Scores 0–20 indicate little or no burden, 21–40 mild to moderate burden, 41–60 moderate to severe burden, and 61–88 indicating severe burden.
Modified Caregiver Strain Index (CSI)	Klemm/2014	It consists of 13 items designed to measure the effects of caregiving in major domains that include employment, finances, physical, social, and time. Scores range from 0 to 26, with higher scores indicating higher caregiver stress. Internal consistency reliability was reported between .86 and .90.
Burden Scale for Family Caregivers-short form (BSFC-s)	Pandya/2020	Comprises 10-item (e.g., reduced life satisfaction, physical exhaustion, depersonalization, caregiving is taking strength) rated on a scale from 0 (strongly disagree) to 3 (strongly agree). The score ranges from 0 to 30 and a high degree of agreement indicates higher subjective burden for the caregiver. For the present study: Cronbach’s α  =  .92; item-scale intercorrelation  =  .89; Pearson's r  =  .89.
Perceived Stress Scale (PSS)	Ali/2014 Bartels/2019 Chu/2019 Dowling/2014 Moskowitz/2019	The PSS describes individuals’ perceptions of their lives through the degree of prediction and control they have and whether they are overloaded. It contains 14 items answered on a 5-point scale ranging from “0  =  never” to “4  =  very often.” The reliability of the PSS-scale measured by Cronbach's alpha is 0.82.
NEO Five Factory Inventory (NEO-FFI)	Boots/2018	It was used to identify individuals who are prone to psychological distress, by assessing 6 traits: anxiety, angry hostility, depression, self-consciousness, impulsiveness, and vulnerability. Scores ranged from 0–24; where higher scorers are likely to be sensitive, emotional, and more prone to experiencing feelings that are upsetting.
13-item Modified Caregiver Strain Index	Boezeman/2018 Piette/2015	An example item is “I feel completely overwhelmed (e.g., I worry about the person I care for; I have concerns about how I will manage).” The scale score ranges from 0 to 26; a high score indicates a high level of caregiver stress.
Parenting stress index (PSI)	Ibañez/2018 Kuravackel/2017 Raj/2015 Stjernswärd/2017 Tse/2015 Vander Stoep/2017	The PSI/SF consists of 36 items measuring three domains of parental stress: (a) Parental Distress (PD); (b) Parent-Child Dysfunctional Interaction (PCDI); and (c) Difficult Child (DC). The PSI/SF has strong psychometric properties and has been used extensively with families of children with disabilities, including ASD; the three domains demonstrated good internal consistency reliability (α  =  0.84–0.91) with the current sample. Each domain contains 12 items that are scored on a five-point Likert scale ranging from “strongly disagree (1)” to “strongly agree (5).” Scores for each domain range from 12 to 60, with higher scores indicating higher levels of parenting-related stress.
CarerQoL7-D	Stjernswärd/2017	This self-rating instrument measures seven dimensions of caregiver burden using seven items with a 3-point response scale (0  =  no problems to 3  =  a lot of problems): fulfillment, relational dimension, mental health dimension, social dimension, financial dimension, perceived support, and physical dimension. It includes the CarerQoL-VAS, indicating the level of happiness with caregivers’ experiences and encompassing both negative and positive aspects, ranging from 0  =  completely unhappy to 10  =  completely happy.
Caregiver Strain Questionnaire (CGSQ)	Tse/2015 Vander Stoep/2017	CGSQ has 21 items rated on a 5-point Likert scale that assess “demands, responsibilities, difficulties and negative psychic consequences of caring for a relative with special needs.”50,51 Higher scores indicate greater strain. Internal consistency of the CGSQ was 0.92.
Neuropsychiatric Inventory (NPI)	Dowling/2014 Van Mierlo/2015	This instrument assesses 12 neurobehavioral domains and the severity of caregiver's distress. The presence or absence of the behaviors in each domain is determined by a yes or no screening. If respondents answer affirmatively, further questions are asked to rate the behavior in terms of frequency, severity, and caregiver distress. The behavioral domain score is the product of the frequency and severity. A total scale score is calculated by adding the domain scores. The total caregiver distress score is the sum of each domain rating.
Edmonton Symptom Assessment Scale (ESAS)	DuBenske/2014	The ESAS measures distress arising from 9 physical and psychological problems and 1 overall “distress” item on a 0–10 scale, with 10 indicating highest symptom burden. This modified scale included 6 original symptoms: 4 physical (pain, nausea, appetite, and shortness of breath) and 2 psychological (depression, anxiety). Based on feedback from study oncologists, 3 other physical symptoms (activity, drowsy, and well-being) were replaced with symptoms common in lung cancer: fatigue, constipation, and diarrhea. The overall symptom distress scale score was calculated by summing the scores of the 9 symptom items (pretest α  =  .79).
Guilt
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Caregiver Guilt Questionnaire (CGQ)	Duggleby/2018	CGQ was used to measure perceptions of guilt using a 22-item, five-point scale with a maximum total score of 8. It has been used in a study of male spouses of women with breast cancer with reported internal consistency of r  =  0.93.
Affective Mood
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Homeostatically Protected Mood Scale	Fuller-Tyszkiewicz/2020	Respondents were asked to rate how well 3 positive affective terms (content, happy, and alert) describe their feelings about their life in general, rated using an 11-point scale, ranging from 0 (not at all) to 10 (extremely), with higher scores on each indicating that higher affective mood. In this study, internal consistency ranged from 0.75 to 0.87 across groups and time.
Differential Emotions Scale (DES)	Dowling/2014 Moskowitz/2019	This 20-item version of the DES was modified to include additional positive affect items as well as those that are likely to tap into trait positive affectivity. The full scale assesses interest, enjoyment, surprise, sadness, anger, disgust, contempt, fear, guilt, shame, shyness, amusement, awe, contentment, gratitude, hope, love, pride, sympathy, and sexual feelings (e.g., “I felt sexual, desiring, and flirtatious”). The scale is scored for total positive and negative affect, grouped according to where respondents would fall on the circumplex model of affect (eg, high activation vs. lower activation). This modified DES has shown acceptable reliability with the positive affects subscale (a  =  0.79) and the negative affects subscale (a  =  0.69).
Herth Hope Index	Duggleby/2018	The Herth Hope Index (HHI) is a 12-item, four-point Likert-type scale that delineates three factors of hope: (a) temporality and future, (b) positive readiness and expectancy, and (c) interconnectedness. A higher score denotes higher hope, with a maximum score of 48. The HHI takes about five minutes to complete and has been demonstrated to have good reliability (test–retest r  =  0.91, p < 0.05) and validity (concurrent validity, r  =  0.84, p < 0.05; criterion, r  =  0.092, p < 0.05; divergent, r  =  −0.73, p < 0.05) in patients with cancer.
Self-Esteem
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Rosenberg Self-Esteem Scale	Fuller-Tyszkiewicz/2020	This scale consists of 10 items assessing self-esteem (eg, “At times I think I am no good at all”), with response options completed using a 4-point scale, ranging from 1 (strongly disagree) to 4 (strongly agree). In an Australian sample, the measure has demonstrated excellent test–retest reliability (rs  =  0.53–0.69 over 4 years) and internal consistency (α >  .85) and was shown to correlate with constructs theoretically related to self-esteem, such as self-compassion (rs  =  0.36–0.63). For this study, Rosenberg's original 5 positive items were included, thereby a single construct best described as positive self-esteem, with higher scores indicating higher self-esteem. In this study, internal consistency ranged from 0.65 to 0.86 across groups and time.
Cognitive and Behavioral Strategies/Skills
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Primary and Secondary Control Scale (PSCS)	Fuller-Tyszkiewicz/2020	The PSCS consists of 25 items assessing specific cognitive and behavioral strategies aimed at either control of environmental circumstances (primary control; eg, “when bad things happen, I put lots of time into overcoming it”) or control of internal states (secondary control; eg, “when bad things happen, I ignore it by thinking about other things”), to minimize psychological impacts. The response options were completed using an 11-point scale, ranging from 0 (do not agree at all) to 10 (agree completely), with higher scores indicating higher primary/secondary control. In this study, internal consistency ranged from 0.75 to 0.87 for primary control and from 0.61 to 0.76 for secondary control across groups and time.
Parent Behavior Survey	Ibañez/2018	This survey comprised 15 specific parent behaviors that examined parents verbal instructions/directions (e.g., using simple instructions, repeating verbal directions if the child fails to comply [reverse coded], praising the child for completing all or part of a routine), use of visual supports (e.g., use of pictures and timer), and their approach to structuring the routine steps. After reverse coding, the one undesired behavior, higher scores on each item indicated more optimal use of evidence-based strategies. The items demonstrated acceptable internal consistency reliability, α  =  0.79. Items are averaged to obtain a mean total score that ranges from 1 to 4, with higher scores indicating increased use of desired strategies.
14-item Functional Status II(R)measure	Looman/2017	Functional status of the child was operationalized as a score between 0 and 100 on the 14-item Functional Status II(R) measure, on which higher scores indicate better functional status. This measure inventory behavioral manifestations of a health condition that interferes with an individual's performance on age-appropriate activities and is intended for use with children ages 2–18 years. The items are scored as a percentage of the total possible points (0with children ages 24-item Functional Status II(R) measure, on which higher scores indliability for this measure in our sample was 0.78.
Questionnaire for Family Caregivers of People with Dementia	Meichsner/2019	The 36neraire for Family Caregivers of People with Dementiaesources related to Well Caregivers , sources related to Well Caregivand physical fitness”), Resources related to Coping with Daily Hassles (e.g., sources related to Well Caregivand physical fitness”), Resources related to Coping with Dai, sources related to Well Caregivand physical fitness”),  =  never to 5  =  very often) is used to indicate the frequency with which resources have been utilized during the last four weeks. Internal consistencies for the scales are satisfactory with Cronbach's alpha ranging from 0.76 to 0.87.
Five Facet Mindfulness Questionnaire-Short Form	Muscara/2020 Stjernswärd/2017	It consists of 39 items, rated on a 5-point Likert scale (1  =  never or very rarely true, 5  =  very often or always true), assessing five facets of mindfulness: Non-reactivity to inner experience (7 items), Observing (8 items), Acting with Awareness (8 items), Describing (8 items), and Non-judging of Experience (8 items). Cronbach's alpha for FFMQ in the current study was 0.92.
Perceived Change Index-13 (PCI-13)	Pandya/2020	The PCI-13 is a brief 13-item measure that captures whether caregivers perceive care challenges and their own well-being as improving, worsening, or staying the same reflecting back over a 1-month period. A 5-point scale is used to rate whether a caregiver's life situation has become worse (1) or improved (5) over the past month. Examples of scale items include caregivers’ ability to sleep through the night, ability to manage day-to-day caregiving, and feelings of being overwhelmed. Scores range from 13 to 65, higher scores indicating greater improvement in caregiver's life situation. For the present study, Cronbach’s alpha  =  .90; item-scale intercorrelation  =  .87; Pearson's r  =  .89.
Social Problem-Solving Inventory, Revised (SPSI-R)	Phipps/2020	SPSI-R is a widely used, well-validated 52-item measure of problem-solving skills, which assesses two dimensions of problem orientation (positive vs. negative) and three dimensions of approach to problems (rational; impulsive-careless; avoidance), and also yields a total score. Internal consistency in the current trial was excellent (a  =  0.95).
Family Empowerment Scale (FES)	Tse/2015 Vander Stoep/2017	FES consists of 34 items that indicate caregiversls, which assesses two dimensions's health problems and their ability to advocate for their needs. The five response options range from “never” to “very often.” Higher scores indicate greater empowerment in advocating for children's needs. Internal consistency was 0.93.
Social Support
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Multidimensional Scale of Perceived Social Support	Fuller-Tyszkiewicz/2020	This comprises 12 items assessing the perceived adequacy of support from family, friends, and significant other (eg, s. negative) and three dimensions of approach to prmfort to me”). Responses are recorded on a 7-point scale, ranging from 1 (very strongly disagree) to 7 (very strongly agree), and scoring is calculated for 3 subscales reflecting the 3 social support sources of (1) family, (2) friends, and (3) significant other. Higher scores indicate higher perceived social support from each social support source. In this study, subscale-level internal consistency estimates ranged from 0.75 to 0.93 for family support, from 0.80 to 0.92 for support from friends, and from 0.80 to 0.93 for social support from others.
9-item, brief version of the Social Support Scale	Northouse/2014	Assessed patients rief version of the Social Support Scaleone another (e.g., e.g.notheref version of the Social Support Scale illnesshereEvidence of concurrent validity and high internal consistency was reported with the original, longer version of the scale. The alpha reliability was 0.87 in the present study. Self-efficacy was assessed with the 17-item version Lewis Cancer Self-efficacy Scale that assessed patients’ and caregivers’ confidence about managing the illness (e.g., he alpha reliability was 0.87 in the present study. Self-efficacy was assessed with the 17-item version Lewis Cancer Self-efficacy Scale that assessed p
Empathy
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Interpersonal Reactivity Index (IRI)	Hattink/2015	This questionnaire consists of 28 items that were answered on a 5-point scale ranging from “does not describe me well” to “describes me very well” and with 4 subscales: (1) perspective taking (tendency to adopt the psychological point of view of others), (2) fantasy (tendency to imagine oneself into fictitious characters in books and movies), (3) empathic concern (“other-oriented” feelings of sympathy and concern for unfortunate others), and (4) personal distress (“selforiented” feelings of anxiety and unease in tense interpersonal settings). The range was 0–28 for each subscale.
Self-Compassion Scale-Short Form (SCS-SF)	Stjernswärd/2017	This 12-item scale measures six components of self-compassion using six subscales with two items each: Self-Kindness, Self-Judgment, Common Humanity, Isolation, Mindfulness, and OverIdentification. Items are rated on a 5-point response scale (1  =  almost never to 5  =  almost always). A short version was used in the present study, for which Cronbach's alpha in the current study was 0.86.
Impact of Health Conditions
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Family Impact Module (FIM)	Looman/2017	Six subscales (28 items) in the FIM assess parent functioning in the following six domains: physical, emotional, social, cognitive, communication, and worry. Two scales measure daily activities (e.g., household tasks, 3 items) and family relationships (e.g., solving problems, 5 items). All scales use a 5-point response scale with responses ranging from never a problem to almost always a problem, reverse-scored and linearly transformed to a 100-point scale, with higher values representing higher functioning. The developers of the PedsQL measure report that a difference of 4.5 points on the PedsQL measure is a minimally clinically important difference. Internal consistency reliabilities for all subscales and total scales were at or above 0.70 in our sample.
Coping
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Caregiver Grief Scale (CGS)	Meichsner/2019	Used to assess caregivers’ coping with predeath grief. The scale consists of 11 items (e.g., “It burdens me not to be able to talk to her/him anymore.”) that are rated on a five-point Likert scale (1  =  strongly disagree to 5  =  strongly agree). A total score can be computed along with scores on the four subscales Emotional Pain, Relational Loss, Absolute Loss, and Acceptance of Loss. The scale has demonstrated a satisfactory internal consistency (Cronbach's alpha  =  0.89).
Posttraumatic Stress Disorder
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Posttraumatic Stress Disorder Checklist-Version 5 (PCL-5)	Muscara/2020	It contains 20 items that assessed 20 criteria for posttraumatic stress disorder in the Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition). Parents were asked to complete the measure in relation to their child's diagnosis. The total score (range, 0–80) was used, with higher scores indicating greater PTSS. Internal consistency for the total score in the current study was α  =  .93.
The Impact of Events Scale-Revised (IES-R)	Phipps/2020	IES-R included the three subscales: intrusion (eight items), avoidance (eight items), and hyperarousal (six items); we used only the hyperarousal subscale. The 5-point Likert scale response options were used (0–4): not true at all (0), rarely true (1), sometimes true (2), often true (3), and true nearly all of the time (4). The score ranges are from 0 to 24, and higher scores indicate more stress.
Appraisal
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
11-item, modified Benefits of Illness Scale	Northouse/2014	Assessed benefits arising from the illness or caregiving experience (e.g., “Having cancer has brought my family closer together”). The original scale was developed by Tomich and Helgeson who examined the psychometric properties of the scale and reported it was a unidimensional scale with high internal consistency 32. The alpha coefficient in this study was 0.92. Higher scores indicated more perceived benefits of illness/caregiving.
Control
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Control Attitudes Scale-Revised (CAS-R)	Srisuk/2016	The CAS-R assesses how much perceived control, or how helpless individuals feel about managing their family member's heart problems. The total score can range from 8 to 40; higher scores indicate greater perceived control. The CAS-R is scored by adding the item scores-each item is rated on a scale from 1 (totally disagree) to 5 (totally agree); ratings on Items 5 and 8 are reversed before scoring. Internal reliability of the CAS tested in 21 carers of patients with HF was 0.75. In this study, Cronbach's alpha was 0.60.
Satisfaction with the technology aspect of the intervention
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Satisfaction questionnaire	Book/2020	On the day of the operation, the parents or guardians were handed questionnaires to be filled out while their child was under anesthesia. There were only 9 questions specifically about the videos.
User Satisfaction Questionnaire (USQ)	Ibañez/2018	Satisfaction with the clinical content of the tutorial was evaluated using the User Satisfaction Questionnaire. It contains 15 statements covering different dimensions of the user experience (e.g., “There were sufficient examples and illustrations”). Items are rated on a four-point Likert scale ranging from “strongly disagree (1)” to “strongly agree (4).” Total scores range from 15 to 60, with higher scores reflecting greater satisfaction with the clinical content of the tutorial.
Satisfaction questionnaire based on TAM based on four references	Northouse/2014	Evaluation of participants’ satisfaction with the web-based program was based upon the Technology Acceptance Model. At Time 2 follow-up, we assessed three characteristics of the web intervention 1) perceived usefulness (e.g., I found the program useful), 2) perceived ease of use and usability (e.g., program ran smoothly) and 3) satisfaction with the overall experience. Subscale scores ranged from 1 (low) to 7 (high). Participants also were asked if there was any additional content they would add to the program.
Usability of the technology aspect of intervention
Study instrument for data collection	Study article	Brief summary of the Instruction and Reference (from study article)
Male Transition Tookit evaluation questionnaire	Duggleby/2017	The MaTT evaluation questionnaire was previously used in the evaluation of a similar intervention (Changes Toolkit) (Duggleby, Cooper, et al., 2012) to assess ease of use, acceptability, and feasibility based on the program evaluation frameworks of McKenzie and Smelzer (1997) and Timmreck (1995). The first part of the evaluation asked participants to indicate if they used specific sections of the MaTT (yes or no). The second part featured nine questions focusing on ease of use, acceptability, feasibility, and effectiveness using, with 0 indicating no agreement and 5 indicating strong agreement. The qualitative questions asked were: • What worked well with MaTT? • What would you suggest to improve MaTT? • Any other comments?
Mobile Application Rating Scale	Fuller-Tyszkiewicz/2020	This scale comprises 23 items rated on a 5-point rating scale. The Mobile Application Rating Scale consists of 4 subscales: engagement, functionality, aesthetics, and information. The mean item score across the 4 subscales was used to determine an objective measure of the overall quality of the app, with higher scores indicating higher app quality. Furthermore, the Mobile Application Rating Scale also includes a subscale assessing the subjective quality of the app, consisting of items assessing whether the participant would recommend the app to others, plans to use the app again in the next 12 months, would pay to use the app, and their overall rating of the app out of 5. In this study, an adapted version of the Mobile Application Rating Scale was used, excluding the items assessing the entertainment value and evidence base for the app. These items were removed from the mean score calculation according to the guidelines.
Usefulness, Satisfaction, and Ease of use (USE) questionnaire	Hattink/2015 Van Mierlo/2015	This questionnaire contained 29 questions with 2 open questions, 20 other questions that could be answered on a 5-point scale ranging from “strongly agree” to “strongly disagree” (eg, “I instantly knew where to click”), and 7 questions on usefulness in which participants rated the usefulness of specific parts of STAR on a 4-point scale from “very useful” to “useless.” Also, users were asked to indicate which modules they had followed and to grade each module on usefulness (1–10) to account for the fact that not all participants may have followed all modules of the course.
System Usability Scale (SUS)	Ibañez/2018	The SUS is a reliable, well-validated 10-item scale designed to evaluate usability and user satisfaction with web-based applications and other technologies. Each item is rated on a five-point Likert scale ranging from “strongly disagree (1)” to “strongly agree (5).” Satisfaction with technical usability was examined using a total score that ranges from 10 to 100 (total scores are multiplied by 2), with higher scores reflecting greater satisfaction with the usability of the tutorial.
System Usability Scale (SUS) - Swedish Version	Stjernswärd/2017	A Swedish version of the System Usability Scale (SUS) was used to assess the program's usability. It is a 10-item 5-point Likert scale giving a global view of subjective assessments of usability. Possible scores range between 0 and 100 with higher scores indicating better usability. A system with a SUS value >70 can be estimated as good and >85 as excellent, although it does not guarantee high acceptability in the field. Additional questions with room for free-text answers about usability, confounding factors (other sources of support, negative life events, patient's health status), and negative effects of training were also included for separate analysis.

All 18 studies that included HCD evaluation components in their research described positive results for satisfaction, usage, high scores on usability questionnaires, or other HCD evaluation concepts. Fuller-Tyszkiewicz et al. ⁴² did find low engagement scores on the Mobile Application Rating Scale. They noted “interventions that are personalized and flexible in their design, with advances in technology offering the potential for ubiquitous, tailored support.” Only four studies provided limitations for their HCD evaluation.^34,35,42,51 The limitations could be technology-related, or they could be studied as design-related.

Caregiver outcomes

Based on our previous work, ¹¹ we categorized the caregiver outcomes into the following categories: (1) Psychological health: less anxiety, depression, stress, burden, irritation, and isolation; (2) Self-Efficacy and Hope: self-efficacy/hope/resiliency/comfort; (3) Knowledge/Skills/Communication/Management of Patient; (4) QoL; (5) Social support: social support, social functioning, and needs to be met; and (6) Problem Coping Skills: ability and skills in coping and solving problems, identifying strategies for attaining goals, and getting information or support to make decisions. More specifically, thirty-four studies (85%) reported that caregivers significantly improved caregiver outcomes. Eleven had statistically significant decreased depression and anxiety symptoms,^{23,24,29,31,36,39,42,51,53,55} six had significantly improved self-efficacy, positive attitudes, and sense of control, 13 had significantly reduced stress, strain, and distress,^{21,23,27,35,44,45,50,51,56,58,61,63} 10 had significantly increased resiliency, competence, compassion, and coping skills,^{22,23,28,40,41,45,46,53,54,62} four had significantly decreased sense of burden,^22,37,41,49 one study had reduced substantially social isolation, ³⁸ and six had significantly increased well-being and QoL.^{21,28,29,31,35,37} Seven studies (18%) reported that the caregiver did not statistically significantly improve post-intervention compared to the control group.^{26,30,32,34,43,47} A summary of all instruments used to measure is in Table 5.

Discussion

This systematic review identified 40 studies employing different digital health tools to deliver interventions to caregivers of people with various conditions globally published in English. More than 85% of the studies in the review showed significant improvements in the caregivers’ outcomes. Our updated and expanded review found that digitally enhanced health interventions improved caregiver outcomes in the following aspects: psychological health (reduced anxiety, depression, stress, strain, burden, irritation, and isolation), self-efficacy (increased confidence, hope, resiliency, and comfort), caregiving skills (improved communication with the patient, patient symptom management), QoL, social support (enhanced social connection and functioning), and problem coping skill (strengthened problem coping and solving skills, goal attainment, and decision-making). In the 18 studies that included HCD methods, the results were generally positive for the technology delivering the intervention.

The studies in this review came from 34 journals, 10 fields, and 19 countries. In addition, this review covered patients with varying age ranges, from children, adolescents, adults to older people, with a wide range of conditions and symptoms requiring different caregiver skills and workloads. Furthermore, the relationship between caregivers and patients was comprehensive. Most studies included spouses or intimate partners; some were children or children-in-law, and several were siblings, grandparents, or friends/volunteers as caregivers. Using MMAT, we found the quality of the 40 included studies is high (presented clear research questions, collected appropriate data, and the majority addressed randomization, adherence, and sampling methods). The results of this review are likely to be generalizable. With 32 (80%) RCTs and 8 (20%) quasi-experimental designs, evidence strength is high, the included studies’ quality is high, and the findings provide significant directions for future studies. By using GRADE, we assessed the certainty of the evidence for each included study. The results showed that 37.5% were rated with high certainty, 42.5% were rated with moderate certainty, 15% (n  =  6) were rated with low certainty, and 5% (n  =  2) were rated with very low certainty.

Three-quarters of the interventions were delivered to caregivers at home. Digital health tools enabled robust interventions that provided high-quality assistance to caregivers and saved travel and time, especially during the pandemic or when patients needed intensive monitoring. In addition, the most common digital health tools were interactive web resources (telehealth) and real-time interactive communication (telemedicine). In most studies, the researchers employed multiple intervention components to deliver to caregivers. For example, most studies combined education and data collection components, with or without peer social support, real-time support with professionals, or psychotherapy.

Our review also reveals some limitations across the reviewed studies. Typical constraints include limited generalizability (due to small sample sizes, demographic characteristics of the sample were not broadly representative, care recipients’ symptoms may be more/less severe, Hawthorne effects, self-selection bias, self-report bias), high attrition rate, difficulty to perform intention-to-treat analysis, unknown confounding factors (e.g., disease progression, changes in the family context, caregiving pattern changes), and technology instability (intervention interfaces did not always function correctly, the low usage of the technology intervention). For quasi-experimental studies, the outcomes could not be compared to a group of caregivers who did not have access to the intervention.

Our review found that 75% of studies used a theoretical framework to inform the intervention development. It has been shown that interventions with a theoretical basis are more successful than those without, as theories may focus on determinants that predict or explain outcomes or means of engendering changes in the determinants.^76,77

This review was limited to English-language publications and excluded unpublished or ongoing studies, abstracts, editorials, reviews, pilot studies, or dissertations. Other limitations include the issues of publication bias, the search issues of balancing comprehensiveness with precision, and missing studies not published in peer-reviewed journals. Although the researchers attempted to be comprehensive by searching multiple databases, they did not search for gray literature. This updated systematic review was registered at PROSPECO, and the registration ID is CRD42023400030. The review protocol can be found and publicly available. We also listed the search strategies in the supplement to be available for other researchers.

This systematic review and analysis of digital health interventions provide implications for clinical practice. The studies showed that digital health interventions were effective to support informal caregivers, with the majority showing improvements in caregiver outcomes. Because informal caregivers are essential when providing care to patients, health professionals need to ensure informal caregivers also receive the necessary psychosocial assessments and support. The assessment of the caregivers needs to have their capacity (physical condition, mental concerns, and self-efficacy) for caregiving. Lastly, health professionals should also improve their digital health literacy and leverage technologies to better support caregivers.

Future research should include caregivers from diverse backgrounds as participants, especially those from marginalized communities. More efforts should be focused on improving the accessibility and usability of the technology tools and tailoring the intervention content to be more culturally sensitive and linguistically appropriate. Moreover, it is vital to engage end-users in the design process. Without considering input from caregivers and care receivers in the intervention development process, many technologies may have low adoption due to insufficient feasibility and acceptability. A comprehensive HCD approach is essential in intervention design and development.

Conclusion

This review identified 40 studies that investigated the effectiveness of digital health interventions in supporting informal caregivers. Digital health enhanced interventions compensate for the limitations of traditional human interactions and can be tailored to meet both caregivers’ and care recipients’ needs. Digital health interventions also provide a platform that allows interactive communication and provides up-to-date information to users in real time. These interventions may also prevent hospitalizations and unnecessary clinical visits. This updated and expanded review shows the great value of technology in delivering interventions to family caregivers. Technology-delivered intervention could potentially increase access to self-care for caregivers burdened with caring for their care recipient, reduce caregivers’ anxiety and depression, improve their QoL, coping skills, communication strategy, and relationship with care recipients, and eventually increase patients’ and caregivers’ well-being. Future studies designing and testing digital health interventions for family caregivers should tailor to caregivers from diverse communities, and comprehensive human-centered design approaches have the potential to support this effort.