Process evaluation of a social support platform ‘Inlife’ for caregivers of people with dementia

Abstract

Introduction

Informal caregivers of persons with dementia have an increased risk of facing social isolation. Online social media interventions might offer a new opportunity to increase access to social support. An online social support platform, ‘Inlife’, was developed and launched in the Netherlands to enhance social support, positive interactions and information sharing in informal support networks.

Objective

A process evaluation was performed to evaluate the internal and external validity of the Inlife intervention.

Methods

Implementation, sampling and intervention quality were evaluated by both qualitative and quantitative methods. Analyses were performed using descriptive statistics and inductive content analysis. Analyses were conducted following participants' completion of the intervention after 16 weeks.

Results

The overall participation rate in the study was 27% (96/351). The Inlife intervention was generally well-received by the primary caregivers. Inlife facilitated empowerment, openness, involvement, and efficient care organization. Still, adherence was not optimal for all Inlife users. Determinants for Inlife use were identified on the level of the Inlife innovation, the users, and the socio-political context.

Conclusions

Inlife was evaluated as a useful instrument for efficient central care coordination and mutual involvement. This study emphasizes that the personal attitudes of the Inlife users to seek and provide support warrant attention, next to the characteristics of the actual Inlife innovation for optimal intervention uptake. Online and offline support might be integrated to raise awareness of caregiver social support needs and attitudes and provide insight into caregivers' available social capital.

Trial registration

Dutch trial register NTR6131, Registered on 20 October 2016.

Highlights

• •Users found that Inlife increased empowerment, openness, involvement, and efficient care organization.
• •Lower adherence rates may have affected internal and external validity.
• •Determinants for Inlife use were found at the level of the innovation, the users, and the socio-political context.
• •Implementation strategies should focus on attitudes and barriers of Inlife’s users, as well as Inlife’s characteristics.
• •Offline guidance might be integrated to raise awareness of caregivers’ social capital, their support needs and barriers.

1. Introduction

Informal caregivers of persons with dementia (PwDs)1 have a significant role in caring for their relatives with dementia. Much of the care and support for PwDs is provided by informal caregivers, such as family, friends and neighbours (Alzheimer's Association, 2017). The care process might be demanding due to the progressive functional decline during the disease process (Schulz and Martire, 2004). Caregiving has negative as well as positive impacts on daily life. Although some caregivers report enrichment of relationships (De Boer et al., 2012), others experience heavy burden and social isolation (Schulz and Martire, 2004). Therefore, psychosocial interventions for caregivers are essential to enhancing a supportive environment and preventing overburdening. Research findings have demonstrated that supportive multicomponent interventions that focus on stimulating both the capacities of the PwD and the caregiver in the early phase of the disease are most effective (Boots et al., 2014; Dröes et al., 2011; Olazarán et al., 2010). Recently, innovative E-health interventions have shown beneficial results for caregiver self-efficacy, feelings of competence and depression (Boots et al., 2014). Social media interventions could have the potential to stimulate positive interaction and capacities of PwDs and their informal caregivers, regardless of time and mobility constraints. Hence, in the Netherlands, an innovative intervention was designed entitled ‘Inlife’: an online social support platform to increase positive interaction and social support within the social network of the PwD (Dam et al., 2017c).

The development and evaluation of Inlife were structured according to the guidelines of the Medical Research Council (MRC) framework for developing and evaluating complex interventions (Campbell et al., 2000). According to this framework, randomized controlled trials are considered the gold standard for evaluating the effectiveness of the intervention (Craig et al., 2008). Recently, the updated MRC guidance recognized the added value of a process evaluation prior to an effectiveness evaluation to disentangle intervention fidelity and quality of implementation and to identify contextual factors that might affect future implementation (Moore et al., 2015). Although the MRC framework recommends potential elements to be included in a process evaluation (e.g., fidelity, reach, contextual factors) (Moore et al., 2015), it does not provide consensus regarding the essential elements or the best model to conduct a process evaluation. Therefore, the present process evaluation will be structured according to the model described by Leontjevas (Leontjevas et al., 2012). This model is commonly applied to conduct process evaluations of psychosocial interventions in dementia care (Zwijsen et al., 2014). It provides information on first order process data (sampling and intervention quality such as fidelity, dose, and reach), and on second order process data (e.g., contextual barriers and facilitators). These data might aid future implementation of the intervention and interpretation of the effectiveness and generalizability of the results by understanding the context in which the intervention was delivered. The objective of this process evaluation is the evaluation of the internal and external validity of the newly developed Inlife intervention by examining sampling, intervention quality and potential determinants for use.

2. Methods

This process evaluation was performed prior to the effectiveness study. This study was structured according to a randomized waiting list-controlled design (RCT). In order to provide a clear context for this process evaluation in this section we first describe the design of the RCT. The complete details of the method can be found elsewhere in the complete study protocol (Dam et al., 2017b). This section is followed by a description of the method used to conduct the process evaluation.

2.1. Study design and participants

A RCT was undertaken in the Netherlands, alongside the process evaluation (described below). Based on our power calculation, we originally aimed to include 122 primary caregivers (due to recruitment difficulties, 96 were eventually successfully recruited). Caregivers were recruited via regional and national dementia community services and online channels. A waiting list control design was chosen to optimize acceptability and adherence to the research protocol in the control group and decrease attrition effects (Campbell et al., 2000). Data were collected at four time points: pre-intervention (T₀), 8-week follow-up (T₁), 16-week follow-up (T₂), and 42-week follow-up (T₃). The 16-week follow-up assessment served as a primary endpoint to compare group effects. In- and exclusion criteria were being a primary caregiver of a PwD (all sub-types of dementia), having access to the Internet, basic (tablet) computer skills and with at least two social network members who are willing to join the Inlife platform. The exclusion criteria were being overburdened, having serious health problems that could interfere with participation (e.g., burn-out or surgery) as assessed by the study staff rated in the telephone screening, and being un-available for more than four weeks during the study period. A detailed description on the study design and participants can be found elsewhere (Dam et al., 2017b).

2.2. Intervention

Inlife is an online social support platform for caregivers of PwD aiming to enhance positive interaction, involvement and social support. Inlife was developed in an iterative development process together with potential users, clinicians and web-designers. Details about the developmental process are described elsewhere (Dam et al., 2017c). On the Inlife platform the primary caregiver can invite friends, family and significant others into three personal support circles (i.e., inner, middle and outer circle depending on the closeness of the relationship), with different privileges. The platform consists of the following functionalities: Profile, Circles, Timeline, Calendar, Helping, Personal Messages, Care book and Compass. These functionalities provide opportunities to share care information, messages, pictures and requests for support. Details about these functionalities can be found elsewhere (Dam et al., 2017c). Participants could access the platform via (tablet) computer and smartphone by using a personal username and password. The RCT was conducted over a period of 16 weeks. Participants could use Inlife at their own pace, and the platform remained accessible after the intervention period. Participants in the control group remained on the waiting list for 16 weeks, receiving care as usual. After the 16-week follow-up assessment (T₂), they had the opportunity to register on the Inlife platform.

2.3. Design of the process evaluation

The present process evaluation, which was conducted alongside the RCT, was based on methods used in previous research (Leontjevas et al., 2012; Moore et al., 2015; Zwijsen et al., 2014). Both first- and second-order data were collected to provide a complete overview of the trial quality and the implementation process. The process data were evaluated prior to the effectiveness analysis.

2.3.1. First order process data

For evaluation of sampling quality (e.g., recruitment, reach), we examined the procedures for recruitment, informed consent, treatment allocation, reach, and barriers and facilitators of recruitment. Data were extracted from the digital case record files designed for this study, contained in an online system including demographic information of the participants, contact information, and contact history entered into text fields. A case record was kept for each participants including the answers provided during telephone calls. This data was exported and files were analysed by summing up the responses and grouping qualitative similar answers by author (CMJS).

For evaluation of intervention quality (e.g., relevance, feasibility, adherence to protocol), data were extracted from the program participation questionnaire (PPQ), which was completed by 36 responders in the intervention group after the 16-week follow-up (T₂) to evaluate the feasibility of and satisfaction with the Inlife platform. Answers on the PPQ were provided on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree), and higher scores reflect greater feasibility. The items on the PPQ are shown in the Appendix. Furthermore, we collected qualitative data from semi-structured interviews (n = 10) to examine the feasibility and relevance of the intervention. The topic list is depicted in the Appendix. This subsample of participants in the intervention group that completed the 16-week follow up (T₂) was selected to participate in the interviews by a randomization procedure stratified for gender and user activity.

The interviews were conducted face-to-face in the home setting of the participants by one of the authors (AD). The interviews lasted about an hour. The topic list depicted in the Appendix was designed together with researchers, clinicians, and experts knowledgeable about implementation of innovative psychosocial interventions (MdV, MvB).

2.3.2. Second order process data

Second order process data regarding the implementation of Inlife (e.g., barriers and facilitators) was collected by semi-structured interviews with a randomized selected subsample (n = 10, see Appendix) of caregivers participating the intervention group for 16-weeks.

2.4. Analysis

The quantitative data of the PPQ were analysed using descriptive statistics (SPSS version 24.0). The qualitative interviews were audiotaped and transcribed verbatim. Subsequently, the qualitative data were analysed using Atlas.ti (Version 1.0.14 for Apple Macintosh). Two authors, AEHD and HLC, performed inductive content analysis (Evers, 2015). In the inductive content analysis, open coding, continuous examination and comparison by the two researchers resulted in categories and themes emerging from the data. Open codes that were frequently reported were grouped into categories and these categories are grouped into higher-order themes. After thorough discussion between the researchers, consensus was reached about the categories and themes (AEHD, HLC, and MEdV).

3. Results

3.1. First-order process data

3.1.1. Sampling quality

3.1.1.1. Recruitment and randomization

For the RCT, 475 primary informal caregivers of PwDs were recruited via the Dutch Alzheimer Association (Alzheimer Nederland) and caregiver support services (e.g. day care centres, caregiver support groups), online advertisements (e.g., Facebook, online newsletters), written advertisements (e.g., local newspapers), and via brochures distributed on regional memory and mental health clinics. These 475 caregivers received information, were contacted and screened for eligibility. 351 met the inclusion and exclusion criteria and were eligible to participate. If interested, caregivers were sent an information letter (n = 267) and those who declined were asked for their reason to decline. A total of 96 of the eligible caregivers agreed to participate in the study and signed the informed consent form (27%, 96/351). Reasons for declining to participate in the study are depicted in Table 1. After the baseline assessment, participants (N = 96) were randomly assigned to either an intervention group (n = 48) or a waiting-list control group (n = 48). A computer program controlled by an independent researcher performed the randomization. The self-reported follow-up measurements were completed online through a custom and confidential questionnaire system. The 16-week follow-up measurement was the primary endpoint for evaluation of the group differences (Dam et al., 2017b). In total, 92.7% of the caregivers (n = 89) completed the 16-week follow-up assessment. The number of non-responders included 5 in the intervention group and 2 in the control group.

Table 1.

Caregiver reasons for declining to participate in the Inlife RCT.

Reason	Number of caregivers who declined to be sent the information letter	Number of caregivers who declined after being sent the information letter
Intervention was considered too time-consuming or burdensome	21	25
Inability to be contacted	18	17
Not enough interest within the personal network	12	18
A social network that was too small (<2 persons)	6	23
Already using other available online care tools	0	20
No current need for support	13	0
Having other expectations of the intervention	6	9
No current need for support	13	0
Unknown	6	5
Feeling unfamiliar with using online devices	0	7
The PwD rejects support or is suspicious	0	3
Health problems of the PwD	1	2
Not willing to participate in academic research	1	0
Problems with privacy on the web	0	1

3.1.1.2. Barriers and facilitators for recruitment of participants

Data on barriers and facilitators for recruitment were extracted from the digital case record files including contact history, participants' responses, and responses from health care professionals that assisted in the recruitment. Barriers for recruitment included concerns of additional burden or lack of willingness of caregivers' social network members to participate in the Inlife circles. Case managers who assisted in recruitment process reported that having a small network size and concerns about the use of online technological devices were primary barriers, whereas being younger and having familiarity with online devices facilitated program recruitment. Since the recruitment turned out to be more challenging than expected the recruitment period was extended with three months.

3.1.1.3. Reach

An estimated total of 971,304 caregivers of people with dementia were reached through various recruitment methods. The reach was approximately 0.05%, 475 out of these 971,304 caregivers contacted the authors for more information on the trial. Data was collected on the recruitment method through which they were reached: through flyers (38 recruited from 500 flyers distributed), Alzheimer Netherlands Facebook, website ads and online newsletters (240 recruited from 950,000 cumulative page views), as well as ads in local parish newsletters (19 recruited from 20,528 parish newsletters distributed), community services (56 recruited from 101 attendees of local Alzheimer Café's or caregiver meetings), case-manager referrals (25 recruited from 25 referrals), caregivers known to the research group through previous studies (82 recruited from 150) and communication with known relatives or acquaintances (n = 15). 351 of these caregivers were eligible to participate in the trial. In total, ninety-six caregivers participated in the trial.

3.1.2. Intervention quality

Qualitative data on the relevance and feasibility of the Inlife intervention were derived from the semi-structured interviews. Characteristics of the interviewed caregivers are shown in Table 2. Seven themes emerged from the inductive content analysis (Table 3). The themes shown in these tables are solely based on the qualitative analysis of the interviews and these themes are supported in the next sections by quantitative data on the usability and user-friendliness of Inlife collected from the Inlife Program Participation Questionnaire (PPQ), that was completed after the 16-week follow-up (T₂) by 36 participants in the intervention group (five did not fill in any questionnaire of the 16 week follow-up and seven participants declined to fill in this final survey, the PPQ).

Table 2.

Background characteristics of the interviewed caregivers (n = 10)^a.

Respondent (R)	User activity	Gender	Caregiver relationship	Age	Years of caring	Hours of caring per week
1	High	Female	Daughter	54	5	12
2	High	Female	Grand-daughter	26	6	30
3	Low	Female	Daughter	49	4	10
4	High	Male	Spouse	59	9	85
5	High	Male	Spouse	63	4	14
6	Low	Female	Daughter in law	60	0.5	3
7	Low	Male	Spouse	71	6	1.5
8	Low	Male	Son	64	6	12
9	High	Female	Daughter	59	3	20
10	Low	Female	Daughter	62	7	12

^aA sub-sample that completed the 16-week follow-up measurement (T₂) was selected to participate in the interviews by a randomization procedure stratified by user activity and gender. Classification of high and low user activity in the intervention group was based on a mean split of the total clicks on the Inlife platform after 16 weeks (mean = 1617).

Table 3.

First order data: themes and categories process evaluation (n = 10).

Theme	Category
Reasons for Inlife use	• Care planning • Involvement with others • Using specific Inlife features • Altruistic motives (i.e. improve care/research)
User-friendliness of Inlife (first order data)	• Clear navigation via circles • Clear colors and symbols
Usage of Inlife	• Circles: useful for privacy • Calendar: planning of appointments • Timeline: sharing on daily basis • Helping: overview of needs and offers • Care book: transfer of care • Personal messages: quick messaging • Compass: finding information
Relevance of Inlife (advantages/disadvantages)	• Increased feelings of control • Central care organization (i.e. monitoring) • Openness and connectedness • Addressing stigma • Limited effects on feelings of well-being and support • Disappointment due to lack participation
Satisfaction and recommendations for improvement of Inlife	• Customer journey (i.e. minimize log-ins) • Lay-out changes (i.e. positioning dyad) • Add content or functionalities (i.e. video calling)

3.1.2.1. Reasons for Inlife use

Some caregivers had altruistic motives for participation such as contributing to academic research, development of new technologies, or the improvement of future dementia care. Others were interested in specific features of Inlife, such as the support circles and calendar that could assist with care planning and involvement in the social network.

“For the future, that has actually been one of my reasons for participation. I think my children will at least have to know that such a thing exists”

(R6, Daughter-law, 60 years, low-active user)

3.1.2.2. User-friendliness of Inlife

Participants used both the website and the app version. The app version for smartphones and tablets had added value since it enabled faster information transfer wherever and whenever, even in the moment.

Participants reported that the structure and layout of Inlife were clear. Data collected from the PPQ (i.e., using a 5-point Likert scale) demonstrated that the structure of the Dashboard (M 3.9, SD 1.1) and clear symbols (M 4.0, SD 0.8) enabled easy navigation on the website (M 3.9, SD 1.1). The circular structure of Inlife was valued (M 3.7 SD 1.1) for the privacy and autonomy, since carers could decide for themselves what was shared with whom. This setup increased feelings of control. The instructions were evaluated as sufficient (M 3.9, SD 0.9). Some participants mentioned that instructions on paper could have been a helpful addition.

“We have the first circle with the people who provide immediate care. The second circle was just the family that often visits. In addition, the third circle was then family, or more friends who once wanted to do something or contributed to the front yard for example. It enabled us to share everything around the care, such as the medication with the first circle, the second circle does not have to know that.”

(R5, Husband, 63 years, high-active user)

3.1.2.3. Usage of Inlife

The goal, content and number of functionalities were considered appropriate (M 4.0, SD 0.8) and offered the possibilities that the participants expected (M 3.6, SD 1.1). However, the distinction between the different functionalities was sometimes not clear (e.g., the personal messages and timeline were sometimes used interchangeably). The calendar was used for planning appointments and requesting support for a specific moment. The helping functionality was used to convey requests and offers of support. These help-seeking functionalities were not used regularly. Caregivers reported that either help was not yet needed or that caregivers preferred to ask for help in person, outside the online context of Inlife. The timeline was used to share information, pictures and activities on a daily basis.

“I think the power is that Inlife works with photos, that you do not have to read. Take a look, oh nice, the lightness of the photos lets them speak for themselves and place the accents on the good moments, which are mainly achieved through photographs.”

(R9, Daughter, 59 years, high-active user)

The personal messages were used for quick messaging with (groups of) individuals. The care book functionality assisted in the transfer of care-related information.

“In the care book, I wrote things that I thought were important when others go out with her (the PwD). So that others know what they can and cannot expect.”

(R1, Daughter, 54 years, high-active user)

The compass provided specific dementia-related information. Participants did not regularly consult the compass. It was suggested that more updated information could be provided. It was reported in the interviews and shown by the number of clicks on the website that the calendar and timeline were used most frequently (Table 4)

Table 4.

Percentages of clicks per Inlife functionality within the intervention group during the 16-week study period.

	Percentages (%) of clicks per functionalitya
Viewing Timeline	23.7
Viewing Calendar	14.8
Posting Calendar	6.8
Viewing Circles	4.7
Viewing Personal Messages	3.8
Viewing Helping	2.6
Adapting helping Preferences	2.3
Posting Timeline items	1.8
Viewing Care book	1.6
Posting Personal messages	1.3
Adapting Care book	0.8
Requesting/providing support in Calendar	0.6
Total clicks within the functionalitiesa	64.8 (n = 22,994)
Other clicks on the webpageb	35.2 (n = 12,486)
Total clicks on the webpage	100 (n = 35,480)

^aThis table includes percentages of clicks per functionality on the Inlife webpage. Note, clicks on the mobile app are not included in this overview (64.7% n = 42,121/n = 77,601, total clicks on webpage and mobile app), since these were not measured separately per functionality by the system.

^bOther clicks contain clicks outside the functionalities such as viewing the homepage, log-ins, and adjusting messages or pictures.

3.1.2.4. Relevance of Inlife

Overall, Inlife was experienced as useful (M 3.5, SD 1.2), and it moderately assisted in the organization of care (M 2.9, SD 1.3). The interviewed caregivers reported qualitatively that Inlife is a promising online tool to ease the organizational burden of care, serving as a convenient platform for central care organization:

“The good part is that I already find an incredibly big plus of Inlife, that you can refer people to a place and usually they do that themselves. I think that in its totality it just strengthens the whole picture of the situation and the support, under which some people not only say that they want to help more but also do that.”

(R4, Husband, 59 years, high-active user)

Additionally, Inlife contributed to feelings of control over care, a sense of involvement in daily life, and openness and connectedness within the social network. Although it was reported that Inlife increased involvement within the social network (M 3.0, SD 1.4), it did not directly facilitate seeking online support (M 2.7, SD 1.4). However, some caregivers reported that circle members were triggered to offer more implicit offline support in daily life because participation in Inlife facilitated up-to-date information on the current situation of the caregiver and PwD.

“You have the communication flow on Inlife and it is because of that, that beyond Inlife it has become easier because you know, well, we are all part of Inlife too.”

(R8, Son, 64 years, low-active user)

In sum, it was reported that Inlife is an accessible medium for central care coordination that enables the monitoring and safe exchange of information on a daily basis, which otherwise might not have become visible for all other involved network members. The increased visibility of the daily life of the caregiver and PwD not only increased openness but was also helpful to address stigma around the topic of dementia since the topic was discussed more openly. Although Inlife was experienced as helpful in alleviating the care process, it was felt that it did not directly improve experienced well-being, due to the on-going challenges faced by the caregiver during the progressive disease trajectory.

Furthermore, we identified some unexpected side effects. Primary caregivers reported that it was sometimes difficult to motivate network members to join the Inlife network or to get them involved. Some network members were reluctant to post reactions on Inlife, which occasionally caused disappointment and frustration and, in some cases, prevented Inlife usage.

“I was disappointed that people were not willing to participate and that also the involved care institution said no, I did not expect that.”

(R10, Daughter, 62 years, low-active user)

3.1.2.5. Satisfaction and recommendations for improvement of Inlife

In general, participants were satisfied with the possibilities that Inlife offered (M 3.8, SD 1.0).

Participants found the platform meaningful (M 3.4, SD 1.4) and would recommend it to other caregivers (M 4.2, SD 1.1). Inlife received a sufficient feasibility rating on a scale ranging from 1 to 10 (M 7.6, SD: 1.6). Participants suggested improvements in Inlife for the customer journey (e.g., reducing the number of log-ins, ability to upload documents, increasing text lay-out options), the content (e.g., including a chat function, video-conferencing) and the layout (e.g., visual disconnection of the dyad members).

Furthermore, it was suggested to make Inlife accessible for caregivers of people with other (chronic) conditions.

3.1.3. Adherence to the protocol

Caregivers reported spending approximately 1.1 h per week on the Inlife platform. Users also reported spending enough time on Inlife to understand the functionalities (M 3.7, SD 1.3). The most clicks were registered on the calendar and timeline. Primary caregivers invited (M 6.3, SD 5.4, range 0–20) network members into their circles. Having a minimum of two circle members was a prerequisite to participate on Inlife. However, five participants were unsuccessful in inviting at least two circle members. The primary study period was 16 weeks. After this period, participants in the waiting-list control condition (n = 48) could start using Inlife. Of the participants (n = 48) randomized into the intervention group, 47 registered on the Inlife platform. Based on the log data, 76.5% (36/47) continued usage of the platform for at least 16 weeks. During the 16-week study period, user activity (operationalized as the total number of clicks on the Inlife webpage and mobile app) varied widely (M 1651, SD 2165, range 2–10,699). The majority of the participants in the intervention group that used Inlife for at least 16 weeks also continued to use it after the 16-week study period (range 17–73 weeks).2

3.2. Second order process data

3.2.1. Determinants for Inlife use

The determinants that were reported in the semi-structured interviews (N = 10) either facilitating or impeding the use of Inlife could be divided into different levels: the level of the Inlife application, the level of the users (i.e., primary caregiver and their circle members) and the socio-political context (see Box 1 and Table 5).

Box 1. Second order data: levels of determinants for use and suggested implementation strategies. (For details see Table 5.).

Alt-text: Box 1

Table 5.

Second order data: themes and categories (n = 10).

3.2.1.1. Determinants of the level of the innovation

The Inlife application contained elements that were considered user-friendly (e.g., clear symbols, colors and circles). However, some elements warrant attention such as the login procedure and uploading pictures. Ease of use of the innovation was considered a significant factor for successful implementation. In addition, positive user experiences determined continued use of Inlife to a large extent.

“It worked because I receive reactions, When I sent a personal message, people often tend to forget about it. Or say “oh I'll do that later”, but if they see something with a photo and they also see that she (the PwD) is enjoying her food, so that she is in her element, then they think oh nice and then they react immediately, it triggers earlier.”

(R2, Granddaughter, 26 years, high-active user)

3.2.1.2. Determinants at the level of Inlife users (caregivers and circle members)

The level of engagement with Inlife was determined by the current dementia phase, caregiver needs and prevailing attitudes of both the caregiver and the circle members concerning seeking and providing support. Because of these personal attitudes, caregivers might feel reluctant to share information on Inlife:

“You don't want to see your mother with dementia like that. So, if you keep it for yourself then it is not so severe, then it is not that bad. But if it is communicated with the outside world, then it is also true and then I have to deal with it myself, otherwise I can also leave it behind me. Inlife will open the doors and then it may also be seen.”

(R10, Daughter, 62 years, low-active use)

Inlife was implemented more easily when there was a primary caregiver that actively motivated circle members and referred continuously to Inlife for central care coordination so that it could become part of the daily routine.

“People have to get used to Inlife. And you have to continue to stimulate because if it does not come from the other person, I also noticed that if I did not send something out to the person, then it would stop being used.”

(R10, Daughter, 62 years, low-active user)

In addition to sufficient computer and information and communication technology (ICT) knowledge and skills, the characteristics of the involved circle members were important determinants for implementation. In particular, the overall network size and the number of reactions or posts by network members determined continued use of Inlife.

3.2.1.3. Determinants at the level of the socio-political context

The prevailing view that Internet usage is not yet widely accepted in all layers of society (e.g., depending on age, computer literacy) impeded participants from actively using Inlife or inviting others into their circles. Furthermore, uncertainty about the security and privacy affected the usage.

In contrast, the presence of other widespread online tools (e.g., WhatsApp) influenced involvement with Inlife. People were sometimes inclined to use WhatsApp instead of Inlife because it was more routinely used in their daily practice.

3.2.2. Suggested strategies for future implementation

The final theme that emerged described potential implementation strategies. On the socio-political level, caregivers reported that guidance by volunteers or coaches in the community would have been valuable to provide additional user guidance and awareness of social support opportunities. Furthermore, better profiling of the Inlife brand could increase participation.

“It would be nice if you are able to speak to someone who also uses Inlife. Then, you both know what you are talking about and then you can also explain the possibilities. Not that I know everything. However, I know how it works … I think for Inlife itself it is of course nice if you can reach as large an audience as possible. And it is a pity if people drop out because they do not find it clear enough. It might be easy if someone who is very positive about it shares the experiences and shows you here we use it for.”

(R1, Daughter, 54 year, high-active user)

Improvements at the level of the Inlife application itself could facilitate future implementation. For example, a visual step-by-step manual provided after the first login was proposed, which could potentially help circle members to start using the application. It was suggested that personalized notifications sent by the system could potentially stimulate circle members to post pictures on the timeline after they finished their appointment as planned in the calendar.

4. Discussion

This process study examined the quality, relevance and determinants of implementation of the Inlife intervention. First-order and second-order process data were evaluated to gain insight into the internal and external validity to guide interpretation of the effectiveness and generalizability of the results.

4.1. First-order data

4.1.1. Sampling quality and randomization

The overall participation rate in the study was 27% (96/351), which was lower than expected. Despite the various recruitment strategies and the extension of the recruitment period, the intended goal to include 122 participants was not accomplished. The response rate in caregiver intervention studies is often problematic and depends on the recruitment strategies, selection criteria and content of the intervention (Leach et al., 2016). In line with previous studies, online advertisements and mailings were the most effective recruitment strategies (i.e. 240/475, 50.5%) (Leach et al., 2016; Morrison et al., 2016). Because of the smaller sample size, the statistical power might be limited to establish significant differences between the groups in the effectiveness evaluation.

The primary recruitment barriers were unfamiliarity with online devices, unwillingness of the social network to participate, and considering the intervention too time-consuming or burdensome. Caregivers who already experienced a high burden might have been more inclined to reject participation. Moreover, experiencing stigma and a high threshold to seek support might explain the high refusal rate (Clement et al., 2015). This study might inevitably be subject to selection bias, since people that have better health and computer literacy are more likely to participate (Kreps and Neuhauser, 2010), which might influence the external validity of the results.

4.1.2. Intervention quality

Overall, Inlife users were positive about the content, functionalities and opportunities that Inlife offered.

The structure, layout and content of Inlife were considered clear and user-friendly. In particular, our target group appreciated the circle structure and clear symbols for navigation.

Inlife was generally evaluated as a beneficial instrument to efficiently coordinate care in one central place and exchange information on a daily basis that otherwise might would not have become visible. In addition, Inlife enabled increased feelings of connectedness, control, and empowerment. Notably, Inlife also had side effects, such as frustration and disappointment when circle members did not actively participate. Possibly, Inlife might have created awareness that not enough support was available, which in the usual care situation would have remained unnoticed. The reported side effects might explain why the adherence was not optimal. However, it is not uncommon in e-health research to find low adherence rates (Kelders et al., 2012). The present study provided room for improvement in the customer journey, content and layout of the Inlife system.

With regard to the first order data, in general, the Inlife intervention was received well. This finding validates the evaluation of the results for a future effectiveness study. However, the internal and external validity warrants attention due to the suboptimal participation and adherence rates.

4.2. Second-order data

4.2.1. Implementation knowledge

Our findings emphasize that in addition to determinants of the innovation (e.g., user-friendliness) and determinants of the socio-political context (societal acceptance of technology), determinants of the users are important for the uptake of Inlife (see Table 2). Not only were positive user-experiences and an enthusiastic central care coordinator important, but the circle's reactivity, as well as the prevailing beliefs and attitudes regarding providing and offering support, also determined Inlife uptake. In this respect, a previous study demonstrated that cognitive biases play a role in seeking support (Dam et al., 2017a). Caregivers and their social network members tend to think for others and therefore are reluctant to seek or provide actual support. Although there is not much evidence of the role of personal attitudes in informal care, it has been demonstrated that access to formal care services is also influence by personal beliefs, such as a perceived lack of support need or awareness (Brodaty et al., 2005).

Inlife was most successful when circle members were responsive and had an open mind towards online technology and support. However, participants frequently reported that online support was not yet needed since it was too early in the disease process. This finding is in line with previous research indicating a support paradox: caregivers in a later phase regret that they did not use available support in an earlier phase (Boots et al., 2015).

On the one hand, we showed that determinants on the level of the user are important since the uptake of Inlife could have been driven by deeply rooted beliefs and implicit personal attitudes (i.e., fear for stigma, seeking support or burdening others). Surprisingly, on the other hand, we found that the implementation strategies suggested by the subsample of interviewed participants did not address these personal barriers for use. Instead the suggested implementation strategies focus mainly on adapting the Inlife intervention (e.g., adding notifications) or on the socio-political context (e.g., guidance by a peer, volunteer or coach in the community) and not on changing personal attitudes of the users themselves (Table 2). Therefore, future implementation plans should not only focus on improving the product itself, but also on creating awareness of psychological barriers of potential users. Early identification of social support barriers and awareness might be essential for successful engagement with online innovations.

4.3. Lessons learned for future research and clinical implementation

Our findings reveal that not all identified determinants for Inlife use are addressed by participants in their suggested implementation strategies. Future implementation plans should include also psychological factors that determine intervention use. Potentially, some caregivers lack insight such that prevailing personal beliefs and attitudes might actually impede seeking active online support. Therefore, it is essential that healthcare professionals within dementia-friendly communities (Heward et al., 2017; Lin, 2017) raise awareness about the importance of timely access to social support. Therefore, an intervention such as Inlife could have more potential if embedded in local and national dementia community services. For example, within the context of Alzheimer Cafés or in regional and national dementia care services, awareness could be raised concerning social support needs, attitudes and support opportunities. Furthermore, face-to-face personal contact might increase adherence to the Inlife intervention and enable sharing best-practice stories of Inlife users. In addition, as observed in the interviews, caregivers had the tendency to underestimate the richness of their own social network. Therefore, potential adaptions to the customer journey to pro-actively think about existing social ties and social capital might increase user adherence, such as by completing a survey or online ecogram after the first log-in to the Inlife platform, which would give users a better insight in their potential support network upfront. Providing insight into caregivers' individual social capital might increase interactions on Inlife. Previous research has demonstrated that available social capital is associated with increased social engagement and mental health (Ehsan and De Silva, 2015; Keating and Dosman, 2009).

4.4. Strengths and limitations of the present study

In line with the updated MRC framework, the added value of this process evaluation alongside the RCT is that it aids the interpretation of the results of the upcoming effectiveness analysis and provides valuable insights and recommendations for future implementation. This process evaluation also had some limitations. First, the evaluation focused mainly on the primary caregiver, and it would be valuable to additionally interview other stakeholders such as the person with dementia (if possible), involved Inlife circle members, and health care professionals.

Second, the qualitative analysis was conducted in a sub-sample only. Although we used a stratified sampling procedure, it might be possible that caregivers who encountered barriers might have dropped out before conducting the qualitative interviews after 16 weeks, and as a consequence the sample might be biased to some extent. Moreover, we only selected a sub-sample of participating caregivers for the interviews. This might have reduced the transferability of the results. However, the participants were randomly selected for the qualitative interviews and none of them declined the invitation for an interview. Furthermore, the researchers observed saturation in the data after interviewing ten caregivers in the intervention group indicating that no new themes or categories could be obtained from the data.

5. Conclusion

The present study was a crucial step in evaluating the innovative Inlife intervention, resulting in promising insights that should be evaluated in an upcoming effect evaluation and incorporated into the next version of Inlife. Furthermore, this study provided insight into factors that might improve implementation of Inlife on a broader scale. Our data indicated that Inlife contributes to feelings of empowerment, openness, or involvement, and facilitates more efficient care organization. The adherence was not optimal for all users due to disappointment and frustration caused by a lack of responsiveness of circle members.

Determinants of Inlife use were identified on the level of the innovation, the users, and the socio-political context. Surprisingly, not all indicated determinants for Inlife use were addressed by participants in their suggested implementation strategies. More specifically, our study emphasizes that personal attitudes of the Inlife users to seek and provide support warrant attention next to characteristics of the actual product and socio-political context in future implementations plans. To improve implementation and engagement with Inlife, online and offline support should be integrated to raise awareness of social support needs and attitudes and provide more insight to future users into their available social capital.

Appendix A

Topic list for the semi-structured qualitative interviews

1. How did you receive information about the existence of Inlife? 2. Why did you start using Inlife? 3. What did you think about the design/structure/layout of Inlife? 4. Was the use of Inlife clear (also for the circle members)? 5. Which format is most useful? (e.g., the website/app version) 6. Does the app version of Inlife have added value? 7. Did you experience technical problems? 8. Was there sufficient instruction and guidance for Inlife use? 9. Did you continue using Inlife for the entire study period? Why? 10. Was Inlife useful? How? 11. How/in what way did you use Inlife? 12. Which functionalities were or were not useful for you? 13. Could you invite enough people into the Inlife circles? 14. Were the circle members involved in the Inlife platform? 15. Did you encounter barriers for Inlife use? 16. How could these barriers be solved? 17. Were there other factors that influenced Inlife use? (e.g., contextual factors) 18. Did it bother you that Inlife was part of an academic research project? 19. To what extent did Inlife meet your needs in the care for a person with dementia?  Prompts:  Did Inlife use enlightened care?  Did the use of Inlife affect your feelings of competence? 20. Did Inlife make a difference in your experience of social support?  Prompts:   Did Inlife use influence involvement within the social network?   Did Inlife lower the threshold to ask for support   Did Inlife enlarge your circle size?   Did the use of Inlife affect your feelings of loneliness?   Did the use of Inlife affect your mood or stress level? 21. Did you use other online tools for the care of your relative with dementia? (e.g., Facebook WhatsApp) 22. Did Inlife have any unexpected effects on your daily life? 23. Are there other factors that influenced the extent of success of Inlife? 24. In general, are you satisfied with Inlife? How would you rate Inlife on a scale from 1 to 10? 25. Would you recommend Inlife to other caregivers? Why? 26. What do you need to use Inlife better in the future? 27. Do you have any other recommendations for improvement of Inlife?

The Inlife Program Questionnaire

		I totally disagree				I totally agree
1. I found Inlife useful/helpful		□ 1	□ 2	□ 3	□ 4	□ 5
2. The usage of Inlife made asking for help easier		□ 1	□ 2	□ 3	□ 4	□ 5
3. The usage of Inlife made organising help easier		□ 1	□ 2	□ 3	□ 4	□ 5
4. Inlife increases involvement of the own social network		□ 1	□ 2	□ 3	□ 4	□ 5
5. I have the impression that other people in my network found Inlife useful		□ 1	□ 2	□ 3	□ 4	□ 5
6. I used information, advice or tips that were offered in the Inlife network		□ 1	□ 2	□ 3	□ 4	□ 5
7a. I used the circles	□ Yes □ No
7b. I found the circles meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
8a. I filled out my profile	□ Yes □ No
8b. I found my Profile meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
9a. I filled out the Timeline	□ Yes □ No
9b. I looked at the Timeline	□ Yes □ No
9c. I found the Timeline meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
10a. I looked at the presentation modus of the Timeline	□ Yes □ No
10b. I found the presentation modus of the pictures meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
11a. I used the Notifications	□ Yes □ No
11b. I found the Notifications meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
12a. I used or looked at the Helping function	□ Yes □ No
12b. I found the Helping function meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
13a. I looked at the Calendar	□ Yes □ No
13b. I used the Calendar to ask for support	□ Yes □ No
13c. I found the Calendar meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
14a. I looked at the Compass	□ Yes □ No
14b. I found the Compass meaningful		□ 1	□ 2	□ 3	□ 4	□ 5
15. I found the goal and the functions of Inlife clear		□ 1	□ 2	□ 3	□ 4	□ 5
16. The functions of Inlife do what I had expected		□ 1	□ 2	□ 3	□ 4	□ 5
17a. How many hours per week did you spend on Inlife?
17b. I spend enough time on Inlife to understand the possibilities that Inlife offers		□ 1	□ 2	□ 3	□ 4	□ 5
18. The overview in Helping supported me to ask for help more easily		□ 1	□ 2	□ 3	□ 4	□ 5
19. The ‘questions for support’ which could be asked in the Calendar helped me to organise care		□ 1	□ 2	□ 3	□ 4	□ 5
20. I found the reminder e-mails a good addition		□ 1	□ 2	□ 3	□ 4	□ 5
21. I found the bi-weekly update e-mails a good addition		□ 1	□ 2	□ 3	□ 4	□ 5
22. I found working with Inlife was easy		□ 1	□ 2	□ 3	□ 4	□ 5
23. The start page on Inlife was clear		□ 1	□ 2	□ 3	□ 4	□ 5
24. The symbols/icons on Inlife were clear		□ 1	□ 2	□ 3	□ 4	□ 5
25. The texts on Inlife were easily readable		□ 1	□ 2	□ 3	□ 4	□ 5
26. In general, the context of the texts on Inlife were appealing to me		□ 1	□ 2	□ 3	□ 4	□ 5
27. The instructions for Inlife usage were clear to me		□ 1	□ 2	□ 3	□ 4	□ 5
28. I found the information that was offered sufficient		□ 1	□ 2	□ 3	□ 4	□ 5
29. I have enough technical skills to use Inlife		□ 1	□ 2	□ 3	□ 4	□ 5
30a. I did not experience problems with privacy on Inlife		□ 1	□ 2	□ 3	□ 4	□ 5
30b. I experienced no problems with privacy on Inlife during contact with network members		□ 1	□ 2	□ 3	□ 4	□ 5
30c. I experienced no problems with privacy on the Timeline		□ 1	□ 2	□ 3	□ 4	□ 5
31. In general, I am satisfied with the possibilities that Inlife offered		□ 1	□ 2	□ 3	□ 4	□ 5
32. Inlife was useful for me		□ 1	□ 2	□ 3	□ 4	□ 5
33. I would recommend Inlife to other caregivers of people with dementia		□ 1	□ 2	□ 3	□ 4	□ 5
34. How would you grade Inlife on a scale from 1 to 10?	1–10