e-Health intervention development: a synopsis and comment on “What Design Features are Used in Effective e-Health Interventions? A Review Using Techniques From Critical Interpretive Synthesis”

Abstract

In this synopsis and commentary on the Morrison and colleagues article published in Telemedicine and e-Health (18:2, 137–144, 2012), we provide a brief review of effective design features of e-Health interventions as well as a discussion on future directions. The Internet is being used more frequently to deliver health behavior interventions; however, it is unclear which design features contribute to intervention outcomes. Morrison and colleagues conducted a review using critical interpretive synthesis techniques to identify design features that mediate the effects of e-Health intervention outcomes. A total of four design features were identified (social context and support, contacts with intervention, tailoring, and self-management) that may mediate the effect of the intervention on outcomes. This review provides a preliminary conceptual framework to guide future evaluations of the effects of e-Health design features on intervention outcomes. Future research should target optimizing e-Health interventions to determine which design features should be included as well as how they contribute to outcomes.

INTRODUCTION

We present a synopsis and commentary on Morrison and colleagues [1] article “What Design Features are Used in Effective e-Health Interventions? A Review Using Techniques from Critical Interpretive Synthesis.” Internet-delivered interventions have been found to have a small positive effect on behavior change, yet the reported efficacy and effectiveness of individual interventions are inconsistent [2–4]. Although frameworks exist to guide the development of health interventions [5–7], supplementary guidance is lacking to guide delivery of content over the Internet. One review technique that provides an opportunity to examine intervention content is a critical interpretive synthesis (CIS). CIS methods incorporate similar processes of a traditional systematic review; however, they also include techniques from qualitative research. CIS methods can help to generate a theory based on a synthesis of a diverse sample of the evidence [8]. The aim of this review, which uses CIS techniques, was to develop a conceptual framework to examine specific design features used to deliver the content of Internet-based health interventions. This conceptual framework enables the evaluation of how specific features of the e-Health intervention design may impact health outcomes.

METHODS

A CIS was conducted to identify and compare design features of e-Health interventions. Specifically, the authors completed three phases to identify themes (diversity sample), clarify classification of design features (theoretical sample), and test the conceptual framework and hypothesis generalizability (representative sample).

The aim of phase 1, the diversity sample, was to select a diverse sample of articles covering a wide range of e-Health design features of interventions published between 2000 and 2009. The authors searched Ovid, ISI Web of Knowledge, PubMed, Science Direct, and Google Scholar using the following terms: Internet, health, intervention, quantitative, behavior, review, efficacy, evaluation, and use. All Internet-based interventions encouraging individuals to change health-related behaviors using educational and self-management strategies were required to be fully automated and report quantitative analyses. Interventions were excluded if they were computer-based, included face-to-face contact, or targeted the treatment of mental health disorders. The authors compiled 27 resulting articles and identified the design features of how intervention content was delivered. Interventions were then classified as more effective, less effective, or ineffective, and the associations between design features and intervention effectiveness was examined. Phase 2, the theoretical sample, aimed to clarify the design features classification from phase 1 and to examine explanations for the variability in the effectiveness of the design features. The theoretical sampling resulted in 23 articles.

The final phase, the representative sample (phase 3), tested the conceptual framework to determine whether the hypotheses developed in phase 1 were consistent. Using similar inclusion/exclusion criteria as in phase 1, the representative sample included a total of 25 articles published between 2001 and 2005. Following the identification of any design features not included in the diversity sample, the interventions were coded by effectiveness.

RESULTS

The researchers identified 11 design features from the diversity sample, with no additional features uncovered in the representative sample. Four design features were highlighted: social context and support, contacts with the intervention, tailoring, and self-management. These features were labeled as interactive because they were hypothesized to mediate the outcome of the selected interventions. Morrison and colleagues [1] note that the focus of this review was not to reach any definitive conclusions or test generated hypotheses about said features, but rather, to outline potential associations identified during the phase 1 diversity sampling, between design features and effective interventions.

Social context and support

The authors divided the concepts of social context and support into three categories: simulations of person-to-person interaction, delivery of synchronous- or asynchronous-mediated contact with other users, and provision of information about other users. The diversity sample suggests that using automated dialogue, as opposed to avatars, for simulated person-to-person interaction, was more commonly found among effective interventions. The authors suggest that individuals may expect more from an automated dialogue than a robotic avatar because of its ability to more realistically mimic human interaction; the automated dialogue may perform better with users as a result. In reviewing studies concerning synchronous- vs. asynchronous-mediated user contact, the authors did not find an effect but propose that the effectiveness may be dependent on a number of variables (e.g., active posters vs. “lurkers,” perceived credibility of online forum). Furthermore, supplying information about other users, through techniques like social norming, varied in effectiveness based on the sampling technique. The diversity sample, which included more hypothetical user testimonials, found no association with intervention outcome, while the representative sample, using testimonials from real users, found more user information was linked to more successful interventions. The authors hypothesize that facilitating person-to-person interaction, synchronous- or asynchronous-mediated communication with peers, or providing information about other users may positively affect intervention outcomes.

Contacts with the intervention

Morrison and colleagues [1] identified two types of intervention contact from the diversity sample: expert-initiated vs. user-initiated contact. The authors further divided expert-initiated contact into two subgroups: contact utilizing behavior change techniques and contact promoting use of the intervention. The first subgroup, contact to deliver behavior change techniques, was commonly seen in more effective interventions, whereas the second subgroup, contact promoting use of the intervention, was generally found in the less effective interventions. The authors postulate that several factors may interfere with determining the effectiveness of contacts delivering behavior change techniques. These factors include only reaching those already engaged and ceiling and context factors. Motivational emails or reminders, for example, will only serve a purpose if the user is not already making the desired changes and has an appropriate context (e.g., time, energy) to view said reminders for change. The authors hypothesize that intervention contacts that incorporate behavior change techniques may be more efficacious than simply reminding participants to use or engage with the intervention.

Tailoring

The process of tailoring, as defined by the authors, refers to the delivery of intervention-related information that is relevant to one individual. Tailoring can be grounded in theory, based on demographic variables, or can depend on a user’s behavior. The representative sample did not see a difference in the number of tailored variables and intervention effectiveness; however, the diversity sample suggests that tailoring on multiple variables contributed to more effective interventions. The effectiveness of tailoring is proposed to be hierarchical: theory-based tailoring is more efficacious than tailoring based on behavior, which is then more useful than demographic-based tailoring. Although there were differences in associations found between the representative and diversity samplings, the hypothesis generated by the authors is that tailoring on a number of variables may result in better intervention outcomes than tailoring based on only one variable.

Self-management

The self-management features identified from the diversity and representative samples included activity planning and self-monitoring. These features were found to be in both more effective and less effective interventions. Both feature types were received favorably by users, but often not utilized. Morrison and colleagues [1] state that recent meta-analyses of self-monitoring have indicated this component’s effectiveness, especially when paired with other components such as goal setting. One limitation identified by the authors in determining self-management effectiveness is the lack of detailed intervention protocols. It is often unclear how activities are planned as well as what and how behaviors are monitored, which makes it difficult to extract whether components were or were not effective for the intervention. With more detailed descriptions of the interventions in manuscripts, it may be possible to clarify the mixed results of the effectiveness of these self-management features. The authors hypothesize that self-management strategies that are grounded in behavior change theories will be more effective than those strategies that exclude theory.

CONCLUSIONS

The authors suggest that four interactive design features mediate health outcomes in Internet-based interventions based on the sampling techniques and associations identified: social context and support, contacts with intervention, tailoring, and self-management. The authors hypothesize that interventions that incorporate these features will produce improved outcomes as compared to interventions that do not include them. While the conceptual framework includes design features proposed in previous frameworks (i.e., tailoring and self-management), it also includes new features that have not been defined within existing frameworks (i.e., social context and support and contacts with interventions).

COMMENT

As electronic and mobile health interventions continue to become more widespread, a better understanding of which intervention design features are most effective is needed. The review by Morrison and colleagues attempts to address this question and proposes that four interactive intervention design features mediate outcomes. Of the four design features identified, two (tailoring and self-monitoring) are more common among existing interventions. The other two features identified, social context and support and contacts with the intervention, have not generally been a focus or highlighted feature of interventions. Future studies will need to further examine the role and effectiveness of these newly identified features within health-related interventions. Several limitations were mentioned, including limited descriptions of intervention design features and the small number of identified “ineffective” interventions. Interventionists developing e-Health interventions would greatly benefit from more manuscripts and expansive intervention protocol descriptions discussing extensive details of how design features were designed and implemented within the intervention. Furthermore, publishing both effective and ineffective interventions could prevent future studies from including ineffective components and testing out new or additional features instead.

Morrison and colleagues also suggested the use of different research methods to optimize the design of e-Health interventions. Designs and frameworks like the multiphase optimization strategy (MOST) [9] and sequential multiple assignment randomized trial [10] (SMART) may be appropriate methods to first develop and optimize behavioral interventions prior to testing out in a full randomized controlled trial. The technology within e-Health and m-Health interventions continuously changes and becomes more innovative and sophisticated at a rate faster than the current models of research (e.g., randomized controlled trials). Having an idea of which features are the most important and effective for enhancing intervention outcomes will ensure that new, advanced technologies can incorporate these important components from the onset of development. Furthermore, identifying the most effective features will help reduce costs of technology development and implementation by avoiding unnecessary spending on features that will have little or no effect on intervention outcomes.