Development and implementation of an integrated, multi-modality, user-centered interactive dietary change program

Abstract

Computer-tailored behavior change programs offer the potential for reaching large populations at a much lower cost than individual or group-based programs. However, few of these programs to date appear to integrate behavioral theory with user choice, or combine different electronic modalities. We describe the development of an integrated CD-ROM and interactive voice response dietary change intervention that combines behavioral problem-solving theory with a high degree of user choice. The program, WISE CHOICES, is being evaluated as part of an ongoing trial. This paper describes the program development, emphasizing how user preferences are accommodated, and presents implementation and user satisfaction data. The program was successfully implemented; the linkages among the central database, the CD-ROM and the automated telephone components were robust, and participants liked the program almost as well as a counselor-delivered dietary change condition. Multi-modality programs that emphasize the strengths of each approach appear to be feasible. Future research is needed to determine the program impact and cost-effectiveness compared with counselor-delivered intervention.

Introduction

An increasing number of interactive computer programs for health-related behavior change are available, and a recent review of automated dietary change interventions suggests that such programs can be effective [1]. Several well-controlled studies have reported significant long-term improvements from technology-based interventions in dietary behaviors, weight loss or both, relative to randomized control conditions [2–4]. However, almost all such programs are single modality [e.g. Internet, CD-ROM/DVD, tailored mailings, Interactive Voice Response (IVR)] only, and few provide participants much choice concerning goals or strategies.

Most automated interventions to date have been based on the appeal of a given technology, rather than a functional analysis of the relevance of various components to the intervention goals, and use of this analysis to decide upon the most relevant technologies. Thus, we have Internet-based programs that have great convenience (for those with access), but which have difficulty delivering high-performance media and retaining users [2, 5]. Or we have CD-ROM programs that can easily present videos, audios and rich graphics, but are often inconvenient and hard to disseminate. There are IVR programs that appear successful at reaching and engaging a high percentage of participants, and especially lower literacy populations [6, 7], but place major limits on the complexity of issues that can be addressed and are limited to a single auditory modality. For these and other reasons, it often proves less feasible than anticipated to implement computer-tailored health behavior programs in busy, real-world health care settings [2, 6, 8].

Another key issue is the amount of human contact that is optimal and cost-effective. There have been several demonstrations in which entirely automated programs were successful in producing behavior change [1, 2, 9], but more often programs combine automated and human contact [3, 7, 10]. Importantly, little is known about how much human contact is optimal, for what types of patients and with what types of behavior change targets. This paper assesses whether entirely automated programs were able to be implemented to the same level and produced user satisfaction ratings as high as staff-administered programs.

We have developed the WISE CHOICES interactive program to help adult women increase their consumption of fruits and vegetables and decrease consumption of fat. The new, fully automated intervention program is based on earlier interventions that successfully used a combination of automated and in-person counseling to improve dietary quality [3, 5, 11]. While these interventions were effective, the reliance on skilled counselors for delivering part of the program limited the likely scope of dissemination to higher risk patients. The goal of the present study is to develop an effective dietary change intervention that can be delivered entirely through automated means (CD-ROM and IVR), and would therefore have the potential for cost-effective and scalable applications in large populations. The effectiveness of this intervention is being compared with other intervention programs in a randomized trial. Results from that trial will be available in later years. This current paper focuses on the theoretical, operational and technical challenges in developing and implementing a fully automated, multi-modality dietary change program.

We describe the WISE CHOICES program, which combined different interactive computer modalities to help adult women make and maintain reductions in total fat consumption and increases in fruit and vegetable intake. This paper (i) discusses the theoretical basis and operationalization of different program components in the WISE CHOICES program, (ii) describes the rationale for and use of multiple interactive modalities in a single program, (iii) summarizes changes made to the program as a result of formative feedback as the program was developed and (iv) presents implementation and user satisfaction data on the program under completely automated and staff-assisted conditions.

Methods

Setting and participants

Using electronic medical records, a random sample of female Kaiser Permanente Northwest (KPNW) region members, aged 40 years and older, were selected who met preliminary age, membership and geographic criteria. Women screened for the study completed a series of three questionnaires, including a 27-item dietary intake survey [12]. Those eligible for the study were invited to the research clinic for their first in-office study visit. At this visit, after completion of informed consent, the first intervention session was conducted. At the end of this visit, future contacts were scheduled as described below and included a second intervention visit in approximately 2 weeks. The flow chart in Fig. 1 summarizes both the content and sequence of intervention elements. All procedures were approved by the local Institutional Review Board (IRB).

Fig. 1.

Flow chart of WISE CHOICES program.

Automated intervention system overview

The WISE CHOICES program brought together several media modalities and maximized the potential of each in the process of engaging users and helping them reach and maintain behavior change goals. The rationale and/or theoretical basis for each of these components is presented when that component is described. The system design for WISE CHOICES included three major technological components—an intervention CD-ROM containing an array of audios, videos and appealing graphic elements; an IVR system using automated telephone reminder calls for following up with participants and an intranet-based data management system for collecting and storing user responses and guiding participant interactions with the CD-ROM and the IVR (Fig. 2).

Fig. 2.

WISE CHOICES interactive components.

We implemented WISE CHOICES as the centerpiece of a dietary change project. The program was developed with two approaches, both using the media-based system as the core element. One version is a stand-alone automated program, relying primarily on the CD-ROM and IVR for generating user engagement. The second version includes the use of a health coach to provide human contact to the participant as an additional component to the technology-based interaction. These versions were compared with each other and to two other conditions: health coach counseling with no automated components and a control condition consisting of automated counseling for physical activity in a 2 × 2 randomized design.

Action plans

Based upon problem-solving theory [13–15], a user-driven, self-management Action Plan was the foundation for the intervention (see screen shot in Fig. 3). The problem-solving steps toward creating a successful Action Plan were to (i) assess and receive feedback on current dietary habits, (ii) identify personal benefits of healthful eating, (iii) select a specific dietary goal (e.g. fat reduction or fruit and vegetable increase), (iv) identify barriers that were most likely to interfere with achievement of the goal, (v) choose strategies for overcoming those barriers (see screen shot in Fig. 4) and (vi) review the overall plan and make any desired changes (see Fig. 1). Over the course of the program, participants were guided through all these steps, which yielded an on-screen Action Plan that was printed out for the user to take home and review on a regular basis.

Fig. 3.

Sample WISE CHOICES Action Plan.

Fig. 4.

WISE CHOICES screen shot of strategy selection.

Participants first completed a brief dietary questionnaire, either online or in a telephone interview [16]. The baseline survey, which collected estimated daily servings of fruit and vegetables as well as overall fat intake [12, 17], was stored on a central database and linked to the other media components of the system. Since this was a cancer prevention project, the focus was on overall fat intake, rather than a particular type of fat. The majority of examples and strategies, however, targeted animal, saturated and trans fats. The information in this database was used by the CD-ROM to tailor messages and to facilitate user interactions with the computer program. The IVR telephone system also used information from the central database to follow up with individual participants, present users with opportunities to report on their activities and goal attainment and to revise their Action Plan, if necessary.

CD-ROM component

We wanted to maximize user engagement and ensure that they received the key elements of the intervention. Because we wanted to make full use of video, audio, animation and rich graphic elements to enhance user engagement, we selected CD-ROM as the primary delivery modality (this did, however, require in-office visits). The interactive computer session opens with a ‘Welcome’ video message from the two hosts of the program—a health educator, who also serves as an on-screen and voice-over narrator, and a KPNW physician. The health educator appears frequently throughout the program to deliver key health messages and provide a sense of familiarity and personalized attention for the user. Based upon information gathered from the participant's dietary questionnaire, the WISE CHOICES program then draws from the local database and presents animated graphs that display the user's current eating habits (i.e. estimated total fat intake and fruit and vegetable consumption) and comparisons with national recommendations. The program was designed to enhance comprehension across a broad spectrum of literacy abilities by using both audio instructions and visual cues. At this point in the program, the participant chooses whether to focus on one of two dietary change domains: increasing fruits and vegetables or reducing fat. Later, when she returns to the clinic for her second session, she creates an Action Plan for the other dietary domain.

If the user sets a goal to reduce fat, she is prompted to work on two areas where she might make improvements in her eating habits—food choices and portion size. In each of these areas, the program draws from the information gathered from the dietary questionnaire, including a list of the high-fat foods in her current diet. To complete the goal-setting process, the user chooses several foods that she could eat less frequently, prepare differently, reduce portion size or substitute with healthier foods.

In the next part of the program, the user selects two barriers from an updated list that we compiled from earlier dietary intervention studies. These barriers have been reported as common obstacles to achieving healthful eating goals [3, 5]. But before the user moves on to looking at strategies for overcoming barriers, the program uses motivational interviewing principles to ask how willing she is to work on the barriers she has selected. On this screen, the user is prompted to click on a sliding scale to identify her level of willingness to work on her barriers. If she has some concern about moving forward with her Action Plan, she is encouraged to select different barriers that might be more relevant or easier to overcome.

Once the barriers have been chosen and confirmed, the participant is presented with each barrier and a list of suggested strategies for overcoming that barrier. On this screen, the user has the opportunity to explore each strategy in more detail by viewing ‘More Info’ or using the ‘Write My Own’ feature to type in a strategy that is not on the list. The strategy selection screen was designed to be user-friendly and intuitive, offering the participant graphic, photo and text elements to assist the selection process, easy-to-use check boxes and a convenient way to write in her own strategy, if necessary (Fig. 4). After choosing two or three strategies for each barrier that she wants to try over the next few months, the user is presented with an on-screen version of her completed Action Plan.

At this point, the user can choose to go back to any part of the Action Plan and make changes. By clicking on the Goals, Barriers or Strategies sections of the on-screen plan (Fig. 3), the user is sent back to the appropriate parts of the program and prompted to make new choices. She then has a chance to review her plan and make additional changes. After the user has reviewed and settled on her plan, she clicks a button and the on-screen narrators deliver an exit message, while a copy of her Action Plan is printed. This process is repeated in the second in-clinic session for the other dietary change area.

For each participant, individualized Action Plan elements—Goals, Barriers and Strategies—were stored in a local Microsoft Access™ database, which was linked directly to the CD-ROM. These data fields were transmitted, on a daily basis, using a data transformation service, from the local database to the central database, where they were available to be used by other components of the media-based system, and for later data analyses.

IVR component

Another innovative aspect of the study was the integration of automated telephone follow-up delivered by an IVR system. We wanted a convenient, low-cost, proactive, automated way to reach participants for follow-up contacts and to reinforce progress. Therefore, we selected IVR technology. Data collected during the computer sessions were used by the telephone response system to make brief phone calls to the participants. The IVR programming system allowed a large number of branching response levels (e.g. tailored responses based upon both current and past levels of progress). This flexibility allowed us to tailor each call to include information from a previous visit or call (goals and barriers) and participant-specific information (greeted by name and confirmation of delivery of intervention to correct person). The timing and number of calls made to reach each participant was implemented via communication between the tracking system and the IVR delivery system (Fig. 2).

The IVR calls ask participants, who are informed in the first session that they will be receiving these calls, to review their Action Plan goals and their motivation for continuing with their plan. They are given encouragement and the opportunity to hear some tips or motivational messages.

Automated intervention summary

For the WISE CHOICES study, the key to user engagement and retention centered on a multiple-modality concept, where each component—CD-ROM, IVR and networked database—was used because of its potential strength for delivering a particular aspect of the intervention (Fig. 1). Each contact with the participant, including initial screening assessments, two CD-ROM sessions at the health clinic and IVR calls that followed the CD-ROM sessions, generated information that was stored and managed in a central database that linked user responses to other modalities. The CD-ROM, using high-performance media, including audio, video and animation, was designed to provide an easy-to-use, stylish experience for the user without much previous computer or Internet experience and to deliver tailored messages about lifestyle change in a personalized, engaging fashion. The program was structured to enable a high degree of user choice and to provide the participant with tools to focus on her particular dietary goals, obstacles and strategies for change. The IVR, with its high reach and ability to generate brief, frequent contacts with participants at low cost was used to follow up with respondents after intervention sessions. These three modalities, in concert with each other, presented a seamless, coordinated display for the participant, and focused the user interactions with technology on functions that were appropriate for each medium.

Changes as a result of formative research

There were three major revisions and numerous minor changes made during the formative development process for the WISE CHOICES program. Formative testing was conducted on an individual basis, primarily for appeal to different cultures and races. Testers went through the program ‘talking aloud’ and provided reactions. Developmental work for the earlier program had included more systematic focus group formative research [3, 11]. Most of these revisions were made to achieve a balance among on-screen functions—maximizing the amount of available information, automated tailoring to display options and facilitating user choice. The first revision related to presenting a menu of problem-solving strategies tailored for specific patient-identified barriers. To accommodate the range of choices and to facilitate user selection, all available options were presented on a single computer screen. This was challenging because some of the strategies, such as ‘Learn how to estimate reasonable portion sizes’, were not intuitively obvious and required a visual example or more space than was available on the primary screen. To address this, we created ‘More Information’ icons associated with each such strategy. Clicking on the icon activated a ‘pop-up’ window, which included a graphic or photograph, and provided a more detailed text explanation.

The second major revision involved substantial modifications to the dietary change barriers and strategies used in the program. This revision included changes in the content, wording and reduction in the number of barriers and strategies presented so that they would fit on a single screen. These changes were based on the analyses of selection frequency by women in prior projects, comments from pilot participants and inclusion of items relevant to African-American populations.

The final major revision had to do with the action plan that summarized user choices throughout the program. The primary change involved creating a more efficient and user-friendly process for allowing participants to review their initial plan on-screen and to make revisions to the plan. The original process involved assessing the user's self-efficacy, and if this was <7 on a 10-point scale [18], leading the user through a series of time-consuming steps to see if they wished to revise key goals, barriers or strategies. The final process simplified this sequence by asking users whether they wished to make any changes to their initial plan, and if so, allowing them to simply click on the action plan element that they wished to revise (Fig. 3).

Measures

The primary outcomes for the overall study were percent of calories from fat, caloric intake and servings of fruit and vegetables as assessed by food frequency questionnaires. Measures relevant to this paper were measures of intervention implementation and user satisfaction.

Implementation was assessed by automated records of computer use including time to complete sessions and options selected. Records of completion of telephone calls were obtained from automated data from the IVR system for automated calls and interventionist records for live counselor calls.

User satisfaction was assessed by questionnaire at a 3-month follow-up. Participants were asked to rate the helpfulness of the six-program components in Table I on 6-point Likert scales ranging from 1 = not helpful to 6 = extremely helpful. Finally, participants were asked to rate their satisfaction with the overall program on a similar 6-point scale.

Table I.

Helpfulness and satisfaction ratings by condition

Item	Automated mean (SD)	Combined mean (SD)	In-person mean (SD)	Exercise control mean (SD)
Video/computer information and recommendations	4.2 (1.4)A	4.3 (1.2)A	3.2 (2.0)B	3.4 (1.4)B
Project staff	4.6 (1.3)A	4.9 (1.1)A	4.9 (1.2)A	4.0 (1.4)B
Phone calls	3.2 (1.7)B	3.8 (1.6)A	4.0 (1.5)A	3.0 (1.6)B
Printed materials	4.7 (1.3)A	4.9 (1.1)A	4.5 (1.3)A	3.7 (1.4)B
Personalized feedback	4.4 (1.5)A	4.8 (1.2)A	4.4 (1.3)A	3.6 (1.5)B
Action plan	4.3 (1.5)A	4.4 (1.3)A	4.4 (1.3)A	3.8 (1.4)B
Overall satisfaction	3.9 (1.0)A	4.2 (0.8)A	4.2 (0.9)A	3.5 (1.1)B

Conditions having different superscript letters are significantly different from each other. Those sharing the same superscript are not different from each, based upon Tukey’s post-hoc follow-up tests. Ratings on a 1–6 scale with 1 = ‘not helpful’, 4 = between ‘moderately’ and ‘extremely helpful’ and 6 = ‘extremely helpful’. SD = standard deviation.

Analysis

Most results were descriptive, involving means and standard deviations or percent of participants. Between-condition comparisons were made on the helpfulness and satisfaction ratings using analysis of variance with Tukey’s post-hoc comparisons where there was a significant overall F value.

Results

Implementation

Almost all participants completed both the interactive CD-ROM office sessions (100% completed session one, 91% session two). Each session averaged a total of 40- to 50-min duration, approximately 30 of which were spent on the computer. Completion of the follow-up phone calls was more variable. The program was evaluated both in an entirely automated condition and when combined with a health counselor. Live phone callers were extremely successful in reaching participants, with completion rates ranging from 99% for the first call to 89% for the third call. Calls averaged 4–5 min. The automated IVR system was somewhat less successful in completing the calls, with rates ranging from 72% for the first call to 68% for the third call.

Relatively few participants used the write-in option provided on the CD-ROM. Across the list of strategies for reducing dietary fat intake and increasing fruit and vegetable intake, users wrote in their own strategies from 2% to 6% of the time.

User satisfaction

Users rated the various aspects of the WISE CHOICES program as fairly helpful/fairly satisfied. On the 6-point scale, with 1 = ‘not helpful’ and 6 = ‘extremely helpful’, most ratings were between four and five. As can be seen in Table I, there were significant differences among the various conditions of use on all seven of the rating scales. Tukey’s post-hoc comparisons revealed that on helpfulness/satisfaction items on which in-person, automated and combined conditions could reasonably be compared, the in-person and combined conditions were rated slightly but non-significantly higher than the totally automated condition. The one exception was that follow-up phone calls were rated as less helpful in the automated condition than in in-person or combined conditions. In addition, participants in both computer-delivered or assisted conditions rated their programs as significantly more helpful than the control condition (a computer-administered physical activity change program) on six of the seven ratings. Users in the entirely automated condition were moderately satisfied overall (M = 3.9) and found the action plan (M = 4.4), printed materials (M = 4.7) and personalized feedback received (M = 4.4) helpful, ratings equivalent to those of participants in the in-person condition on these items.

Discussion

This report demonstrates that a relatively complex, fully automated dietary behavior change program that integrates multiple interactive technologies can be developed and successfully implemented within the context of a real-world health delivery system. Possibly because of the prior experience of the collaborators in both dietary change and interactive technology projects, the development process went relatively smoothly. One key to having an attractive, easy-to-use and understandable end product was allowing adequate time to pilot the initial version of the program, to fully test the various data exchange interfaces and to make modifications based upon pilot feedback.

Both the automated and staff-assisted conditions were consistently implemented and rated as helpful by participants. The IVR aspects of the program were delivered somewhat less consistently and rated somewhat lower than the CD-ROM program components, possibly reflecting our team's greater experience with CD-ROM modalities. If later outcome data demonstrate that the automated program is effective in producing dietary changes, it appears that the WISE CHOICES intervention may be a scalable, relatively low-cost method of delivering behavior change services relative to in-person counseling. Obviously, future research is needed to document program effectiveness at both short-term and longer term follow-ups and the intervention costs.

These preliminary data suggest that it may be possible to fully automate health behavior change interventions. Our implementation and user satisfaction results suggest that CD-ROM elements worked very well, but that the IVR aspects of the program could be enhanced or possibly that some follow-up activities should be conducted by staff to enhance the connection of users to the clinical setting offering the program.

Other lessons learned include the advantage of delivering health education information and behavioral strategies via multiple redundant channels (e.g. text, video models and voice-over narration) to provide benefit for a wide range of users. The use of a central database to coordinate various intervention modalities also worked well and proved to be both robust and secure. In WISE CHOICES, we did not allow users to select different modalities, but rather decided upon the best technology for each intervention purpose (e.g. initial motivation, later follow-up, active problem-solving, goal monitoring). Future research may want to experiment with providing users their choice of modalities (e.g. Web versus IVR versus in-person) for components such as follow-up contact.

The satisfaction data suggest that the menu of strategies provided were seen as relevant to this target population. Although used infrequently, the write-in option may be important because having the option of further personalizing one's action plan—regardless of whether it is actually used—may contribute to perceptions of relevance and control. Unlike some interactive technology-based programs that completely tailor user strategies, we attempted to balance theoretical and experience-based suggestions with provision of choices for users. We feel that this element of user-tailoring adds value to the program.

The various technologies used in WISE CHOICES all had strong ‘reach’ [19], but the CD-ROM component did require participants to come to a clinic to complete the two action planning sessions. We chose to use CD-ROM rather than Internet because we did not want to limit participation to health plan members who had high-speed computer access, but future investigations may want to investigate mailed DVD or Internet-delivery options, especially as part of patient portals offered by an increasing number of health plans and organizations such as the Veterans Administration [20, 21].

In conclusion, we encourage future projects to explore multi-media programs that combine different technologies in creative and integrated ways, rather than relying on a single technology/modality to carry the entire weight of the intervention [22]. Other future directions include testing different program variations to identify the optimal and most cost-effective use of human contact and the impact of interactive technology programs on the behavior of different patient subgroups (e.g. across levels of education, race/ethnicity computer experience and health literacy).

Funding

National Cancer Institute (grant no. RO1 Ca 098496).

Conflict of interest statement

None declared.