A Tablet Based Embodied Conversational Agent to Promote Smoking Cessation among Veterans: A Feasibility Study

Abstract

Embodied Conversational Agent (ECA) offer a new means to support smokers as a virtual coach and motivate them to quit smoking. In this study we assess the feasibility and acceptability of an ECA to support quit smoking (“aka ECA-Q”). ECA-Q, a 14-days program, delivered through Tablet computers, interacts with participants with supporting messages for quit smoking and motivates them to set a quit date. Study participants (n = 6) were Veterans receiving medical care at Boston VA Healthcare System who responded to an open advertisement. Participants completed a survey at baseline and after 14 days follow-up. All participants were satisfied with the ECA program and liked the features of the agent; three out of six participants had set a quit date by the end of the 14 days. Participants reported several positive and less important features of the agent and made suggestions to improve the agent. This study shows that a conversation agent is acceptable to smoking veterans to help them in setting a quit date with an ultimate goal of quit smoking. Insights gained from this study would be useful to redesign the current version of ECA-Q program for a future randomized controlled trial to test the efficacy.

1. INTRODUCTION

The negative health consequences of smoking and secondhand smoke exposure are well documented [1,2]. Cigarette smoking is a significant health problem within the US military. Smoking prevalence in the Veteran population has been reported to be similar to the general US population with approximately 20% of Veterans reporting smoking in 2010 [3]. However, a 2008 study [3] reported higher rates of smoking (33%) among Veterans of the Iraq and Afghanistan. In the same study [3], the rates of smokeless tobacco use was 14%. Estimated costs of smoking among US military health care system beneficiaries were more than $900 million per year in 1995 ($584 million in direct health care costs and $346 million in lost productivity among active duty personnel) [4]. Recent data suggest the prevalence of smoking might be increasing among active duty military personnel [5], underscoring the need for innovative interventions to help those who want to quit smoking.

Smoking cessation interventions are cost-effective and save lives [6–8]. The 2008 Clinical Practice Guidelines confirmed the efficacy of medication and counseling for smoking cessation [9]. The Veterans Administration/Department of Defense Clinical Practice Guideline for the Management of Tobacco Use outlined the requirements for clinicians at the VA facilities to promote smoking cessation among the Veterans who smoke [10]. To help Veterans quit smoking, the VA offers screening for tobacco use during primary care visits; individual counseling; prescriptions for nicotine replacement therapy, such as a nicotine patch or gum, or other medications; and participation in evidence-based smoking cessation programs. However, the utilization of smoking cessation services by the Veterans that use traditional methods of in-person counseling or advice are low with low quit rates [11], underscoring the need for innovative interventions to improve quit rates.

Because of the rise of e-health technology the use of computerized interventions represent a viable solution to support efforts to provide behavioral interventions in a cost-effective way [12,13]. In the recent literature, the use of an Embodied Conversational Agent (ECA) – computer characters that simulate face-to-face conversation using voice, hand gesture, gaze cues and other nonverbal behavior, and that use simulated social behaviors to establish trust and therapeutic alliance with patients [14–16], has shown preliminary efficacy on a wide range of health behaviors including breastfeeding [17], exercise [18], and the treatment of depression [19] and schizophrenia [20]. However, the use of ECA to promote smoking cessation is not yet reported. In this study we assessed the feasibility and user opinion of an ECA program to support quit smoking (aka “ECA-Q” program), to help smokers think about quitting and set a quit date after 14 days of use by Veteran population.

2. MATERIALS AND METHODS

2.1. Research Design

A prospective cohort study research design was used. This included a baseline assessment of smoking behavior and the use of the ECA-Q for 14 days. Data regarding the use of ECA was continuously collected and maintained in the Tablet computer used by each participant. Opinion about the usability and features of ECA-Q was assessed with questionnaire survey at follow-up after 14 days of use. The study protocol was approved by the Institutional Review Board (IRB) of the Boston VA Healthcare System (Study #2927).

2.2. Setting and Participants

Participants were patients attending VA Hospital in Jamaica Plain (Boston, MA, USA) who responded to an open advertisement. Open advertisement included publicizing the study in the notice boards available at the lobby of the VA hospital, waiting area of the Pharmacy and the waiting area of the VA shuttle bus service. Inclusion criteria were being aged 18 or above, receiving medical care from VA Boston, current smoker (smoke at least 1 cigarette daily), willingness to use a Tablet computer loaded with ECA-Q daily for about 5 min (given by the program), ability to speak and read English (because the initial version of the ECA-Q was developed in English only), capacity to see and hear clearly (because the preliminary version of the ECA-Q does not include assistive technologies), and not receiving another smoking cessation program.

After initial screening over the telephone, participants attended an orientation meeting where the study process was explained and any questions or queries were answered. Those who agreed to participate at this point were asked to sign the consent form. The operation of the Tablet computer, loaded with ECA-Q, was demonstrated to each participant before dispatching the Tablet to the participant. A follow-up assessment after participants had used the ECA-Q for 14 days was conducted. Each participant was given a small subject fee ($40) as an incentive to actively participate in the study, which was given after completion of the follow-up interview and after returning the Tablet computer. This fee is comparable to other similar studies involving VA population. Participants were not required to use the Tablet daily to receive the incentives. As shown in Fig. 1, 21 subjects responded to the open advertisement, of which 11 subjects attended the orientation, and nine then entered the study and attended the follow-up after 14 days of ECA-Q use. However, consent forms for three participants were incomplete, which was reported to the VA IRB as soon as it was identified. The committee of VA IRB decided that the data for these three subjects must be excluded from the analyses. Thus, the remaining six subjects were considered as participants in the study and was included in the analyses.

Figure 1.

Flow diagram of subject screening and recruitment

2.3. About “ECA-Q” Program

The ECA-Q program was designed as a simple conversational program, delivered in 14 days, to encourage smokers to think about quitting and set a quit date. The smoking cessation source materials for the agent was based on the US Clinical Practice guidelines [9]. Similar to previous ECA programs [14–20], the program features conversations with an agent based on the responses provided by the participant. The agent character was designed to look like a female African American professional counselor with an apparent age of 30–35 years. After the program was developed by one of the co-authors (TB), the program was loaded in Tablet computers and pilot tested with five undergraduate students of Northeastern University in Boston to identify and address any technical problems. The final ECA-Q program was then loaded in four Tablet computers provided by Northeastern University. To use the program, subjects had to log in daily and should use the program daily for 14 days. We restricted participants to receive one session daily as in the real world setting individuals will meet a counselor once daily. Also, individuals will need some time to start making changes of their behavior which would not happen multiple time in a day. Each session ranged from 5 to 10 minutes in length, based on the individual query and circumstances. Participants could choose their smoking stages of change, smoking triggers or reasons for quitting, etc. from a list that would appear in the Tablet. Based on the individual situation (i.e. Stages of Change), the program guided participants toward setting a quit date. There was continuity between sessions; the agent would initiate discussion in the new session based on the recorded information in the previous session. However, the participants did not have access to the information that they entered in the previous session. Participants who wanted to quit immediately, would be guided with tips as to how to move forward with their quitting plan and would be suggested to call the State Quitline. Participants who would like to know more about smoking cessation would be told about the benefits of quitting smoking and the available help to support smoking cessation, including the availability of pharmacotherapy and nicotine replacement therapy. Topics covered within the 14-day program would cover the following aspects: Reasons for smoking (identify triggers), reasons for quitting, withdrawal symptoms for quitting, relapse prevention and identification, ways to avoid smoking (certain situations), importance of social support, planning something special for quit date, strategies for quitting (i.e. cold turkey, counseling, pharmacotherapy), how to prepare for the quit date, importance of setting a quit date and stay motivated, cost of smoking, reminders of number of days left until the quit date; setting quit date, what it means, how to prepare, what to do; setting goals for cutting down, if not willing to quit at this time; and dealing with stress and anxiety; benefits of being smoke free.

Each session would start with a greeting by the agent (i.e., hello, how are you today; nice meeting you again). The conversation will then move to the topic of the day and briefly review the previous session, will encourage subjects to keep motivated with their plan (or suggest subjects to think about quitting, if not yet decided to quit); ask them to set a quit date (or remind them number of days left before their quit date). Each session ended with a reminder to talk on the following day, except the final session. The final session ended with reinforcing messages to encourage participants to think about quitting or stick with their set quitting date, and suggested participants to seek support from the State Quitline or VA smoking cessation program.

2.4. Measures and Analysis

Our main outcome measures were feasibility and acceptability of the program. Feasibility was measured by assessing “usefulness” and “usability” of the agent. Acceptability was measured by assessing participants’ “satisfaction” and “willingness to refer friends to the program”. We also assesses short-term effect of the ECA-Q program on smoking behavior of the subjects.

Measures from this study were derived from baseline and 14-days follow-up interview, and from computer records of use of the ECA-Q program. The baseline, self-administered, survey gathered demographic, and smoking and quitting characteristics of the participants. The follow-up assessment, a self-administered questionnaire, gathered information about the usability and usefulness of the program and short-term effect of the program on their smoking behavior. Both structured and open-ended questions were included in the survey questionnaire. The usefulness, the primary indicator of feasibility, was measured by asking: “How would you rate the usefulness of the information provided by the computer character about smoking, secondhand smoking and quitting smoking?” with response categories of: “Very unsatisfactory”, “Unsatisfactory”, “Satisfactory”, “Very satisfactory”. The usability, a measure of feasibility, was assessed by asking a “Yes/No” forced choice question of: Was the agent (Tablet computer based program) user friendly?”.

Participant’s satisfaction with using the ECA and any barriers to use the agent or general suggestions to promote cessation service using the agent was also gathered. Satisfaction was measured by asking: “How would you rate your overall satisfaction with the Agent (the computer character) to help you think about quitting smoking?” with response categories of: “Very dissatisfactory”, “Dissatisfactory”, “Satisfactory”, “Very satisfactory”. To measure satisfaction, we also asked a question: “Would you recommend similar program to a friend who is interested in quitting smoking?” with response categories of “Yes/No”. Barriers were assessed by asking two open ended questions of: “What do you think are the least important or least useful parts of the relational agent (i.e., the computer character) for you?” and “What are some possible drawbacks of this agent (Tablet computer-based program) that need attention?”.

We measured short-term effect of the ECA-Q on the smoking behavior (quitting smoking, reducing smoking, and making quitting attempt) and adopting no smoking policy at home. Quitting was measured by a “no” response to the following question: “Have you smoked a cigarette, even a puff, within the past 7 days?”. The reduction in number of cigarettes smoked is based on the following question: “Did you reduce the number of cigarettes smoked daily during the last 14 days?” with response categories of: “No, I did not make any change.” And “Yes, I reduced smoking from _______ cigarettes daily to _______ cigarettes daily.” Quitting attempt was measured by asking the question: “In the last 14 days, how many times have you tried to quit smoking and not smoked for at least 24 hours?” with response categories of: “None”, “Once”, “More than once”. Household smoking restrictions was measured by asking the following question: “Which statement best describes the rules about smoking in your home now?”, with response categories of “a. No one is allowed to smoke anywhere in the home”, “b. Smoking is permitted in some places or at some times”, “c. Smoking is permitted anywhere”.

We conducted a retrospective review of the Tablet records to characterize the engagement of each participant with the ECA (i.e., average login in 14 days, average time spent in each login).

Also of interest was the response to the question asked by the agent “How do you feel about talking to a computer character”? with response categories of: “It’s kind of neat”, “It’s Okay”, and “It’s kind of strange”.

We conducted descriptive analyses to summarize the findings. We described (in numbers) the demographic and other behavioral characteristics of the subjects, and summarized open-ended feedback or comments provided by the subjects. We described the findings according to the focused areas that Veterans were asked about (i.e., satisfaction and user friendliness of the ECA). The short-term effectiveness of using the program was evaluated by examining number of subjects who have already set a quit date or made a quitting attempt or started quitting already.

3. RESULTS

As shown in Table 1, a majority of participants were male (4/6), unemployed (5/6), never married or divorced (4/6). The mean age of the participants (n = 6) was 56.3 (range, 37–76). All participants received education to high school or above and most (5/6) were familiar with Tablet use. Three out of the six participants had tried to quit smoking during the intervention period (i.e., during 14 days of using ECA-Q). However, none of these demographic characteristics are significantly related with the outcome variable (whether the participant tried to quit during the intervention period).

Table 1.

Demographic characteristics by quitting action during the intervention period (n = 6)

Independent variables	Tried to quit during the intervention period		Total
	Yes	No
Sex
Male	2	2	4
Female	0	2	2
Education
High school graduates	2	0	2
Some college	0	1	1
College graduates	1	2	3
Race
Caucasian	2	1	3
African American	1	2	3
General health condition
Excellent	1	0	1
Very good	1	1	2
Good	0	1	1
Fair	1	1	2
Employment status
Unemployed	2	3	5
Retired	1	0	1
Marital status
Never married	1	1	2
Married	0	1	1
Divorced	2	0	2
Widowed	0	1	1
Tablet use experience
No	0	1	1
Yes	3	2	5

The average number of cigarettes smoked daily by participants was 17 (range, 4–45). All the six participants reported ever making attempts to quit smoking. The average number of attempts made was 7.4 (range, 1–20). In the last year, the average number of attempts made was 1.5 times (range, 0–10).

Table 2 shows the smoking behavior and attitudes of the participants. One out of six participants used other tobacco products, five out of six participants were seriously thinking to quit in the next 30 days, and three out of six had total smoking ban at home. Only four out of six participants heard about third hand smoke while all had heard about e-cigarettes.

Table 2.

Smoking behaviors and attitude toward quitting at baseline by quitting action during the intervention period (n = 6)

Independent variables	Tried to quit during the intervention period		Total
	Yes	No
Seriously thinking to quit within next 30 days
Yes	2	3	5
No	1	0	1
Use other tobacco products
Yes	0	1	1 (pipe or cigar)
No	3	2	5
Number of other smokers in the household
0	1	2	3
1 or more	2	1	3
Smoking rules in the household at baseline
Total ban	2	1	3
Partial or no ban	1	2	3
Perceived importance of quitting at baseline
Very important or important	3	1	4
A little or not at all important	0	2	2
Perceived difficulty of quitting at baseline
Very difficult or difficult	0	2	2
A little or not at all difficult	3	1	4
Perceived confidence of quitting at baseline
Very confident or confident	2	1	3
A little or not at all confident	1	2	3
Heard of third hand smoke
Yes	1	3	4
No	2	0	2
Heard of e-cigarettes
Yes	3	3	6
No	0	0	0

At 14-day follow-up, the average number of cigarettes smoked daily by participants was 11 (range, 4–25). As shown in Table 3, five out of six participants reduced the average number of cigarettes smoked during the last 14 days and three out of six had tried to quit. Five out of six participants adopted stricter smoking rules in the household during the intervention period. The average number of cigarettes reduced is 6 with a range of 2–20 (data not shown). As shown in Table 4, the overall appraisal for the relational agent is very positive.

Table 3.

Smoking and quitting behavior at 14 day follow-up (n = 6)

	Number of participants reporting
Smoking rules in the household at follow-up
Total ban	5
Partial ban or no ban	1
Change of smoking rules in the household from baseline to at 14 days follow-up
More strict	5
No change or less strict	1
Tried to quit smoking in the last 14 days
None	3
Once or more	3
Reduced number of cigarettes smoked during the last 14 days
Yes	5
No	1

Table 4.

Patterns of interactions with the relational agent (n = 6)

	Number reporting
Used the program daily
Yes	6
No	0
Followed the guidance of the agent
Yes	6
No	0
Satisfaction with the guidance provided
Very satisfied or satisfied	6
Very dissatisfactory or dissatisfied	0
Usefulness of the information provided
Very useful or useful	6
Not useful at all	0
User friendly
Yes	6
No	0
Will recommend the agent to friend
Yes	6
No	0

3.1. Findings from Ancillary Analyses

Based on the open-ended questions, we found that almost all participants were impressed with the overall presentation of the ECA and described the agent as helpful (6/6), easy to interact (5/6), and lifelike (4/6). One participant reported the agent as somewhat skeptical. Participants reported several good and useful features of the agent: reinforcement and positive feedback (4/6), friendly, caring and knowledgeable (3/6), repetition of key messages (2/6), tips to deal with craving (2/6), supportive interpersonal relationship (1/6), and connection with me (1/6). Participants also reported several barriers or less important features of the agent: Log in difficulty (3/6), tablet is too heavy (3/6), too much pressure to name a quit date (2/6), only one interaction daily is not enough (2/6), the order of response is not good (1/6), and not lively as human (1/6). Additional suggestion included more opportunities to interact with the agent as needed (2/6), and the use of smartphone rather than the tablet (3/6). All participants thought that a comprehensive program with similar features would be acceptable to Veterans who smoke to support their quitting effort.

4. DISCUSSION

This study provides insights into the feasibility of using ECA to support the process of smoking cessation. All the participants were satisfied with the ECA program to support quit intention, would recommend the program to others and were impressed with the features of ECA. It was encouraging that all the participants used the program daily, which might be due to the short period of time required (<5 min) each time by the program. This underscores the importance of keeping each session short in a future full version of ECA-Q program to encourage regular daily use by the participants. While regular interaction and engagement may result in behavior change [21], it remains to be seen to what degree of engagement will result in smoking cessation in a similar program. In this study, three out of six (50%) of the respondents had set a quit date during the 14 days of ECA use. This outcome is promising, however, we did not collect further data on whether these three smoker Veterans sought additional support and eventually were successful in their quitting effort. In previous studies, repeated counseling support was associated with successful quitting [22,23]. Our future full version of the ECA-Q program will be available to provide long-term quitting support with a referral mechanism to a live counselor or the State Quitline, which would also include a long-term (6 months to 1 year) follow-up.

Some participants mentioned about the inconvenience in using Tablet computer and suggested using smartphone to deliver ECA for smoking cessation. In this study we were not able to use smartphone due to lack of funding, however, the feasibility of using ECA-Q in a Tablet computer suggests that the program would be easily acceptable if provided through smartphone [24–27]. Some participants were not happy with the fact that the system did not allow them to use multiple session in a single day (Note: The system allowed only one session each day, but they could use the same session multiple time within a day.). It is recommended that the future ECA systems be designed in a way so that users can move across sessions according to their needs and that the program be delivered for a longer period.

The strengths of this study is that it tested a novel ECA program that could be used to support smoking cessation among the Veterans and collected information from the users of the program in real life situation. Weaknesses include that there was no control group, short follow-up period and that the sample size was small. Also, all the information are based on self-reports without any biochemical validations. Another concern is that we provided $40 incentive to each participant for their participation in the study, mainly to cover participant’s travel-related expenses. There was no need to use the Tablet each day to receive this incentive. Therefore, we do not believe that the incentive would have any effect of subject’s utilization of the agent.

In conclusion, this study shows that a conversational agent, ECA-Q, is acceptable to Veterans who smoke to help them in setting a quit date with an ultimate goal of quit smoking. Insights gained from this study would be useful to redesign the current version of ECA-Q program for a future randomized controlled trial. The future trial should examine, beside whether the ECA-Q system works and acceptable, the efficacy of the program to improve smoking cessation.