Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Jul 22.
Published in final edited form as: Addict Behav. 2011 Nov 17;37(4):517–520. doi: 10.1016/j.addbeh.2011.11.015

Can Acceptance & Commitment Therapy be Delivered with Fidelity As a Brief Telephone-Intervention?

Allison Schimmel-Bristow a,b, Jonathan B Bricker b,c, Bryan Comstock b,d
PMCID: PMC3718556  NIHMSID: NIHMS474506  PMID: 22192865

Abstract

This study examined therapist adherence and competence of the first telephone-based Acceptance & Commitment Therapy (ACT) intervention for smoking cessation. An independent rater assessed 100% (n = 54 sessions) of the recorded phone sessions for adherence and competence on a 1 (never addressed) to 5 (addressed extensively) scale. A separate 20% random sample was rated by both the same independent rater and a second independent rater. The two coders were in perfect agreement (kappa of 1.0). Overall adherence (x = 4.61, SD = 0.63) and competence (x = 4.81, SD = 0.39) was high. Except self-as-context, benchmark ratings (i.e., 4 or more) were between 93% and 100%. Results suggest ACT therapy can be delivered competently over the telephone, opening many options for future research on the potential uses of telephone-based ACT for smoking and other behaviors.

Keywords: therapist fidelity, ACT, telephone, intervention, smoking

1. Introduction

Telephone-based interventions for smoking cessation address a critical problem: access to care (Lichtenstein, Zhu, & Tedesci, 2010). Telephone-based quitlines, available in all 50 U.S. states, all provinces in Canada, and many countries in the European Union, have saved millions of dollars in medical costs (Fiore et al., 2004).

1. 1 Why Fidelity Matters

The purpose of this brief report is to determine the extent to which a new telephone-based intervention for smoking cessation can be delivered with fidelity. Two rigorous and universal measures of fidelity are: (1) adherence and (2) competence (Bellg et al., 2004; Breitenstein, Gross, Garvey, Hill, Fogg, & Resnick, 2010; Collins, Eck, Kick, Schroter, Torchalla, & Batra, 2009). Adherence is defined as the extent to which the therapist delivered the intervention according to the protocol. Competence refers to the degree to which the therapist delivered the intervention skillfully (Breitenstein et al., 2010; Denton, Johnson, & Burleson, 2009; Long, Grubaugh, Elhai, Cusack, & Frueh, 2010). As recently reported in this journal (Collins et al., 2009), both measures were used to evaluate fidelity of a manual-based smoking cessation treatment.

Treatment fidelity is critical for a number of reasons. First, determining fidelity provides a stronger logical inference about the extent to which the outcomes of an intervention were due to the intervention (Bellg et al., 2004; Collins et al., 2009). Second, fidelity helps determine whether theory-based processes of the intervention were the primary mechanisms of change in outcomes. Third, it allows for precise replication and comparisons amongst interventions (Hester, Baltodano, Gable, Tonelson, & Hendrickson, 2003). Fourth, fidelity improves statistical power by reducing random and unintended variability. Finally, fidelity is critical for successful dissemination of an intervention because it provides practitioners clear guidelines for implementation (Breitenstein et al., 2010).

While treatment fidelity has been rarely reported in the past (for a meta-analysis, see Perepletchikova, Treat, & Kazdin, 2007), a growing number of studies, including some published in this journal (e.g., Collins et al., 2009), are now being conducted that focus specifically on fidelity (Brietenstein, et al., 2010; Hardeman, Michie, Fanshawe, Prevost, McLoughlin, & Kinmonth, 2008; Long et al., 2010). For example, Long et al., (2010) recently reported on therapists' fidelity to a manualized in-person cognitive-behavioral intervention for PTSD. They examined a 20% random sample of 57 sessions among the 20 study participants in a single-arm open trial. Fidelity ratings showed high levels of protocol adherence (77-100%) and competence (“very good” or higher).

1. 2 Acceptance & Commitment Therapy

Acceptance & Commitment Therapy (ACT) is an emerging psychological intervention focused on helping individuals develop a willingness to experience physical sensations, emotions, and thoughts while committing to changes guided by their values. Numerous randomized in-person delivered ACT treatment trials have been published to date, with a wide variety of positive outcomes (Ost, 2008; Powers, Zum Vorde Sive Vording, & Emmelkamp, 2009).

We recently reported on a single-arm pilot study of 14 smokers on the feasibility of the first telephone-based ACT intervention focused on smoking cessation (Bricker, Mann, Marek, Liu, & Peterson, 2010). Results were the following: (1) 100% of participants felt respected by the counselor and 93% said the intervention helped them quit; (2) from baseline to post-treatment, there was an increase in acceptance of physical cravings, emotions, and thoughts that cue smoking (p = .001, p = .038, p = .085, respectively), and in commitment to quitting (p = .01); and (3) in intent-to-treat analyses, 29% had not smoked at all in the past 12 months (93% retention). The 12-month post-treatment self-reported quit rate of 29% was over double the self-reported 12% quit rate of standard telephone-based smoking cessation interventions (Stead, Perera, & Lanchester, 2006).

1. 3 This Study

Besides Bricker et al. (2010), ACT has only been tested in face-to-face contexts. And while ACT for smoking cessation has been delivered with fidelity in face-to-face contexts (Gifford et al., 2004 Gifford et al., 2008; Hernandez et al., 2010), there are a number of reasons why it would be challenging to deliver this intervention with fidelity via telephone, including: (1) the ACT therapist's reading of the client's non-verbal behavior and responses greatly facilitates delivery of the intervention, (2) a number of ACT exercises are acted out or involve visual demonstrations. Moreover, each ACT intervention sessions is usually 45 minutes long (Hayes, Strosahl, & Wilson, 1999). With these challenges in mind, our novel ACT telephone intervention was specifically designed to be delivered (1) in brief sessions, (2) with an attunement to the client's words, voice inflection, and pauses, and (3) with metaphors and exercises adapted for rich and concrete verbal description. The important and novel question is whether this telephone-delivered ACT protocol could be delivered with fidelity.

Given our adaptations for telephone-delivered ACT, we hypothesize that ACT for smoking cessation can be delivered with fidelity by telephone.

2. Methods

2. 1 Participants

The sample of 14 participants was 57% racial minority (8 African American) and 64% below Federal poverty level. They were recruited from Seattle, Washington, and Dallas, Texas, by free advertisements through the media. Further information on the participants and their recruitment is available in Bricker et al. (2010).

2. 2 Procedure

2. 2. 1 Manualized Intervention

The ‘ACT Now’ telephone-delivered intervention, adapted from prior ACT for smoking cessation protocols (Gifford et al., 2004; Hernandez-Lopez, Luciano, Bricker, Roales-Neito, & Montesinos, 2009) provided up to five scheduled counseling sessions (average of 30 minutes for the first call and average of 15 minutes for all following calls) with a total of 90 minutes per participant. Following a delivery structure now common in quitlines (North American Quitline Consortium, 2009), all calls were proactive: once a participant was enrolled in the study, the counselor and the participant agreed on times in which the counselor would call the study participant. Call 1 (n=14) focused on the core ACT processes of values and acceptance. Call 2 (n = 14) focused on the ACT processes of being present and committed action. Call 3 (n = 12) focused on the ACT processes of cognitive defusion and committed action. Call 4 (n = 7) focused on the ACT processes of self-as-context and committed action. Call 5 (n = 5) reviewed material from prior calls and finalized a committed action plan for quitting. Quit dates were set collaboratively with the participants and thus occurred at any time during the intervention.

2. 2. 2 Therapist

The intervention was delivered by a licensed PhD-level psychologist with two years of prior training and experience in ACT, but no prior training in either ACT for smoking cessation or in any telephone-delivered interventions. During the intervention, the therapist had bi-weekly 60 minute case consultations from Dr. Elizabeth Gifford, an ACT-trained researcher and psychologist (Gifford et al. 2004; 2011).

2. 2. 3 Therapist adherence and competence

All 54 calls were recorded. Consistent with standard rating methods (Bellg et al., 2004; Collins et al., 2009; Long et al., 2010), (1) an independent rater coded 100% of the 54 therapy sessions, (2) a 20% random sample was then re-rated by the same independent rater for intra-rater reliability, and (3) the same 20% random sample was rated by a separate independent rater for inter-rater reliability. The amount of time between these ratings was one month. The raters had 120 hours of total training in ACT and the telephone fidelity rating protocol (Bricker & Hayes, 2009).

2. 3 Measures

The ‘ACT Now’ telephone-delivered smoking cessation intervention was rated using a multi-dimensional ‘ACT Now’ therapist fidelity rating manual (Bricker & Hayes, 2009). The ‘ACT Now’ smoking cessation manual was adapted from two in-person ACT for smoking cessation intervention fidelity rating manuals (Gifford et al., 2004; Hernandez-Lopez et al., 2009) which reported good reliability (alpha ranged from .70 to .99) and was based on the Barber & Crits-Christoph (1996) fidelity rating scale development methodology. The first set of dimensions rated was the six core ACT processes: (1) Values: the deeply meaningful things in a person's life, or, chosen life directions, (2) Committed action: specific and concrete action plans guided by one's values, (3) Acceptance: openness to experience urges, emotions, and thoughts as they are and without any intent that they change, (4) Being present: being fully aware of the present moment with openness, interest, and receptiveness, (5) Cognitive defusion: stepping back from the process of thinking, (6) Self-as-context: unchanging part of a person that witnesses everything that a person experiences. The second set of dimensions rated was basic behavioral intervention processes: (a) therapy support (i.e., therapist providing emotional support) and (b) contingency management [i.e., therapist providing feedback and overt reinforcement (e.g., praise) on past practice of specific action plans]. The third and final set of ratings was: (a) overall adherence and (b) overall competence. Each dimension was rated on a 1 to 5 scale: 1 “never addressed,” 2 “addressed a little,” 3 “addressed somewhat,” 4 “addressed considerably,” and 5 “addressed extensively.” All 54 calls were rated on these dimensions.

2. 4 Analyses

Each of the 54 phone sessions ratings were entered into SPSS version 16.0. The level of intra-rater and inter-rater agreement was evaluated with the Cohen's Kappa statistic in STATA version 10.0. Kappa provides a metric of observed agreement beyond chance divided by maximum beyond chance possible for the sample tested. Following the Kappa calculation methods found effective in Malpica et al. (2005), we used the dichotomized kappa that is easy to interpret (Fleiss, 1981). Responses were coded such that 4 or 5 were considered as “addressed” and responses of 1, 2, or 3 were considered as “not addressed.” The means and standard deviations were then assessed for each dimension rated. Percentages were calculated to determine the frequency with which each of the dimensions had a benchmark rating of 4 (“addressed considerably”) or more. Note that 100% of the 54 telephone were used in the calculation of the fidelity results while the 20% random samples were used to calculate the reliabilities.

3. Results

As shown in Table 1, The kappa statistic for both the inter- and intra-rater reliabilities was 1.0 for each of the dimensions on the rating scale. The mean ratings of overall adherence and overall competence were high, with the exception of self-as-context. The percentage of calls attaining the benchmark ranged from 93% to 100% across the dimensions rated, again with the exception of self-as-context.

Table 1. Intra- and inter-rater reliability kappas, means, standard deviations, and benchmark attainment (range 1-5) for all fidelity ratings of the ACT Now telephone-based smoking cessation intervention.

Rating Item Intra-rater kappa Inter-rater kappa Mean SD Percentage (%) attaining benchmark
(rating of 4 or more)
Overall Adherence (All Calls) 1.00 1.00 4.61 0.63 100%
Overall Competence (All Calls) 1.00 1.00 4.81 0.39 100%
Values (Call 1) 1.00 1.00 5.00 0.00 100%
Acceptance (Call 1) 1.00 1.00 4.87 0.35 100%
Committed Action (All Calls) 1.00 1.00 4.83 0.50 100%
Being Present (Call 2) 1.00 1.00 4.67 1.05 93%
Cognitive Defusion (Call 3) 1.00 1.00 4.17 1.19 93%
Self as Context (Call 4) 1.00 1.00 2.43 1.62 43%
Therapist Support (All Calls) 1.00 1.00 4.91 0.29 100%
Contingency Management (Calls 2-5) 1.00 1.00 4.67 0.62 93%
Open in a new tab
*

n=14, 54 total calls

4. Discussion

This brief report addressed the novel and important topic of the extent to which the first telephone-delivered ACT intervention was delivered with fidelity. Supporting our hypothesis, the means and benchmark ratings consistently showed that, with the exception of self-as-context, the core ACT processes were delivered with high fidelity and little variation. The level of fidelity observed in this small-sample study is similar to that of both of the Gifford et al. (2004; 2011) randomized trials of in-person ACT for smoking cessation.

However, the self-as-context ratings were relatively low. One reason is that the therapist found that self-as-context, the most abstract concept of all the ACT core processes, was difficult to convey over the phone using only words. Alternatively, self-as-context may have been low due to having fewer completed calls (n=7). Future research should focus on simplifying self-as-context metaphors and exercises, as well as supplementing their illustration through pictures, videos, and audio recordings.

Results from this study compare well with the Long et al. (2010) fidelity study. Although both had the key limitations of being single-arm trials using small sample sizes (n = 14 in the current study; n = 20 in the Long et al. study), fidelity results showed that both interventions were delivered with high levels of adherence and competence. Further research is needed in order to determine to what extent ACT can be delivered with fidelity via the telephone for a wide variety of behaviors (e.g., smoking, alcohol, depression).

This brief report provides the first evidence, with highly reliable ratings, that a relatively brief telephone-delivered ACT intervention can be delivered with generally high fidelity. This study is an important first step in the direction of new research on the wide potential applications of ACT as a telephone-delivered intervention for smoking and other behaviors.

Highlights.

  • Therapist fidelity of the first telephone-based Acceptance & Commitment Therapy (ACT) intervention.

  • Manual-based treatment for smoking cessation.

  • Overall adherence and competence was generally high.

  • First evidence that ACT therapy can be delivered competently over the telephone.

Acknowledgments

We gratefully acknowledge the volunteer study participants and Dr. Elizabeth Gifford for her valuable case consultation. The authors also thank Anya Luke-Killam for editorial assistance in the preparation of this manuscript and Lara Schiff and Christopher Wysznski for their conceptual input on the manuscript.

Role of Funding Sources: This work was funded by a generous intramural grant from the Fred Hutchinson Cancer Research Center and partially supported by National Cancer Institute grant R01 CA151251-01.

Footnotes

Contributors: Allison Schimmel-Bristow coded the fidelity data and was primary writer of the manuscript. Jonathan Bricker supervised the coding and was secondary writer of the manuscript. Bryan Comstock analyzed all of the data and commented on the write-up of the manuscript.

Conflict of Interest: Authors have no conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Barber JP, Crits-Christoph P. Development of a therapist adherence/competence rating scale for Supportive-Expressive Dynamic Psychotherapy: A preliminary report. Psychotherapy Research. 1996;6(2):81–94. doi: 10.1080/10503309612331331608. [DOI] [PubMed] [Google Scholar]
  2. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, Ogedegbe G, Orwig D, Ernst D, Czajkowski S. Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology. 2004;23(5):443–451. doi: 10.1037/0278-6133.23.5.443. [DOI] [PubMed] [Google Scholar]
  3. Breitenstein SM, Gross D, Garvey CA, Hill C, Fogg L, Resnick B. Implementation fidelity in community-based interventions. Research in Nursing & Health. 2010;33(2):164–173. doi: 10.1002/nur.20373. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bricker JB, Mann SL, Marek PM, Liu J, Peterson AV. Telephone-delivered acceptance and commitment therapy for adult smoking cessation: A feasibility study. Nicotine & Tobacco Research. 2010;12(4):454–458. doi: 10.1093/ntr/ntq002. [DOI] [PubMed] [Google Scholar]
  5. Bricker JB, Hayes SC. Fidelity Rating Manual for the “ACT Now” Telephone Smoking Cessation Intervention. 2009 Unpublished Manual. [Google Scholar]
  6. Collins SE, Eck S, Kick E, Schröter M, Torchalla I, Batra A. Implementation of a smoking cessation treatment integrity protocol: Treatment discriminability, potency and manual adherence. Addictive Behaviors. 2009;34(5):477–480. doi: 10.1016/j.addbeh.2008.12.008. [DOI] [PubMed] [Google Scholar]
  7. Denton WH, Johnson SM, Burleson BR. Emotion-focused therapy-therapist fidelity scale (EFT-TFS): Conceptual development and content validity. Journal of Couple & Relationship Therapy. 2009;8(3):226–246. doi: 10.1080/15332690903048820. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Fleiss JL. Statistical methods for rates and proportions. New York, NY: John Wiley and Sons; 1981. [Google Scholar]
  9. Fiore MC, Croyle RT, Curry SJ, Cutler CM, Davis RM, Gordon C, et al. Preventing 3 million premature deaths and helping 5 million smokers quit: A national action plan for tobacco cessation. American Journal of Public Health. 2004;94(2):205–210. doi: 10.2105/AJPH.94.2.205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Gifford EV, Kohlenberg BS, Hayes SC, Antonuccio DO, Piasecki MM, Rasmussen-Hall ML, et al. Applying a functional acceptance based model to smoking cessation: An initial trial of acceptance and commitment therapy. Behavior Therapy. 2004;35(4):689–705. doi: 10.1016/S0005-7894(04)80015-7. [DOI] [Google Scholar]
  11. Hardeman W, Michie S, Fanshawe T, Prevost AT, McLoughlin K, Kinmonth AL. Fidelity of delivery of a physical activity intervention: Predictors and consequences. Psychology and Health. 2008;23(1):11–24. doi: 10.1080/08870440701615948. [DOI] [PubMed] [Google Scholar]
  12. Hayes SC, Strosahl K, Wilson KG. Acceptance and Commitment Therapy: An experiential approach to behavior change. New York: Guilford Press; 1999. [Google Scholar]
  13. Hernández-López M, Luciano MC, Bricker JB, Roales-Neito JG, Montesinos F. Acceptance and commitment therapy for smoking cessation: A preliminary study of its effectiveness in comparison with cognitive behavioral therapy. Psychology of Addictive Behaviors. 2009;23(4):723–730. doi: 10.1037/a0017632. [DOI] [PubMed] [Google Scholar]
  14. Hester P, Baltodano H, Gable R, Tonelson S, Hendrickson J. Early Intervention with children at risk of emotional/behavioral disorders: A critical examination of research methodology and practices. Education & Treatment of Children. 2003;26(4):362–381. [Google Scholar]
  15. Lichtenstein E, Zhu SH, Tedeschi GJ. Smoking cessation quitlines: An underrecognized intervention success story. American Psychologist. 2010;65(4):252–261. doi: 10.1037/a0018598. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Long ME, Grubaugh AL, Elhai JD, Cusack KJ, Knapp R, Frueh BC. Therapist fidelity with an exposure-based treatment of PTSD in adults with schizophrenia or schizoaffective disorder. Journal of Clinical Psychology. 2010;66:383–393. doi: 10.1002/jclp.20657. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Malpica A, Matisic JP, Niekirk DV, Crum CP, Staerkel GA, Yamal JM, et al. Kappa statistics to measure interrater and intrarater agreement for 1790 cervical biopsy specimens among twelve pathologists: Qualitative histopathologic analysis and methodologic issues. Gynecologic Oncology. 2005;99(3):38–52. doi: 10.1016/j.ygyno.2005.07.040. [DOI] [PubMed] [Google Scholar]
  18. North American Quitline Consortium. All Quitline Facts: An Overview of NAQC 2008 Annual Survey of Quitlines. Author; 2009. [Google Scholar]
  19. Ost LG. Efficacy of the third wave of behavioral therapies: A systemic review and meta-analysis. Behavior Research and Therapy. 2008;46(3):296–321. doi: 10.1016/j.brat.2007.12.005. [DOI] [PubMed] [Google Scholar]
  20. Perepletchikova F, Treat TA, Kazdin AE. Treatment integrity in Psychotherapy research: analysis of the studies and examination of the associated factors. Journal of Consulting and Clinical Psychology. 2007;75(6):829–841. doi: 10.1037/0022-006X.75.6.829. [DOI] [PubMed] [Google Scholar]
  21. Powers MB, Zum Vorde Sive Vording MB, Emmelkamp PM. Acceptance and Commitment Therapy: A meta-analytic review. Psychotherapy and Psychosomatics. 2009;78(2):73–80. doi: 10.1159/000190790. [DOI] [PubMed] [Google Scholar]
  22. Stead LF, Perera R, Lancaster T. Telephone counseling for smoking cessation. Cochrane Database Systematic Reviews. 2006;(3):CD002850. doi: 10.1002/14651858.CD002850.pub2. [DOI] [PubMed] [Google Scholar]

RESOURCES