Practice-Based Referrals to a Tobacco Cessation Quit Line: Assessing the Impact of Comparative Feedback vs General Reminders

Abstract

PURPOSE We undertook a study to assess the impact of comparative feedback vs general reminders on practice-based referrals to a tobacco cessation quit line and estimated costs for projected quit responses.

METHODS We conducted a group-randomized clinical trial comparing the impact of 6 quarterly (18 months) feedback reports (intervention) with that of general reminders (control) on practice-based clinician referrals to a quit-line service. Feedback reports were based on an Achievable Benchmark of Care approach using baseline practice, clinician, and patient survey responses, and referrals per quarter. Comparable quit responses and costs were estimated.

RESULTS Three hundred eight clinicians participated (171 family medicine, 88 internal medicine, 49 obstetrics-gynecology) from 87 primary care practices in Michigan. After 18 months, there were more referrals from the intervention than from the control practices (484 vs 220; P <.001). Practice facsimile (fax) referrals (84%, n = 595) exceeded telephone referrals (16%, n = 109), but telephone referrals resulted in greater likelihood of enrollment (77% telephone vs 44% fax, P <.001). The estimated number of smokers who quit based on the level of services utilized by referred smokers was 66 in the feedback and 36 in the gentle reminder practices.

CONCLUSION Providing comparative feedback on clinician referrals to a quit-line service had a modest impact with limited increased costs.

INTRODUCTION

Smoking cessation interventions have proved to be effective in primary care settings according to the systematic reviews of controlled clinical trials that resulted in Clinical Practice Guidelines on Smoking Cessation by the Public Health Service.^1,2 Clinicians report various reasons for not following the guidelines, such as focusing on acute or chronic care rather than preventive care, having little training in giving brief advice to quit smoking, and not being subject to accountability or feedback.^3,4 Survey findings show that physicians understand the importance of smoking cessation and espouse its value, but they often do not implement the key elements of office-based methods.^3,4 Physicians rarely schedule smokers for follow-up visits or arrange referrals for support services.^1,3–5 Several community-based and health-system-based studies have shown fairly high rates of long-term smoking cessation (20% to 36%) by combining physician identification, advice, and referral for follow-up care with telephone support counseling.^6–11 Telephone counseling services for smoking cessation (quit lines) have become widely available through health plans and state or national services,¹² but they are often underutilized. Methods to enhance clinician referrals to quit-line services are needed.

Feedback on clinical performance has been studied in numerous randomized clinical trials and reported in systematic reviews to have modest effects.¹³ Comparing personal performance with peer performance should be a powerful motivator to change behavior.^14–17 Keife et al¹⁸ reported improved preventive care for diabetic patients by providing clinicians feedback based on Achievable Benchmarks of Care (ABC). The ABC approach identifies the performance of the top 10% of clinicians as the benchmark.¹⁸ Achievable benchmarks that were based on actual performance seemed more realistic to clinicians than preset standards, and they resulted in improved care.¹⁸

We found 9 studies that involved feedback on physician performance in smoking cessation.^18–26 All studies used chart audit to provide feedback, and none reported direct observation of practice. One based feedback on encounter and pharmacy claims.²⁶ The Task Force on Community Preventive Services,²⁷ after reviewing a few studies, reported insufficient evidence to conclude that feedback on smoking cessation increases clinician advice to quit smoking or patient tobacco cessation. No reported studies have based feedback on achievable benchmarks for clinician performance in smoking cessation. Business models of feedback to employees stress that the most effective feedback should be clear, specific to the expected performance, reliable, comparable with peer performance, and repeated over time.²⁸ We developed ABC feedback based on actual clinician performance in making quit-line referrals. We assessed the impact of comparative feedback or general reminders on practice-based referrals to a tobacco cessation quit line and estimated the costs for the projected quit responses.

METHODS

Study Design

We conducted a cluster-randomized clinical trial assessing the impact of quarterly comparative feedback (intervention) vs general reminders (control) to primary care clinicians within practices on their performance in generating referrals to a telephone quit-line service over an 18-month period between January 1, 2003, to June 30, 2004. We randomly assigned each practice to comparative feedback or general reminders so that all participating clinicians within a practice received the same type of feedback. For practices assigned to the intervention group, clinicians were given quarterly comparative feedback on their individual and group referrals and how their referral activity compared with the mean referrals based on the overall top 10% of clinicians. Estimated quit responses and the associated incremental costs of the intervention were calculated based on the level of service obtained by referred smokers.

The quit-line services for this study were provided by Blue Cross Blue Shield of Michigan (BSBEM), which serves nearly 5 million enrollees in Michigan. The BCBSM Quit the Nic Quit Line is a 24-hour service for smoking cessation counseling and relapse prevention support provided by trained nurses using computer counseling software. Smokers are offered 2 levels of service. Level 1 requires participants to set a quit date in the next 14 days and includes an enrollment intake session (lasting 45 minutes with motivational support to set a quit date and prevent relapse), a mailed quit kit, and 6 proactive telephone sessions at intervals of 1, 3, 7, 14, 30, and 60 days after the quit date, with follow-up at 90, 180, and 360 days after the quit date.⁷ Level 2 includes a brief intake session (lasting 10 minutes), a mailed quit kit, and several callbacks inviting them into level-1 service. All participants are advised to use pharmacotherapy to assist with withdrawal symptoms if recommended by their physician. The Quit Line service reports long-term success rates of 25% to 30% for level-1 enrollees, a finding consistent with other reports.^7,9–11 The Quit Line service agreed to provide counseling services to all referred smokers from study participants regardless of their health plan.

Interventions

ABC feedback was based on a method developed by Keife and colleagues¹⁸ and was determined to be the mean number of referrals per quarter for the top 10% of the clinicians. The benchmark calculation for individual clinician referrals was based on an estimate of smoker visits per quarter calculated to account specifically for each clinician’s unique baseline characteristics, including half-days worked per week, adult patients per clinic, and smoking prevalence within the clinic population. A baseline quarter (October 2002 to December 2002) was used to set the benchmark for the first quarter (beginning January 1, 2003) of the comparative feedback intervention. Clinicians received either a mailed comparative feedback report or a general reminder about Quit Line services each quarter for 6 quarters. The benchmark referrals were reset after each quarter based on the clinicians’ performance during the previous quarter.

The comparative feedback report (Figure 1 ▶) was a 1-page report showing 2 graphs. The first graph displayed the quarter benchmark for referrals, the individual clinician, his or her practice group, and the performance of the study group in numbers of referrals and percentage relative to the benchmark. The second graph showed the actual number of referrals made by the individual clinician per quarter.

Figure 1.

Example of a Quit Line referral report.

The general reminders, which were sent to control practices and their clinicians on the same quarterly basis as the feedback reports, were simple postcards informing them only of the Quit Line services.

During the baseline quarter before the feedback intervention, all 308 participating clinicians received a mailed compact disk (developed by W.C.W. and J.S.H.) that included an overview of the clinical practice guidelines on smoking cessation, case examples, and Quit Line referral information. Clinicians who completed a posttest questionnaire (n = 53, 17%) received 2 hours of CME credit, a response rate that was consistent for both intervention and control groups.

Recruitment and Participants

Because the study was a collaboration between Michigan State University and the health plan Quit Line service, all participating clinicians and practices accepted patients covered by the plan. From March to July 2002, we recruited primary care clinicians (in family medicine, internal medicine, and obstetrics-gynecology) from a health plan list of 3,777 primary care practices. Only 858 practices seemed to fit the definition for usual primary care. Of these 858 practices, 102 agreed to participate during telephone recruitment, and completed baseline survey forms were received from 87 (85.3%); 308 of 371 clinicians (83%) within the 87 practices completed baseline survey forms.

Practice directors completed a prestudy survey form assessing practice characteristics (Table 1 ▶). Clinicians completed a prestudy survey form assessing their characteristics (Table 2 ▶), including the number of half-days worked and adult patients seen in each practice.

Table 1.

Characteristics of Participating Practices

Characteristic	Total No.	Interventin No.	Control No.	PValue
Recruited, No.	87	43	44
Location, No.				.69*
Southeast Michigan	16	9	7
Mid-Michigan	23	10	13
Southwest Michigan	17	10	7
Northern Michigan	31	14	17
Size of community, No.				.83*
<5,000	10	6	4
5,000–10,000	7	3	4
10,000–25,000	26	12	14
25,000–100,000	23	10	13
>100,000	21	12	9
Proximity to major city, No.				.76*
Not within 25 miles	29	15	14
Within 25 miles	58	28	30
Practice type, No.				.72*
Solo	13	6	7
2-person	16	6	10
Group practice	37	20	17
Multispecialty group practice	21	11	10
Patients aged >65 years, mean % (SD)		24.7 (17.8)	21.0 (15.4)	.34†
Patients white, non-Hispanic, mean % (SD)		78.0 (23.6)	82.2 (19.0)	.38†
Patients uninsured / Medicaid, mean % (SD)		19.5 (18.4)	23.3 (20.2)	.40†
Estimated smoker visits per quarter, total No.		22,163	23,537

* χ² test.

† t test.

Table 2.

Characteristics of Participating Clinicians

Characteristic	Total No.	Intervention No.	Control No.	PValue
Recruited	308	163	145
Sex				.69*
Male	163	88	75
Female	145	75	70
Clinician type				.95*
Physician	234	124	110
Nurse-practitioner	31	17	14
Physician’s assistant	43	22	21
Specialty				.45*
Family medicine	171	85	86
Internal medicine	88	50	38
Obstetrics-gynecology	49	28	21
Smoking status				.45†
Current	3	2	1
Past	48	29	19
Never	254	129	125

* χ² test.

† Fisher’s exact test.

The 87 participating practices were located across regions of Michigan, including rural, urban, and suburban locations (Tables 1 ▶ and 2 ▶). The practices comprised 308 clinicians, with 55% (171) practicing family medicine, 29% (88) general internal medicine, and 16% (49) obstetrics-gynecology.

Baseline exit surveys of patients were conducted in each practice before the study for 2 half-days in which most clinicians were in the practice. Research assistants approached all patients (aged 18 years and older) as they exited from their visit. A brief (5-minute), 1-page questionnaire assessed clinician’s use of the 5 As (ask, advise, assess, assist, arrange) during the visit; age; sex; insurance status; race/ethnicity; smoking status; and, if a current smoker, number of cigarettes smoked per day and readiness to quit. The questionnaire was pilot tested in non–study clinics, and questions were adapted from previously developed exit surveys of smoking cessation.^29,30

Of the 3,686 eligible patients, 3,239 (87.9%) were approached for inclusion; 1,992 patients completed the questionnaire (54% of all eligible and 61.5% of all approached). Illness and lack of time were main reasons for not completing the questionnaire. Nineteen (19%) of surveyed patients were current smokers (n = 371). The percentages of clinician asking, advising, assessing, assisting, and arranging follow-up were 58%, 42%, 34%, 22%, and 8%, respectively. There were no significant differences in prevalence of smoking or delivering of the 5 As between intervention and control practices at baseline.

All referrals to the Quit Line by study clinicians were identified by the Quit Line staff and categorized based on clinician; practice; level of service utilized—level 0 (clinician advice alone, no Quit Line intake), level 1 (enrolled in Quit Line), and level 2 (intake, information only); and type of referral—facsimile (fax) or telephone.

The study design, questionnaires, and survey forms were all approved by the Michigan State University Institutional Review Board and 25 local boards.

We estimated 12-month quit responses based on the number of referrals in each group and the level of service obtained. We estimated the quit responses based on the expected quit responses reported in the literature for the various levels of service and on the previous results reported by the services for level 1 (enrollment and Quit Line use with usual quit rate of 25%), level 2 (brief intake and information only with usual quit rate of 15%), and level 0 (no intake and clinician advice alone with usual quit rate of 10%).^{7,11,29,31–33}

Measures and Outcomes

Key measures and outcomes included (1) changes from baseline to postintervention (18 months) in clinician referrals in both intervention and control groups, (2) estimated quit responses of referred smokers, and (3) estimated incremental costs of the feedback intervention.

Analyses

Data were pooled and maintained by the health plan data manager and transferred to Michigan State University for evaluation and statistical analysis. We determined baseline differences between practice, clinician, and patient characteristics using χ² tests and t tests, as appropriate. The Mantel-Haenszel χ² test for repeated measures was used to assess the difference in the primary measure (number of referrals) between the intervention and control groups, over time, which was the main unit of analysis. The study had the power to detect a 25% difference, which required nearly 80 practices and 300 clinicians. Intraclass correlation (ICC) was calculated (see the online-only Supplemental Appendix, which can be found at http://www.annfammed.org/cgi/content/full/5/2/135/DC1). All analyses were conducted using SAS statistical software, version 9.1 (SAS Institute Inc, Cary, NC).

Resource costs were used to calculate the costs of services in both groups. The mean annual salary and benefit cost were used for nurse ($54,600), analyst ($72,602), clerk ($26,000), and statistician ($103,125) staff. These costs were applied to 173 hours of the nurses’ time for receiving and processing telephone and fax referrals and to 236 hours of administrative staff time to develop and send the feedback reports as well as other program-related tasks. Actual supply costs were also used for printing and mailing feedback reports, letters, quit kits, and smoking cessation program materials. Costs were categorized as fixed or variable, an important difference in that the latter costs are incurred and change with intervention volumes. Because the Quit Line program was already purchased, available, and staffed, its cost represented a fixed cost. Increased use of the program associated with the interventions did not result in an incremental marginal cost increase relative to the control group, although in the long run, more volume can be expected to generate higher mean costs.

RESULTS

Absolute Quit Line referrals from baseline to 18 months (Figure 2 ▶) showed a significant difference between intervention and control groups over time. Both groups increased during the fifth quarter, which may be explained by usual increases in commitment to quit smoking during January. Overall, referrals were very modest, considering the number of estimated visits by smokers per quarter. Because of a high intraclass correlation (ICC = .902) and a lack of adequate power to detect significant differences, individual clinician- and practice-level analyses were not completed.

Figure 2.

Clinician referrals to Quit Line by quarter.

Mantel-Haenszel χ²: P <.001.

We also analyzed the type of referral (office fax vs telephone). There were far greater referrals by fax than by telephone. The intervention practices had 427 (88.2%) fax and 57 (11.8%) telephone referrals. The control practices had 168 (76.4%) fax and 52 (23.6%) telephone referrals. Telephone referrals (84 enrolled vs 25 not enrolled) compared with fax referrals (249 enrolled vs 342 not enrolled) were significantly more likely than fax referrals to result in Quit Line enrollment (77.1% vs 42.1%, P <.001).

More estimated quits (Table 3 ▶) occurred in the intervention practices than in the control practices (66 vs 36 quits).

Table 3.

Estimated Quits by Level of Enrollment in Quit Line

Enrollment Status	Usual Quit Rate (%)	Intervention (n = 484)	Control (n = 220)
Level 1, enrolled in Quit Line	25	73 × 0.25 = 18.25	72 × 0.25 = 18.00
Level 2, brochure and callbacks	15	126 × 0.15 = 18.90	62 × 0.15 = 9.30
No enrollment, clinician advice only	10	285 × 0.10 = 28.50	86 × 0.10 = 8.60
Total estimated quits		65.65	35.90

* Actual referrals x estimated quit rate = number of quits.

The relative cost per outcome (Table 4 ▶) based on estimated quits shows a higher cost in the intervention group, which was expected because more costs were incurred as a result of program implementation. Incremental costs are summarized in the bottom half of the table, which exclude program development costs, such as feedback report design and testing and database compilation. Incremental costs reflect the ongoing expense of sustaining this type of smoking cessation service of comparable feedback to clinicians.

Table 4.

Program Cost per Referral, Enrollee, and Quit

Outcome	Total $	Intervention $	Control $
Total cost per outcome
Cost per referral	33.89	38.60	23.60
Cost per QTN enrollee	71.65	93.68	38.92
Cost per quit	235.88	285.72	145.28
Incremental cost per outcome
Cost per referral	21.69	23.51	17.70
Cost per QTN enrollee	45.85	57.07	29.18
Cost per quit	150.93	174.04	108.93

QTN = telephone Quit Line service.

Figure 3 ▶ shows the final summary of results, including practices and clinicians who received either intervention feedback or control announcements during the entire study period.

Figure 3.

Participant disposition and summary outcomes.

DISCUSSION

An ABC feedback approach for clinicians can be developed and implemented by a health plan based on comparative referrals to quit-line services and appears to increase referrals over time, though the impact is very modest considering the magnitude of estimated visits by smokers. The ABC benchmark requires data and mechanisms usually not developed by health plans, such as linking referred smokers to individual clinicians, estimating the prevalence of smokers in practices, and knowing clinical time spent in practice per clinician. Being at the interface of clinical care and public health, most quit lines do not feed back data to referring clinicians. The added cost to develop clinician-specific feedback in our study was about $65 per estimated quit, which is minimal when considering almost twice the number of quits in the feedback group and the long-term benefits of smoking cessation.

It has been reported that less than 5% of smokers will accept referrals to receive special smoking cessation services such as group counseling.^1,34 Although we detected a significant difference in referrals between our intervention and control groups, 704 referrals from more than 300 clinicians during 18 months is a very small response even when a low percentage of smokers generally agree to be referred. Estimating that 10% to 20% of smokers in primary care practice are usually willing to quit, there were potentially 2,200 to 4,500 referrals per quarter. The practices and clinicians were representative of primary care practice across Michigan and had performances on the 5 As guidelines that were consistent with other reports.^2–4

Even though the number of fax referrals was greater than telephone referrals, the probability of enrollment was greater for telephone referrals, which may be explained by the level of self-motivation of smokers who telephone on their own initiative. Some clinicians may have referred smokers by fax to insure their feedback credit.

There are several limitations to this study. A high intraclass correlation and lack of statistical power did not permit analysis of individual clinician or practice differences. The exit surveys assessed only 2 half-days per practice and may not accurately reflect longitudinal changes for individual clinicians, but they are valid for the entire group. We estimated the number of quits based on the reported experience of the Quit Line, which uses self-reports.^35–37 Smoker visits per quarter were estimated from baseline surveys only.

We interviewed by telephone 28 of the top-referring clinicians (top 10%) in this study. They consistently reported having a method (usually by staff) to identify all smoking patients, offered referrals to all interested smokers, and appreciated having 1 telephone counseling service. Primary care offices should consider having clinical staff identify tobacco use, offer advice to quit, and initiate referrals,³⁷ which is consistent with the new Ask and Act initiative of the AAFP (American Academy of Family Physicians).³⁸ Electronic medical records could also be used to incorporate easy documentation of the 5 As, offer simple cueing supports for clinicians, and tailor pharmacotherapies and referral services based on individual patient health care coverage.³⁹ Clinicians may also be more motivated to refer smokers for services if they receive information on the outcomes of their patients. Quit lines and health plans should provide reports to clinicians on referred smokers. In fact, the staff-model managed care plan within this study has decided to provide ongoing reports on referred smokers based on our findings.

In conclusion, providing simple, clear, and comparable feedback on referrals to a quit-line service had a modest impact on the referral performance of clinicians and may require a major integration of practice and system supports to become truly effective.