Perspectives in Implementing a Pragmatic Pediatric Primary Care–Based Intervention Trial

Abstract

The 2013 U.S. Preventive Services Task Force (USPSTF) concluded that behavioral interventions are effective in reducing initiation of smoking in youth, recommending primary care clinicians provide education or brief counseling to prevent initiation, and that there are promising trends toward behavioral interventions improving cessation in this population. Our primary care–based intervention RCT conducted between 2000 and 2004, Air It Out, informed these USPSTF recommendations. Our trial was designed to determine whether a pediatric primary care practice–based smoking prevention and cessation intervention would be effective in increasing abstinence rates among adolescents under usual clinic conditions, to inform clinical practice. Therefore, the trial was designed to be largely a pragmatic trial. In this paper, we describe where each of the Air It Out study components falls along the pragmatic–explanatory continuum regarding participant eligibility criteria, intervention and comparison condition design, follow-up and outcomes, compliance and adherence assessments, and analysis. Such an assessment assists researchers by providing a framework to guide decisions regarding study design and implementation. We then share a few principles and lessons learned in developing and implementing the primary care–based intervention trial, focusing on study setting selection, engaging providers who will be delivering the intervention and the target population who will be receiving it in designing the trial and interventions to be tested, and the need to carefully plan recruitment and retention procedures. The hope is to increase the number of well-designed studies that can be included in the evidence reviews to guide future USPSTF recommendation statements.

Introduction

Cigarette smoking is one of the most important preventable causes of premature disease, disability, and death. The majority of adult smokers, about 90%, report initiating smoking before age 18 years.¹ More than 3.6 million (15.8%) U.S. youth currently smoke.² Of those currently smoking, 80% will continue to smoke into adulthood; among those who continue to smoke, half will die 13 or more years earlier than their nonsmoking peers.³ Given that the developing brains of children and adolescents are particularly susceptible to the addictive potential of nicotine,⁴ it is critical to address both prevention of uptake and treatment to support cessation in youth.

The pediatric primary care setting provides a unique opportunity to deliver interventions to youth to prevent smoking initiation and treat nicotine addiction. Pediatricians are well positioned to address tobacco use: they are identified by youth as their preferred source of information regarding smoking and cessation,^5,6 have long-term relationships with their patients, and have many opportunities to intervene with nonsmokers to prevent initiation and with smokers to provide treatment. However, they report providing limited assistance. In a 2010 survey of American Academy of Pediatrics members,⁷ most pediatricians (81%) reported advising their adolescent patients who smoke to quit, but only half (48%) reported helping adolescents assess their reasons to continue smoking versus quitting, only a third (32%) discussed strategies for quitting, and few provided print and other quit materials (15%) or referred their patients to a cessation program or quitline (13%). This represents a critical missed opportunity.

The most recent U.S. Preventive Services Task Force (USPSTF) (2013)⁸ concluded that behavioral interventions are effective in reducing initiation in nonsmoking youth, suggesting that primary care clinicians provide education or brief counseling to prevent initiation in school-aged youth.⁹ The Public Health Service (PHS) Clinical Practice Guideline¹⁰ states that counseling is effective in treating adolescent smokers and recommends that the counseling steps follow the 5A’s model of care (Ask about tobacco use, Advise cessation, Assess motivation to quit, Assist in quitting, and Arrange follow-up), which is a preventive services screening and brief counseling intervention that improves tobacco cessation rates in adults.¹⁰ By contrast, the USPSTF noted limited evidence on the effectiveness of practice-based cessation interventions for youth,⁸ in part due to the lack of studies testing primary care office–based cessation interventions⁹ for this population. The five trials reviewed by the USPSTF⁸ that were conducted in the primary care or dental setting involved the provision of brief advice to quit smoking or counseling using the 5A’s model by healthcare providers; three of the trials used trained counselors to provide intensive counseling and telephone follow-up. Overall, these studies showed a promising trend toward behavioral interventions improving cessation. To date, none of the studies investigating the use of pharmacotherapy in youth have reported effective long-term cessation.¹¹

Our research team conducted one of the trials reviewed by the USPSTF as noted above. The purpose of our trial, called Air It Out, was to determine whether a pediatric primary care practice–based smoking prevention and cessation intervention would be effective in increasing abstinence rates among adolescents.¹² In this trial, conducted between 2000 and 2004, eight pediatric primary care clinics were randomly assigned to either an intervention or a usual care control condition, the latter defined as what the practice currently did regarding adolescent tobacco use, with providers receiving no training and no materials to provide patients. The intervention consisted of brief counseling by the pediatric providers followed by one in-person visit and four telephone calls by older peer counselors aged 21–25 years, both based on the 5A’s model recommended by the U.S. PHS Clinical Practice Guideline.¹⁰ At the time this trial was conducted, there was little evidence regarding the efficacy of brief clinical interventions with adolescents, with existing guidelines simply recommending that clinicians deliver strong messages encouraging abstinence. And although peer counseling had been successfully used to modify the risks of HIV infection¹³ and teen pregnancy^14,15 in adolescents, this approach had not yet been tested with tobacco.

The purpose of this paper is to describe the Air It Out study components along the pragmatic–explanatory continuum of randomized trials and share guiding principles and lessons learned in developing and implementing the primary care–based intervention trial. The goal is to provide a framework to guide decisions regarding study design and implementation for other researchers, thereby increasing the number of well-designed studies that can be included in the evidence reviews to guide future USPSTF recommendation statements.

Placing the Air It Out Study Components Along the “Pragmatic–Explanatory” Continuum

Randomized clinical trials can be categorized along the pragmatic–explanatory continuum to refer to their purpose and structure, terms coined by Schwartz and Lellouch¹⁶ in the 1960s. Briefly, pragmatic randomized trials are designed to determine if an intervention works under the usual conditions in which it will be used; they are intended to inform decisions by clinicians or policymakers. Explanatory randomized trials are designed to determine if an intervention works under ideal conditions, with a goal to maximize any positive effects the intervention may have. Thorpe and colleagues¹⁷ developed the pragmatic–explanatory continuum indicator summary (PRECIS) as a tool to assist researchers in aligning their study design with the trial’s purpose, specifying ten domains on which pragmatic and explanatory differ:

1. participant eligibility criteria;

2. experimental intervention flexibility;

3. experimental intervention practitioner expertise;

4. comparison intervention;

5. comparison intervention practitioner expertise;

6. follow-up intensity;

7. primary trial outcome;

8. participant compliance with “prescribed” intervention;

9. practitioner adherence to study protocol; and

10. analysis of the primary outcome.

Of note is that most trials fall along the continuum and are neither purely pragmatic nor explanatory. Assessing studies along the PRECIS domains is useful to assist researchers in making design decisions that are consistent with their trial’s primary goal. By placing their trials along the pragmatic–explanatory continuum, researchers can determine the extent to which their trial is appropriately designed to meet its intended purpose, and use this information to identify potential inconsistencies and adjust the study design to better align with the study’s goals.¹⁷

Our trial was designed as a pragmatic trial to inform clinical practice. The purpose of the study was to determine whether a pediatric primary care practice–based smoking prevention and cessation intervention would be effective in increasing abstinence rates among adolescent smokers and nonsmokers under usual clinic conditions. Table 1 describes our Air It Out trial according to the ten PRECIS domains; please refer to the original outcome paper for additional detail on the trial interventions, methods, and results.¹²

Table 1.

Assessment of the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) Domains for the Air It Out Trial

PRECIS Domaina	Assessment of domain for Air It Out trial
Participants
Participant eligibility criteria (range from all comers to stepwise selection criteria)	The trial enrolled a consecutive sample of patients aged 13 to 17 years scheduled for an office visit at 8 pediatric primary care clinics in central Massachusetts regardless of their smoking status. Extremely pragmatic: The trial included all comers in the specified age range. Because it was conducted in only 8 clinics in one region of the country, it is not at the farthest edge.
Interventions and expertise
Experimental intervention flexibility (range from highly flexible instructions and practitioner leeway on how to apply the intervention to strict instructions for every element of intervention to be delivered)	Provider-delivered intervention An algorithm outlining the 5As steps was provided to guide delivery of the brief counseling protocol (refer to the original article for the algorithm),12 but the specific delivery was left up to the individual clinician. The intervention incorporated a patient-centered approach in which the provider asked about smoking status, advised cessation for current smokers and continued abstinence for adolescents not smoking, and referred the patient to the peer counselor to develop a personalized strategy for either cessation or maintained abstinence. Very pragmatic: Although providers were given an algorithm of steps to take, the delivery used a patient-centered approach that left the specific delivery of the intervention up to the clinician. Peer counselor intervention The peer counseling protocol combined the 5A’s model with motivational interviewing and behavior change counseling. Study participants met with the peer counselor for an initial 15 – 30 minute face-to-face session immediately after the provider, followed by 10-minute telephone calls after 2, 6, 12, and 21 weeks. Counseling was tailored to the adolescent’s smoking status and adapted to adolescents’ unique triggers, strategies and barriers to quitting (refer to the original article for details on the topics covered12). Explanatory/pragmatic nexus: Peer counselors were provided greater guidance on topics to be covered and were provided feedback on their fidelity to the intervention (see “Compliance/Adherence” below), making this component of the intervention more explanatory. However, the use of motivational interviewing and flexibility in sharing personal experiences is more pragmatic.
Experimental intervention practitioner expertise (range from full range of practitioners to only seasoned practitioners with prior documentation of applying the intervention with high success rates with close monitoring so that “dose” can be optimized)	Provider-delivered intervention All pediatric providers delivering primary care in the clinics were involved, with no restrictions. Providers were trained in a 1-hour group session and met individually for 15 minutes with a study staff member 2 to 4 weeks later to practice the interventions and receive feedback on fidelity to the algorithm. Very Pragmatic: All primary care providers were involved with minimal training. Peer counselor intervention Peer counselors were female college students aged 21 to 25 years selected to have had smoked as adolescents and successfully quit without pharmacologic aids and with difficulty, in order to provide a coping model of smoking cessation for adolescent smokers. Peer counselors were trained over 5 days in the study protocol and motivational interviewing counseling skills and were required to demonstrate competency. Very explanatory: Peer counselors were required to be between the ages of 21 and 25, have specific experience as a former smoker, and to demonstrate competency in the protocol prior to being assigned a clinic.
Comparison intervention (range from “usual practice” to restricted comparison condition)	The intervention was compared to similar clinics providing usual care. Providers in the usual care clinics received no training and no materials to provide patients. Very pragmatic: The comparison intervention was usual practice.
Comparison intervention practitioner expertise (range from full range of practitioners to standardized expertise)	All pediatric providers delivering primary care in the comparison clinics were involved, with no restrictions and no training. Very pragmatic: All primary care providers were involved.
Follow-up and outcomes
Follow-up intensity (range from no formal follow-up visits with participants to more frequent visits and extensive data collection than routine practice)	There were only two scheduled follow-up assessments, both of which were conducted via mailed survey at 6- and 12- months. No clinic visits were scheduled. Pragmatic: Minimal follow-up without direct contact with study personnel, therefore it did not require more patient contact than usual care, consistent with a pragmatic approach. The fact that there was follow-up keeps this from reaching the level of very or extremely pragmatic.
Primary trial outcome (objectively measured, clinically meaningful to participants assessed without special tests, training or central adjudication to outcome known to be a direct consequence of the intervention and requiring specialized training to determine outcome or central adjudication)	The primary outcome was abstinence of smoking in the past 30 days by patient self-report. Pragmatic: Although the primary trial outcome is the outcome on which the experimental intervention was expected to have direct effect (explanatory), it is a patient-important outcome given these were adolescents interested in quitting, making it more pragmatic. Outcome status did not require central outcome adjudication, relying on patient self-report (pragmatic). Also, the longer-term follow-up makes this more pragmatic.
Compliance/adherence
Participant compliance with “prescribed” intervention (range from unobtrusive or no measurement of compliance and no strategies to maintain or improve compliance to close monitoring, pre-requisite for study entry, and strategies to maintain or regain high compliance)	This was an intent-to-treat trial with the expectation that non- compliance with the intervention by patients is a reality in routine medical practice. Compliance to participation in the peer counseling session and calls was measured indirectly by peer counselor records only and purely for descriptive purposes. Very pragmatic: Patient compliance data was not fed back to providers or participants during follow-up. Minimal compliance- improving strategies (phone call reminders) were applied to participants who did not complete follow-up counseling calls with peer-counselors.
Practitioner adherence to study protocol (range from unobtrusive or no measurement of compliance and no strategies to maintain or improve compliance to close monitoring of clinician adherence to even the minute trial protocol details and manual of procedures)	Pediatric providers Adherence of providers to protocol was measured indirectly by adolescents completing a patient exit interview within 48 hours of their visit either in person or by telephone purely for descriptive purposes. No feedback was provided to the clinicians. Extremely pragmatic: adherence measured indirectly only for descriptive purposes. Peer counselors Adherence to protocol was measured directly via review of audiotaped interviews and feedback was provided to the peer counselors (explanatory) to improve their performance. Patient exit interview data was collected for research purposes only. Very explanatory: adherence data measured and fed back to the peer counselors.
Analysis
Analysis of the primary outcome (range from including all patients regardless of compliance (intention-to-treat) to restriction to “compliers” or other subgroups to estimate maximum achievable effect to answer narrowest, “mechanistic” question)	An intention-to-treat analysis was conducted with no restrictions, including all participants regardless of dose of intervention received and patient or provider compliance with intervention protocols. Very pragmatic: All randomized patients were included in the primary analysis. No patients were excluded post randomization.

^aRanges are given from highly pragmatic to highly explanatory.¹⁷

Guiding Principles and Choice Points: Perspectives and Lessons Learned in Developing and Implementing a Pragmatic Pediatric Primary Care–Based Intervention Trial

As noted earlier, the purpose of the Air It Out study was to determine whether a pediatric primary care practice–based smoking prevention and cessation intervention would be effective in increasing abstinence rates among adolescent smokers and nonsmokers under usual clinic conditions. As such, the study was designed as a pragmatic trial to inform clinical practice. A number of choice points were made and guiding principles established that may be helpful to others in designing studies.

#1. Select the Setting Where Your Intervention Is Intended to Be Delivered in Practice

The pediatric primary care clinic setting was chosen because data at the time we were designing the trial showed that the majority of adolescents (63% to 85%) were seen for preventive care each year.^18–20 Also, the American Academy of Pediatrics noted pediatricians were well positioned to take an active role in addressing smoking,^3,21,22 having many opportunities to intervene with nonsmokers to prevent initiation and with smokers to assist in their efforts to quit.^23,24

#2. Engage the Communities of Interest

In trials involving community practice, practice and provider recruitment and retention may be as challenging as patient recruitment and retention. One of the most important guiding principles was to involve both pediatric providers and adolescents in the research process, engaging them in contributing their expertise and perspectives to the research questions, outcomes of interest, and intervention design and implementation.

In the case of our trial and provider/practice engagement, we began by engaging a pediatric subspecialist colleague passionate about smoking prevention and cessation and interested in being involved in research as a co-investigator in the study. His involvement was critical to selecting the types of questions most relevant to pediatric providers and engaging providers and practices in the trial. Each of the pediatric practices recruited as study sites were located in the referral area of our medical center and identified our pediatric subspecialist clinician and his clinical division as a resource for referring their patients for subspecialty care. This prompted the logical approach of having our pediatric subspecialist make the first and early follow-up contacts with the physician leaders and supervisors within each practice location. This process ultimately achieved recruitment of sufficient study sites by engaging the pediatric practitioners as partners in the intervention study.

We then worked in collaboration with the front line primary care pediatric providers to better understand their concerns regarding assisting their adolescent patients in either remaining smoke free or stopping smoking. What research questions were of most relevance to their clinical practice? What would be feasible for them to do in the brief time they have with patients, and what additional supports would they need within the busy primary care setting? It became clear that it was not realistic to expect pediatric providers to deliver intensive preventive and cessation treatment. Rather, pediatric clinicians thought they could assess smoking status, provide advice, and encourage adolescents to be receptive to receiving more intensive intervention. Based on promising findings of peer counseling being used successfully in reducing the risks of teen pregnancy and HIV infection in adolescents, we explored with clinicians the possibility of incorporating peer counselors into the clinical practice to deliver the more intensive intervention, which was very well received.

The next step was focused on designing the intervention to engage adolescent patients. We drafted the brief provider-delivered counseling intervention and a more intensive peer counseling–delivered intervention based on the 5A’s model recommended by the U.S. PHS Clinical Practice Guideline²⁵ and the American Academy of Pediatrics,²² adapted for use with adolescents in the language used. We conducted focus groups with adolescent nonsmokers and smokers to carefully review the draft protocols and adapted the interventions based on their feedback. One of our primary guiding principles was that the interventions and materials we developed reflect the social, cognitive, and emotional development and needs of our target population; therefore we valued their guidance in refining and finalizing our program.

#3. Plan Carefully for Study Participant Recruitment and Retention

Recruitment of adolescents into smoking prevention and cessation trials is a challenge. In an attempt to investigate how to reduce tobacco use by children and youth in the U.S., in 1997 and 1998 NIH funded several dozen adolescent smoking-cessation studies through Requests for Applications and investigator-initiated research.²⁶ Soon, NIH program officials noted numerous anecdotal reports from investigators citing difficulty in both recruiting and retaining youths into their respective studies. Despite the need for research on adolescent tobacco cessation, a major challenge in conducting such research was recruitment and retention of adolescent smokers into studies. Without adequate numbers of youth enrolling and remaining in the studies, the external validity of the research could be challenged. Unfortunately, there is a tremendous gap in our knowledge of recruitment and retention methods and rates in adolescent tobacco cessation studies, as these are typically only briefly summarized in published research papers, with little detail regarding methods, issues, and how issues were resolved. Furthermore, information investigators have gained from hard experience is seldom available in the published literature, and therefore not easily accessible for other investigators to guide the design and conduct of their adolescent cessation trials. Indeed, the authors of an analysis of 55 published adolescent smoking-cessation studies to determine what recruitment methods or other factors were associated with high recruitment and retention²⁷ noted analyses were limited by missing data and information regarding recruitment and retention methods.

In our trial, both onsite recruitment and proactive outreach strategies were used during well and acute visits at the pediatric practices. Onsite recruitment involved a notice posted at the clinic registration inviting adolescents to see a research assistant, and a research assistant who approached adolescents to describe the study and invite participation. Proactive outreach strategies involved sending a letter describing the study on the physician letterhead 1 week prior to the adolescent’s scheduled visit, with the research assistant calling the adolescent prior to the appointment to invite study participation, and meeting with interested adolescents at the practice site. A total of 2,711 adolescents were enrolled in the study, approximately 90% from well visits and 10% from acute care visits. Of the 2,711 enrolled adolescents, 273 (10.1%) were smokers. The two recruitment strategies yielded comparable average acceptance rates (78% for the two sites using onsite recruitment, 76% for the five sites using proactive methods, and 69% for the one site using both). Recruitment methods were tailored to the site, so it is not possible to compare the relative effectiveness of onsite versus proactive recruitment strategies.

Other strategies used to maximize recruitment of adolescents into a practice-based clinical trial include:

1. Recruit during well visits, when patients see their own clinician and providers have more time to intervene around health behavior changes.

2. Determine and evaluate recruitment strategies for low-SES adolescents, as compared with higher-SES students. Low-SES adolescents smoke more and go to physicians less often, and clinics serving low-SES populations may have logistical challenges that provide additional recruitment barriers.

3. Avoid project names with “stop smoking” focus, which is perceived negatively by adolescents per our focus group qualitative research findings.

4. Simplify and streamline the consent process by engaging adolescents early in decision making regarding study participation, and emphasize ease of required study tasks.

5. Reduce parental barriers to providing consent by considering their barriers (e.g., work schedules) and designing consenting procedures to lower their barriers (e.g., telephone consent).

6. Offer incentives for study participation.

7. Carefully establish strong, high-quality working relationships with office staff to facilitate successful practice-based recruitment, promotion of the study by office staff, and overall better access to office personnel and procedures.

8. Recruit onsite by research personnel, strategically scheduling research staff’s time in the office to maximize recruitment.

9. Build in continuing contact with practices, including frequent visits, reminders, and small gift incentives, all serving to remind office staff and clinicians of study details and progress and maintain an ongoing, collaborative relationship.

10. Provide clinics with feedback on their progress toward reaching recruitment goals via monthly newsletters or updates at regular staff meetings.

In terms of retention, of the 262 smokers who completed baseline assessments, 260 (99.2%) and 256 (99.7%) completed assessments at 6 and 12 months, respectively. Of the 2,449 nonsmokers completing baseline assessments, 2,449 (99.6%) and 2,434 (99.4%) completed assessments at 6 and 12 months, respectively.

Effective retention strategies include:

1. Collect extensive contact information at study entry to enhance follow-up and retention, including information on head of household and three additional alternative contacts who can reach the adolescent.

2. Use a multistep follow-up procedure for data collection; for example, mail surveys in a brightly colored envelope with a personalized letter from the research staff, resend with a second personalized letter if not received initially, and then complete the survey by phone if still no response.

3. Personalize the adolescent’s connection with the study. Providing positive reinforcement for participation in study assessments is a promising strategy with adolescents and increases cooperation with subsequent study contacts (e.g., in our focus groups, adolescents reported that altruism was a motivator for study participation, hence the use of personalized “Thank You” notes).

4. Provide a financial incentive, with a higher incentive for more distal assessments. The most common reason adolescents reported for staying in the study over the 2-year follow-up period was the financial incentive.

Conclusions

In summary, it is important to determine up front the main purpose of your primary care–based behavioral counseling intervention trial. Is it to determine if an intervention works under the usual conditions in which it will be used and hence largely a pragmatic trial, or is it to determine if an intervention works under ideal conditions, in which case it will be largely an explanatory trial? The purpose of our Air It Out trial was to determine whether a pediatric primary care practice–based smoking prevention and cessation intervention would be effective under usual clinic conditions and to inform clinical practice; hence, it fell more on the pragmatic end of the pragmatic–explanatory continuum. This assessment will provide a solid framework to guide decisions regarding participant eligibility criteria, intervention and comparison condition design, follow-up and outcomes, compliance and adherence assessments, and analysis. In addition, consider the setting in which you conduct your study, engage providers and your target population in designing the trial and importantly the interventions to be tested, and carefully plan out your recruitment and retention procedures, keeping in mind that you will need to be flexible in responding to realities on the ground as you begin your trial. With careful planning and monitoring, and a strong research and collaborative team, you will be able to design studies that can be included in the evidence reviews to guide the USPSTF’s future recommendation statements.