Teaching tobacco dependence treatment and counseling skills during medical school: rationale and design of the Medical Students helping patients Quit tobacco (MSQuit) group randomized controlled trial

Abstract

INTRODUCTION

Physician-delivered tobacco treatment using the 5As is clinically recommended, yet its use has been limited. Lack of adequate training and confidence to provide tobacco treatment are cited as leading reasons for limited 5A use. Tobacco dependence treatment training while in medical school is recommended, but is minimally provided. The MSQuit trial (Medical Students helping patients Quit tobacco) aims to determine if a multi-modal and theoretically-guided tobacco educational intervention will improve tobacco dependence treatment skills (i.e. 5As) among medical students.

METHODS/DESIGN

10 U.S. medical schools were pair-matched and randomized in a group-randomized controlled trial to evaluate whether a multi-modal educational (MME) intervention compared to traditional education (TE) will improve observed tobacco treatment skills. MME is primarily composed of TE approaches (i.e. didactics) plus a 1^st year web-based course and preceptor-facilitated training during a 3^rd year clerkship rotation. The primary outcome measure is an objective score on an Objective Structured Clinical Examination (OSCE) tobacco-counseling smoking case among 3^rd year medical students from schools who implemented the MME or TE.

DISCUSSION

MSQuit is the first randomized to evaluate whether a tobacco treatment educational intervention implemented during medical school will improve medical students’ tobacco treatment skills. We hypothesize that the MME intervention will better prepare students in tobacco dependence treatment as measured by the OSCE. If a comprehensive tobacco treatment educational learning approach is effective, while also feasible and acceptable to implement, then medical schools may substantially influence skill development and use of the 5As among future physicians.

1. INTRODUCTION

Tobacco smoking continues to be the leading preventable cause of death in the United States making it responsible for more than 400,000 deaths annually.¹ Smoking also engenders a substantial societal financial burden in direct medical expenses and lost productivity.¹ Fortunately, evidence-based cessation treatments are both clinically effective and cost-effective, making tobacco treatment often referred to as the “gold standard” in preventive medicine and health care cost-effectiveness.² Despite this, still nearly 20% of adults in the United States smoke, and the decline in national smoking rates has slowed in recent years.^3,4 Physicians play a vital role in helping patients quit. In fact, physician-delivered interventions such as physician advice and counseling, is strongly supported by a solid body of evidence^2,5 and has been given an A-level recommendation by the United States Preventive Services Task Force (USPSTF), designating it as having “substantial impact” on adult smokers.^6,7

Physician-delivered interventions for tobacco dependence include the following five core components also known as the 5As: 1) Ask about tobacco use at every visit, 2) Advise the patient to quit in a strong and clear personalized manner, 3) Assess the patient’s readiness to make a quit attempt, 4) Assist in increasing readiness or support a quit by providing practical counseling and recommending pharmacotherapy as appropriate, and 5) Arrange a follow-up visit or refer to more intensive treatment.² Broad implementation of the 5As, however, remains limited^8–13 as is the level of tobacco treatment training physicians receive.^14–18 This is unfortunate because physicians, including physicians-in-training (e.g. undergraduate medical students and residents), who are trained in tobacco dependence treatment, are more likely to use these skills than those not trained and therefore have had an impact on patients’ smoking status.¹⁹ In response to the relatively low rate of training and use of the 5As, a subcommittee of the Interagency Committee on Smoking and Health recommended that all clinicians should have the knowledge, skills, and support systems to help patients stop smoking and that competency in tobacco dependence treatment become “a core graduation requirement for all new physicians and other healthcare professionals”.^20–21

Ideally, this “core graduation requirement” would translate into an effective and standardized competency-based teaching method implemented repeatedly and throughout medical school. Educational theories support that optimal learning occurs when knowledge is exchanged and reinforced through multiple practical experiences and when there is interaction between interpersonal, intrapersonal, and organizational factors. This social-ecological framework for learning is thought to facilitate skill acquisition more so than a uni-dimensional or traditional method of instruction (e.g. didactics only).^22–28

Unfortunately, much of current medical school curricula is not typically developed in this manner when training in tobacco use and its treatment.^15,29,30 Medical school curricula, in fact, only devotes on average about four hours out of the four years of medical school training to tobacco dependence treatment.^15,29,30 The optimal needed combination of didactics, interactive counseling practice through role-plays or standardized patient (SP), and actual and observed clinical experiences with smokers is limited.³¹ Medical school tobacco curricula vary among schools and from year to year within any school, limited by an already packed curriculum.^29,30,32 Thus, no two medical school curricula are exactly alike and despite best intentions, medical students do not receive adequate or consistent training in tobacco dependence treatment. The goal of the current study, therefore, was to develop and implement a comprehensive medical school curriculum for teaching the effective delivery of tobacco treatment counseling and to evaluate its impact on acquired student tobacco treatment and counseling skills within the context of a group randomized controlled trial.

2. METHODS

2.1 Study Aims

The MSQUIT study’s (Medical Students helping patients Quit Smoking) primary aim is to refine, implement, and evaluate whether a multi-modal educational (MME) approach for tobacco dependence treatment is more effective than a traditional educational (TE) approach (TE) for fostering 5A skill acquisition. Our MME approach includes: 1) a web-based course during the first-year; 2) a classroom tobacco counseling role-play exercise implemented in tandem with the web course; 3) preceptor training in use of 5A counseling and in observing and providing instruction and critical feedback to students during a designated third-year clerkship rotation (i.e. Family Medicine or Internal Medicine); and 4) a classroom “booster” session also held during that third-year clerkship rotation. These components are designed to address the interpersonal (e.g. 5A self-reported skill, tobacco treatment knowledge), intrapersonal (e.g. experiences observing 5As, experiences receiving 5A instruction), and organizational factors (e.g. clinic/system prompts and reminders) associated with optimal learning. This combination, primarily using our web-based course/role play and preceptor facilitated teaching methods, is hypothesized to provide an efficient and structured vehicle for building tobacco dependence treatment skills among medical school students (see Figure 1).

Figure 1.

MSQUIT’s secondary aims address the potential direct impact of the interpersonal, intrapersonal, and organizational factors on tobacco counseling skills. We hypothesize that these factors mediate the relationship between the MME intervention approach and our primary outcome, observed 5A skill, and that the MME approach will outperform the TE approach in each of these areas. Finally, we explore the feasibility and acceptability of implementing the MME across medical schools. If the MME approach influences students’ tobacco dependence treatment skill, while also acceptable to students and school administration, then it may support national recommendations for training future physicians in tobacco dependence treatment.

2.2 Study Design

MSQUIT is a pair-matched, group-randomized controlled trial implemented in 10 U.S. medical schools. Two methods of teaching tobacco dependence treatment are compared: 1) traditional medical education (TE), and 2) multi-modal education (MME) that adds the aforementioned curriculum components to TE. The primary outcome is observed 5A tobacco dependence treatment counseling skills, as measured by the Objective Structured Clinical Examination (OSCE), the standard method for observing and evaluating medical student skills at all U.S. medical schools. Among 5 MME and 5 TE schools, we will compare school OSCE scores for 3^rd year students from the Class of 2012 who did not participate in the intervention (i.e. school baseline assessment) with the 3^rd year OSCE scores from the Class of 2014 who will have participated in the intervention (i.e. school follow-up assessment). We chose this nested cross-sectional design because we could not obtain pre and post-intervention OSCE school data for the same student cohort (i.e. as 1^st years and then again as 3^rd years) since OSCEs are traditionally not given to 1^st year medical students and these students do not yet see patients. The cohort of 3^rd year students who did not participate in the intervention are called the “comparison cohort”.

Our secondary outcome is self-reported 5A tobacco dependence treatment skills. To determine the impact of the intervention on our secondary outcome, self-reported 5A skill, students from the 10 schools (Class of 2014) will complete questionnaires (i.e. “Baseline Survey”) as 1st year students and will complete them again as 3^rd year students (“Follow-up Survey”). This baseline survey and another survey, the “Randomization Survey” (see Randomization section), administered to another separate cohort of 3^rd year students (Class of 2011), will assess for each of our hypothesized mediators.

To evaluate the feasibility and acceptability of implementing the MME intervention, annual qualitative key informant interviews with school study personnel (e.g. school-site PI, clerkship directors, and research coordinators) are conducted. Interviews focus on the barriers and facilitators of implementing the intervention for the prior year, and documenting how the intervention will be implemented for the following year. We also record the number of students who complete each MME component, the percentage of the class cohort that complete the entire MME, and satisfaction scores from each intervention component. Key informant interviews also are conducted with TE schools, primarily to record their TE curricula content and changes, if any (see Figure 2).

Figure 2.

2.3 Participating Medical Schools

The following 10 medical schools are participating in our study: Creighton University School of Medicine, Georgetown University School of Medicine, Louisiana State University Health Science Center-Shreveport, Stanford University School of Medicine, University of Alabama at Birmingham, University of Iowa Carver College of Medicine, University of Kentucky College of Medicine, University of Louisville School of Medicine, University of Minnesota Medical School, and the University of Pennsylvania School of Medicine. Schools are from each U.S. region (e.g. North, South, West, Midwest) and met the following criteria: 1) an enrollment of 90 or more matriculating first-year (MS1) students; 2) fewer than four hours of tobacco treatment training over the four years of medical school; 3) a third-year Family Medicine or Internal Medicine clerkship and 4) a standard evaluation of third-year students (MS3s) through an OSCE examination with a new videotaped tobacco case. The study was reviewed by Internal Review Boards at each participating medical school.

2.4 Randomization

Before randomizing schools to either the MME or TE intervention, schools were pair-matched based on adjacent ranking in overall school self-reported skill level in conducting the 5As. School 5A skill level was obtained by having 3^rd year students from the Class of 2011 complete a “Randomization Survey” that assessed self-reported skill level for each 5A. Students reported how much they agreed (1= Strongly Disagree to 5=Strongly Agree) that they were “skilled” in completing each 5A. The average score from 4 of the 5As (we did not include scores on “Ask” because there was little school variability) was determined for each school. We decided to pair-match schools because if tobacco dependence treatment curriculum differed by school, then student tobacco treatment skills may also differ.

Although we planned to obtain all OSCE data (used to measure our primary outcome) at the end of the 3^rd year (i.e. MS3 year) post-intervention in June or July of the school year, we learned, that this was not possible since schools typically administer the OSCE either immediately upon completing the designated 3^rd year clerkship rotation (i.e. last component in the MME intervention), or at the very end of the 3^rd year in medical school. Therefore, students from one set of schools would complete their OSCE immediately after completing our intervention, while students from the other set of schools would complete their OSCE anywhere from one day to several months post-intervention. Given a potential difference in OSCE performance due to elapsed time since intervention, we stratified schools on OSCE timing to distinguish these two sets of schools. Within strata, we then ranked schools in overall school 5A skill level and then pair-matched schools adjacent in rank. Schools with the two highest scores formed the first pair, the next two highest scores formed the next pair, and so on. Within pairs, one school was randomly assigned to the MME and the other to the TE.

2.5 Recruitment and Participation of Medical Students

Following randomization, all 1^st year students (i.e. MS1s) at the ten schools were informed that their medical school was participating in a study to test the effectiveness of different teaching methods for medical students’ skills in tobacco dependence treatment and counseling. They were told that the study will include a survey for their class to complete during their MS1 and MS3 medical school years and passive consent was given by completing the surveys. MME intervention component participation among medical students was encouraged in an appropriate method for that medical school (e.g. faculty strongly encouraged students). Student permission to use their OSCE performance for research also was obtained through appropriate and various methods across medical schools. Altogether, from the 10 medical schools combined, 1381 medical students (i.e. Class of 2014) enrolled and are currently participating in the study.

2.6 Medical School Education Interventions

2.6.1 Traditional Education (TE) Intervention

The tobacco TE intervention represents “usual care” and includes the current content and mode for tobacco teaching among schools randomized to the TE. TE content typically includes knowledge in the basic science of tobacco use (e.g. health consequences of tobacco use and passive smoking), and knowledge in the clinical science of tobacco treatment (e.g. 5As implementation, behavioral counseling and relapse prevention techniques, pharmacotherapy). Modes of learning typically include lectures, small group discussions or communication skill-building exercises related to tobacco treatment as part of a patient interviewing, doctoring, or behavioral course, and observation of preceptor-delivered tobacco counseling in the clinical setting, if any, and clinical experiences in treating smokers. Per our inclusion criteria, no more than four hours was typically devoted to the tobacco curriculum at any participating medical school. As mentioned, changes in the TE tobacco curriculum from year-to-year were not discouraged, but are documented in our annual key informant interviews.

2.6.2 Refining the Multi-Modal Education (MME) Intervention

School site PIs provided feedback on a) the proposed MME intervention content, and b) best strategies to implement each component at their school. Three to five medical students representing each school year (MS1 through MS4) participated in a 60-minute focus group conducted over the telephone to also obtain their reactions to the MME content (web-based course modules and preceptor training program) and structure (“Should components be required or not required?” and “What additional curriculum components should be added, if anything?”). Ultimately, the refined MME curriculum consisted of the following: 1) a web-based “Building Tobacco Treatment Skills” program for MS1s; 2) a classroom structured role-play exercise and demonstration for MS1s; 3) a systematic method for training clerkship preceptors and then strongly encouraging preceptors to model, observe, and teach the 5As to MS3s during a designated clerkship; and 4) a structured interactive “booster” session held concurrent with the clerkship experience. Below are specific details for each MME component.

2.6.2a Web-based “Building Tobacco Treatment Skills” course

Our web-based course entitled, “Basic Skills for Working with Smokers” was adapted for the medical student and has previously been found to be effective in teaching core tobacco treatment content areas to medical students.³³ Its goal is to provide standardized information in the following core tobacco basic and clinical science content areas: epidemiology of tobacco use, nicotine dependence and withdrawal assessment, and provision of behavioral and pharmacotherapy tobacco treatment. To facilitate course completion, the course was designed to be four hours in length, student self-paced, and medical education deans and course directors agreed to include the course in the first-year curricula, and to monitor course completion among their students. Upon completion all MS1 students are eligible to receive a certificate of completion. Students will continue to have access to the course through their 3^rd year.

2.6.2b Role-Play Classroom Demonstration

A role-play session was incorporated in tandem with the web-based program. The goal of the role-play session is to provide students with guided, hands-on practice in addressing tobacco treatment with patients. The one-hour session begins with a video demonstration of a non-judgmental, open-ended, and patient-centered counseling approach, which incorporates the 5A intervention presented in the web-based course. Afterwards, faculty instructors led a brief discussion of physician-delivered tobacco treatment challenges. This is followed by at least 30 minutes of a role-play exercise of three cases representing various tobacco treatment scenarios (e.g. patient not motivated to quit, patient motivated to quit). Students role-play either as physician, patient, or coach for each tobacco case using a one-page checklist of the 5A counseling approach. This role-play component was designed to be semi-scripted and guided by designated faculty instructors. These faculty were all either PhD or MD level with familiarity with behavioral change counseling, although tobacco control may have not been their expertise. All also had tremendous experience in facilitating group/classroom interaction.

2.6.2c Preceptor Training and Preceptor-Facilitated Clerkships

The preceptor’s role is to teach, model, provide practice opportunities, and provide constructive feedback to students during the clerkship experience. Preceptors who provide these learning opportunities have a positive impact on students’ clinical skills.^34,35 Preceptors, however, may not be adequately trained in the 5A intervention, nor use it frequently^36–38. Our preceptor-facilitated teaching model, therefore, uses a systematic approach to train clerkship preceptors in a) the use of the 5As for their patients, including incorporating system/clinic changes, b) the use of teaching strategies to motivate students to use the 5As, and c) the provision of modeling, observing, instructing, and providing feedback in tobacco treatment to students as they interact with patients. Preceptors were trained using an academic detailing process, whereby “academic detailers” or ADs provided standardized 30 to 45 minute group training to the preceptors. Academic detailing, also known as educational outreach, has demonstrated effectiveness for training physicians in various intervention skills.^36,39–44 ADs from each MME school attended a 6-hour interactive train-the-trainer session (led by the research team) focused on “how to train the clerkship preceptor” and received ongoing supervision from research staff. MME schools chose a Family Medicine, Internal Medicine, or other similar clerkship as the site for preceptor-facilitated 5A teaching ensuring that every MME school preceptor who would have at least one student to precept over the next 12 months participated in the AD training. Preceptors are strongly encouraged to use the 5As, actively observe and instruct students with smoker patients, and to give critical feedback related to their 5A use. Likewise, students are encouraged to observe, seek guidance, and ask for feedback regarding their tobacco treatment skills from their preceptor. During the clerkship, all MS3 students and preceptors are also provided with study-tailored handouts, brochures, and other reminders of the 5A intervention.

2.6.2d Clerkship “Booster” Session

Along with the preceptor-facilitated clerkship rotation, students participated in a small-group “booster” session. Our team scripted and developed a five-minute video with a team of professional videographers and actors. The video reinforced the following concepts: 1) the use of the 5As, specifically the use of “Arrange” and 2) the use of patient-centered communication strategies, such as asking open-ended questions. It followed a medical student’s interactions with a smoking patient and his clerkship preceptor. The 30-minute “booster” session was semi-scripted for clerkship faculty to facilitate a small-group discussion after viewing the video.

2.7 Intervention Implementation and Fidelity

To guide the implementation of the MME intervention condition, site PIs are encouraged to tailor implementation of the intervention to their institution’s organizational and educational context and unique characteristics and needs. This means that while the curriculum content was standardized across schools, each school has some flexibility in choosing the specific course the educational intervention was to be implemented within, as well as when it was to be implemented (e.g. during the Doctoring and Clinical Skills course). Schools, however, are provided with guidelines for implementation timing so that students across schools would receive the intervention within a common and specific timeframe. These guidelines are as follows: 1) the web-based course was to be implemented when our primary student cohort were 1^st year students; 2) the role-play was to be implemented also during the 1^st year and preferably after the web-based course; 3) the preceptor learning experience was to take place either during the internal or family medicine clerkship during the student’s 3^rd year; and 4) the booster session was to take place simultaneously during the clerkship rotation. Unfortunately, despite the opportunity to offer all elements of the MME curriculum, not all schools could “require” students to complete intervention component or assessment, although it was strongly encouraged.

In order to maximize intervention fidelity, the following steps are conducted. First, we ensured that all Site PIs had a role in the refinement of each intervention component. They or their designee (e.g. whoever implemented the intervention component) was, therefore, familiar with the intervention content, goals, and objectives. Next, our team made sure the content and structure of the intervention components (e.g. role-play exercises, booster session) were standardized and semi-guided with written instructions and examples of discussion points provided for faculty instructors. The web-based course was directly implemented by the UMass research team remotely as it required student login and registration that ensured participation. The preceptor academic detailing session was also scripted for the academic detailer who completed intervention fidelity checklists to ensure they covered all topics. These sessions were audio-taped and accompanied with ongoing supervision by UMass researchers. Finally, conference calls with MME faculty are held both before and after implementing the intervention in order to prepare and debrief with faculty and to ascertain if parts of the intervention were not implemented.

2.8 Measures

2.8.1 Primary Outcome: Observed 5A Tobacco Dependence Treatment and Counseling Skill

Table 1 includes more information about variables measured and when they were measured. The Objective Structured Clinical Examination (OSCE) will be used to objectively measure 5As treatment skills among 3^rd year medical students. We used a tobacco-specific OSCE case, which has been used previously as an assessment tool within a medical school curriculum, and was reviewed and refined with input from each school’s site PI, OSCE course director, and Standardized Patient (SP) trainer. Our research team trained each school’s SP trainer through a series of 2-day trainings. SPs practiced their “role” extensively in order to meet the typical medical school testing standards. For example, SPs must consistently respond similarly to specific questions asked by medical students and they must accurately portray the case history as written. Videotapes of each SP were reviewed by the expert SP trainer and suggestions for improvement to the SP were made, if any.

Table 1.

Measures

Construct	Assessment Instrument	Grant Year
		1	2	3	4	5
Outcome Evaluation
Observed 5A Skill (Primary Outcome)	-OSCE		X		X
Self-reported 5A Skill (Secondary Outcome)	-Student Survey	X	X		X
Other Variables: Mediators
Tobacco Dependence Treatment Knowledge	-Student Survey	X	X		X
Frequency of Instructional Learning		X			X
Frequency of Observational Learning		X			X
Organizational Changes		X			X
Process Evaluation
Feasibility	-KII with Site PI	X	X	X	X	X
Acceptability	-KII with Site PI	X	X	X	X	X
	-Student Satisfaction Evaluation		X	X	X
	-Student Participation Rate		X	X	X
Fidelity	-Conference calls with Site PIs/Faculty		X	X	X
	-Student Participation Rate		X	X	X
	-Academic Detailer Intervention Checklist			X
Reach	-Student Participation Rate		X	X	X
Ongoing Changes and Updates	-KII with Site PI	X	X	X	X	X

OSCE = Objective Student Clinical Exam; KII = Key Informant Interview

Student scores on the OSCE are determined by an OSCE score checklist of 33 behaviors and 4 items of communication skills approved by our research team and an external group of tobacco expert researchers and clinicians. Examples include, “Asked how soon after waking the patient smokes”, or “Informed the patient that the cough was related to the patient’s smoking”, or “Presented information about nicotine replacement therapy to help the patient quit.” All OSCEs are videotaped, and blinded, trained raters score the tapes designating if the behavior was completed or not (Yes/No). Each rater (n=3) is not a part of our research team and is blind to school and its intervention assignment (MME or TE). Raters completed a series of trainings and follow-up booster trainings led by our research team and had several opportunities to practice coding OSCE performances from a pool of practice students. Total scores will be computed as the percentage of completed behaviors in total. Although students are not expected to complete all 33-checklist behaviors, higher overall scores are indicative of better 5A performance. Ultimately, for this nested cross-sectional design, we will compare OSCE scores from our cohort of students who completed the MME or TE intervention (i.e. Class of 2014) to OSCE scores from our comparison cohort of students (i.e. Class of 2012).

2.8.2 Secondary Outcome: Self-Reported Tobacco Treatment Skill

Our secondary outcome was student self-reported 5A tobacco treatment skills. Students are asked for their level of agreement (1=Strongly Disagree to 5=Strongly Agree) on their ability to complete each 5A. Items include, “Asking about smoking at every visit”, “Advising all smokers to quit”, “Assessing patient willingness to quit”, “Assisting the patient with a quit plan”, and “Arranging follow-up contact”. A total score is the average among all 5As. Students in MME and TE interventions completed questionnaires at baseline as MS1s and post intervention as MS3s.

2.8.3 Other Outcomes of Interest: Proposed Mediators

The following interpersonal, intrapersonal, and organizational variables were measured for students in MME and TE interventions through questionnaires at baseline and after intervention completion, and through completion of the “randomization survey” by 3^rd year students (Class of 2011) prior to the intervention. They are hypothesized as study mediators. Interpersonal mediators include self-reported 5A skill (as described above) and tobacco treatment knowledge. Knowledge of tobacco dependence and treatment will be assessed using a series of questions used in our prior research.⁴⁵ These multiple choice or true/false type questions ask about: 1) tobacco use and quitting epidemiology across gender, ethnicity, and age 2) health consequences of smoking and benefits of quitting; 3) nicotine dependence and assessment; 4) clinical practice guidelines regarding pharmacotherapy and behavioral counseling; and 5) awareness of the 5As. Intrapersonal mediators were assessed through the “randomization survey” and include the frequency of instructional and observational 5A training received by students. These 3^rd year students were asked the number of times (0 times, 1–3 times, 4–10 times, >10 times) that they have been instructed to use each of the 5As separately. Students also report how frequently (0 times, 1–3 times, 4–10 times, >10 times) they observed someone, preferably a preceptor, in providing each 5A separately.

Finally, organizational mediators also assessed through the “randomization survey”, include counseling prompts, reminders, or clinic/system changes. Students are asked to report the extent to which preceptors or clerkship sites modified their teaching and practice environment to support the delivery of providing tobacco treatment. Students will report on how much they agree (1= Strongly Disagree to 5=Strongly Agree) that the following strategies were implemented: 1) preceptors set tobacco treatment educational objectives, 2) preceptors made the expectations clear that the review with patients includes tobacco treatment, 3) preceptors provided positive or corrective feedback about how students discussed tobacco use, 4) clinics identified a routine place in the chart to document tobacco counseling, 5) clinic office staff were involved in implementing any of the 5As, and 6) clinics had educational cessation materials displayed or made available to patients.

2.8.4 Process Evaluations: Feasibility and Acceptability of the MME

Feasibility and acceptability data will be gathered through annual key informant interviews (KIIs) with MME school site PIs, and through student survey evaluations. Interview topics will cover the barriers and facilitators to implementing each MME intervention component, problems that arose and remedial actions needed to address them, elements of the intervention that worked well and why, and the “quality” or “fidelity” of the intervention that was actually delivered. Our team will also record whether or not individual students completed each MME component so that we can determine “reach” of the curriculum, defined as the number of students within each school that completed the entire MME intervention. Preceptor attendance at AD sessions will also be collected, as well as the extent to which clinics and preceptors made any teaching of system-level clinic changes. Acceptability data will also include student completion of satisfaction surveys upon completing each MME component. Finally, we assessed for on-going or new curriculum or institutional changes that may impact student training in tobacco dependence also through our annual KII with Site PIs.

2.9 Sample Size and Power Calculation

Results from our prior research examining the effect of an educational module for improving communication skills among undergraduate medical students as evaluated by OSCE scores showed a significant intervention effect.^46,47 Our sample size calculation, therefore, used a moderate effect size of 0.387 standard deviation units. Using our baseline OSCE data, which had a standard deviation of 3.24, we will be able to see a detectable difference of 1.25 OSCE units. Power calculations require two additional estimates: the over-time correlation of school means from the same school and the intraclass correlation (ICC) of observations from different students at the same time and at the same school. Again, using our previous work, we estimated the over-time correlation to be 0.3 and the ICC to be 0.0318.⁴⁷ With these estimates and with at least 90 students at each school at each survey administration, and with 5 schools per condition (10 total), we have 80% power to detect a moderate intervention effect using a two-sided alpha of 0.05.

2.10 Data Analysis Plan

Our primary outcome is 5A tobacco treatment skill as observed through the OSCE evaluation. This nested cross-sectional design will be analyzed in a two-stage mixed model ANCOVA.⁴⁸ This will account for an assumed positive ICC among student observations from the same school. In the first stage, school (10 schools) X time (baseline, follow-up) OSCE means will be calculated, while adjusting for student level covariates. In the second stage, follow-up OSCE school means will be regressed on study intervention and the baseline OSCE school mean, which was collected from an earlier cohort of MS3 students. The result will be an adjusted follow-up OSCE mean for each study condition. The intervention effect will be estimated as the difference between the adjusted means.

Our secondary outcome is self-reported 5A skills. We will use a one-stage mixed model ANCOVA, wherein the follow-up score is regressed on the baseline score (from the same participants as 1st year students), study condition, and any individual-level covariates; school will be included as a nested random effect. The intervention effect is the difference between the adjusted means.

To evaluate if our proposed MME intervention leads to greater 5A skill performance through its impact on our hypothesized mediators (self-reported 5A skills, tobacco treatment knowledge, observational and instructional learning, and clinic counseling prompts and reminders), we will need to establish mediation. We will follow the procedures suggested by MacKinnon⁴⁹ to assess mediation, separately for each dependent variable (i.e. OSCE total score, and self-reported 5A skill).

To analyze our feasibility and acceptability outcomes, we will use traditional qualitative research methodology to analyze our KII interview data. Interviews will be audiotaped and an extended review process will allow our team to obtain comparative thematic findings from informants across MME schools and TE schools. Satisfaction scores and student and preceptor participation rates for each MME component will be summarized using frequency tables and descriptive statistics.

2.11 Ethics and Dissemination

For a few schools, study data was considered exempt from ethics approval due to the nature of the educational research, while others needed ethical approval. Regardless, all students were informed of the study and gave passive consent by participating in surveys and intervention components. Each school site made sure that not participating in any MME intervention component would not substantially influence students’ grades. Site PIs had obtained school approval to incorporate MME intervention components into the existing medical student curriculum well in advance of implementing the component. With regard to OSCE data, some schools required that students sign an informed consent document to release their videotaped performance to the research team, who would score OSCEs, separate from the student’s grade. All surveys, OSCE scores, and MME evaluation component data were de-identified before given to the research team, who will manage databases and analyses. Our findings will be highly relevant to those in medical education and tobacco control fields.

3. DISCUSSION

The clinical and public health impact of physicians helping patients quit smoking when they use evidence-based clinical guidelines, such as the 5As, is noteworthy. Although there is an awareness of the 5A intervention among physicians, consistent and appropriate use of each of the 5As remains fairly limited. For example, a most recent national survey of primary care physicians showed that between 32–54% did not implement “Assist” behaviors with their patients such as referring to a cessation program, discussing medication, or helping to set a quit date, and about 77% had not “Arranged” for follow-up care.⁹ Because both real and perceived deficits in skill to implement tobacco treatment 5As contribute to the low 5A usage rate,^31,50 tobacco dependence treatment training is important and beneficial early on in clinical training. Unfortunately, there is limited tobacco dependence treatment training and training standardization in medical schools and this is an area that should be addressed.

Educational theorists have established that formative learning experiences are crucial such that they “set up” a physician-in-training’s skill level and patterns of future use. Learning is optimal when knowledge and experience is taught early, reinforced consistently, and integrated through all aspects of a curriculum.^22–28 Therefore, early exposure to tobacco dependence treatment competency, through didactics, preceptor modeling, patient observation, instruction with receipt of feedback, and required patient experience can benefit medical students. Additionally, the opportunity to methodically learn complicated behavioral skills such as behavioral counseling may be most beneficial when trainees can safely observe, be observed, and receive immediate critical feedback from preceptors during medical school. A standardized curriculum across medical schools may also impact the culture of a medical institution (e.g. preceptor behaviors), and this would only be beneficial as students leave their undergraduate medical school for another institution in residency.

The MSQUIT study is the first group randomized controlled trial that seeks to establish whether a multi modal educational intervention compared to traditional educational methods will influence medical student tobacco treatment skills. In addition to this important design question, this research has the potential to be influential because the MME intervention itself combines the innovativeness of a known effective web-based didactic course with a systematic method to train preceptors to provide personalized instruction in tobacco education and treatment. The MME intervention is also grounded in learning theory, implemented early, and integrated throughout most of a student’s medical school experience. Finally, because the MME intervention was developed with the goal of possible and eventual national medical school dissemination, the curriculum was refined through opinion from medical students, medical school administrators, education specialists, and tobacco treatment specialists. The MME was also designed to be delivered in a standardized manner, yet allowing for flexibility to tailor the MME components to any school’s specific curriculum. Thus, this educational intervention will likely have a significant clinical impact on the patients of our future physicians.

One potential limitation of the current study is the reality of implementing a multi modal intervention across multiple medical schools with varying school resources, institutional policies, and smoking prevalence context (e.g. smoking rates in Kentucky vary significantly from rates in California). Although we will control for school-level differences in student self-reported 5A skill, differences in the ability to feasibly implement the study across medical schools exist. For example, school-related policies that allow faculty to make curriculum components graded or required versus optional differ among institutions and potentially can affect student participation rates and study outcomes. Schools also may differ in the number of faculty who have interests or expertise in tobacco dependence treatment, indirectly affecting those faculty or preceptors who will affect a subset of students, which may also affect school-level outcomes. Fortunately, our research team will implement a methodical approach to communicate and capture the barriers, facilitators, and eventual action plans for implementing each aspect of the MME intervention, as well as objective data on school-level characteristics, student participation and satisfaction. Therefore, we will be able to understand the contextual factors affecting implementation. Understanding how context affects implementation is a notable strength because it is likely that other medical schools outside of these 10 schools will experience similar barriers. These data, therefore, will be informative for any future potential MME implementation and dissemination.

One other notable strength of this research is its use of objective and observed OSCE scores for its primary outcome, since the majority of research on medical student tobacco dependence evaluation has been from self-reported 5A skill level. This method of evaluation reduces potential student and school-level biases (independent coders code the videotapes in the current study), and develops a unique mechanism to objectively evaluate tobacco dependence treatment skills. Although prior research has shown that medical students positively evaluate their experience with tobacco-specific trained SPs,⁵¹ and that SPs are trained in a plethora of health behavior change cases, including tobacco,⁵² few research, if any, has objectively used a tobacco-specific case and OSCE scores to measure intervention impact on student tobacco dependence treatment skills. If feasible to implement, our OSCE tobacco case, may be included to use for additional training purposes or could be used for required undergraduate medical student standardized evaluation (e.g. Clinical Skills Exam (CSE) of USMLE Step 2).

Overall, a tobacco treatment training intervention or curriculum that can effectively target physicians-in-training, their preceptors, and their broader medical school policies and educational practices, is needed given the important role of formative medical skill training and the clinical and public health impact of the physician on smokers.

Abbreviations

MME

Multi-modal education

TE

Traditional education

OSCE

Objective Structured Clinical Examination

MS1

MS2, MS3, MS4, 1^st – 4^th year medical students

AD

Academic detailer

SP

Standardized patient