Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jan 1.
Published in final edited form as: J Subst Abuse Treat. 2015 Jun 6;60:91–100. doi: 10.1016/j.jsat.2015.05.009

Beyond the Ask and Advise: Implementation of a Computer Tablet Intervention to Enhance Provider Adherence to the 5As for Smoking Cessation

Sara Kalkhoran 1, Nicole A Appelle 1, Anna M Napoles 1, Ricardo F Munoz 2,3, Paula J Lum 4, Nicholas Alvarado 1, Steven E Gregorich 1, Jason M Satterfield 1
PMCID: PMC4670822  NIHMSID: NIHMS697921  PMID: 26150093

Abstract

Background

The 5As for smoking cessation is an evidence-based intervention to aid providers in counseling patients to quit smoking. While most providers “ask” patients about their tobacco use patterns and “advise” them to quit, fewer patients report being “assessed” for their interest in quitting, and even fewer report subsequent “assistance” in a quit attempt and having follow-up “arranged”.

Purpose

This article describes the design of an implementation study testing a computer tablet intervention to improve provider adherence to the 5As for smoking cessation. Findings will contribute to the existing literature on technology acceptance for addressing addictive behaviors, and how digital tools may facilitate the broader implementation of evidence-based behavioral counseling practices without adversely affecting clinical flow or patient care.

Methods

This project develops and tests a computer-facilitated 5As (CF-5As) model that administers the 5As intervention to patients with a computer tablet, then prompts providers to reinforce next steps. During the development phase, 5As content will be programmed onto computer tablets, alpha and beta-testing of the service delivery model will be done, and pre-intervention interview and questionnaire data will be collected from patients, providers, and clinic staff about 5As fidelity and technology adoption. During the program evaluation phase, a randomized controlled trial comparing a group who receives the CF-5As intervention to one that does not will be conducted to assess 5As fidelity. Using the Technology Acceptance Model, a mixed methods study of contextual and human factors influencing both 5As and technology adoption will also be conducted.

Conclusions

Technology is increasingly being used in clinical settings. A technological tool that connects patients, providers, and clinic staff to facilitate the promotion of behavioral interventions such as smoking cessation may provide an innovative platform through which to efficiently and effectively implement evidence-based practices.

Keywords: implementation science, smoking cessation, digital health

1. Introduction

Smoking is the leading cause of preventable morbidity and mortality in the United States (U.S. Department of Health and Human Services, 2014). Approximately 18% of Americans currently smoke, and about 480,000 annual deaths are attributed to smoking (Jamal et al., 2014). The U.S. Public Health Service’s 5As model (Fiore MC, Jaen CR, Baker TB, & et al., May 2008) is considered the gold standard evidence-based practice for conducting smoking cessation counseling. The 5As model consists of asking patients about their tobacco use patterns, advising them to quit, assessing willingness to quit, assisting those who are willing to do so, and arranging follow-up assessments. In primary care, delivery of the 5As is often incomplete (King, Dube, Babb, & McAfee, 2013; Tong, Strouse, Hall, Kovac, & Schroeder, 2010), with commonly cited obstacles including time constraints, lack of expertise, financial incentives, patient privacy, and stigma about smoking (Schroeder, 2005).

Technological tools including telephone quitlines (Stead, Hartmann-Boyce, Perera, & Lancaster, 2013) and Web-based cessation programs (Civljak, Stead, Hartmann-Boyce, Sheikh, & Car, 2013; Munoz et al., 2009) help people quit. Self-help and telephone-based interventions, however, are less effective than in-person clinician interventions, and they don’t connect cessation with ongoing medical care (Fiore MC et al., May 2008). Furthermore, although provider hand-held devices and electronic reminders improve cessation efforts, they often exacerbate clinician time pressures (Marcy et al., 2008). Physician advice to quit is associated with increased odds of tobacco abstinence (Fiore MC et al., May 2008; Stead, Buitrago, et al., 2013), and smokers who ask their physician for assistance with quitting receive smoking cessation services more frequently than those who do not (Quinn et al., 2005); thus, patient factors cannot be overlooked.

Although computer screeners, electronic decision support, and other technological tools are increasingly used for behavioral interventions in primary care (Hunt, Haynes, Hanna, & Smith, 1998; Souza et al., 2011), substantial adoption and implementation barriers to such interventions remain (Sciamanna et al., 2004). An innovative smoking cessation delivery system would include both patient and provider, while still addressing the user and system obstacles of time, counseling expertise, cost, and stigma. This system should capitalize on evidence-based practices like the 5As, be acceptable to diverse patients, tap local cessation resources, and be minimally disruptive to clinic flow.

1.1 The Computer-Facilitated 5As Service Delivery Model

This paper describes the study protocol for a smoking cessation delivery system designed to address potential facilitators and barriers to adoption and implementation of such technology. In an implementation study funded by the National Institute on Drug Abuse, we will evaluate the use of computer tablets to deliver the 5As for smoking cessation in adult primary care clinics. We aim to apply technology to facilitate implementation of evidence-based behavioral counseling practices, without adversely affecting clinical flow or patient care. Thus, we will be evaluating the implementation of the 5As and tablet technology and predictors of adoption of both the 5As and the tablet.

As described by Proctor et al. (Proctor et al., 2009), implementation strategies that influence change span a variety of levels, including systems (e.g. policies), organizations (e.g. health care organizations), groups (e.g. clinics), and individuals (e.g. health care providers). In this study, we specifically focus on addressing individual provider behaviors in implementing the 5As for smoking cessation. On a group level, the implementation of tablet technology into the pre-existing clinic workflow will also be evaluated.

Specifically, we will develop a computer-facilitated 5As (CF-5As) service delivery model and tests its effects on 5As fidelity in primary care, beginning with a patient self-administered computer tablet intervention in the waiting room and ending with a patient-provider exchange to reinforce key messages, provide additional cessation assistance (including pharmacotherapy if appropriate), and arrange follow-up. This takes advantage of the convenience, privacy, and time-savings of computers while using the social influence, (computer-supplemented) clinical expertise, and prescribing capacity of primary care providers. More broadly, this study will also elucidate patient, clinician and system drivers and inhibitors of technology acceptance and 5As implementation to inform future technological approaches in health care settings.

2. Methods/Design

This study will progress through two key phases: (i) program development (Year 1) and (ii) program evaluation (Years 2–4; Figure 1 and Supplementary Figure). Development includes computer programming and alpha/beta-testing of the CF-5As intervention, and development of semi-structured interviews and questionnaires to assess key staff and provider attitudes, beliefs, and behaviors about smoking cessation and technology adoption. Program evaluation consists of a randomized controlled trial (RCT) to assess the effects of the CF-5As intervention on provider adherence to the 5As compared to a control group that does not receive the 5As intervention. Evaluation will also include a mixed methods study of contextual and human factors influencing both 5As and technology implementation.

Figure 1.

Figure 1

Open in a new tab

Study Design

2.1 Conceptual Model for Implementation

This study tests innovative implementation strategies to facilitate both tablet and 5As adoption based on recent adaptations to the Theory of Planned Behavior and Social Cognitive Theory (Ajzen I & Fishbein M, 1980; Bandura A, 1986). Our implementation study is guided by the Technology Acceptance Model (TAM; Davis, 1989, 1993). TAM draws from the Theory of Planned Behavior (and the closely related Theory of Reasoned Action) to explain behavior by looking at the relationships between external stimuli, cognitive responses (beliefs), affective responses (attitudes), and consequent behavioral intentions to act (Venkatesh & Davis, 2000). TAM posits that behavioral intention (to use the technology) is influenced by attitudes, which, in turn, are influenced by perceived usefulness (PU), i.e. the perception that using the technological device enhances job performance, and perceived ease of use of the technology (PEOU; Figure 2). Recent adaptations to the model have added variables such as social norms and influence (SNI), which refers to an individual’s perception of the opinions of important others, and facilitating conditions (FC), i.e. an individual’s perception of environmental factors that may impede or facilitate use of the technology (Venkatesh & Bala, 2008; Venkatesh, Morris, Davis, & Davis, 2003).

Figure 2.

Figure 2

Open in a new tab

Technology Acceptance Model.

In our study, we manipulate PEOU for the 5As by placing the 5As on a computer tablet and doing the majority of the work for the clinicians in the intervention condition. We have further manipulated PU by having the computer tablet create two handouts: a tailored, resource guide for patients, and a clinical summary with decision support tools for the clinician. By increasing PEOU and PU, we hope to promote the “acceptance” of the 5As and the computer tablets. We do not directly manipulate either SNI or FC, but assess these as important co-variants.

Our study design includes the use of iterative mixed methods (structured and semi-structured interviews and surveys) with medical staff and clinicians, and direct observation of social and organizational factors to identify factors that promote and inhibit 5As implementation. Survey items and interview prompts are explicitly based on core TAM constructs–i.e. PU, PEOU, SNI, and FC. These repeated assessments will allow us to determine if our manipulations were successful (for PEOU and PU) and to capture important co-variants (SNI, FC). Thus, our trial will allow us to answer important questions regarding the operative factors in technology acceptance and 5As implementation. Our exploration of the interdependence of patient, tablet, and clinician may point to innovative strategies to better integrate technology into (and around) the medical visit while preserving patient-centeredness and capitalizing on the powerful effects of clinician influence.

2.2 Phreesia Computer Tablet System: A Systems/Operations Perspective

Used throughout the United States, the Phreesia tablet is a wireless, touch-screen, mobile tablet designed for patient self-service check-in and health data collection. It is portable, has a long battery life, and wireless connectivity, thereby allowing patients to use it anywhere in the waiting area. All data is encrypted before being stored in the Phreesia database; no patient data are stored on the tablet. Tablets are supplied and maintained by Phreesia for a minimal subscription charge paid by the practice.

The computerized self-reported “patient interview” is a series of questions asked and answered in text via the tablet about patient demographics and insurance data, personal/family medical history, and other relevant clinical information during the check-in process. It also collects signatures for typical medical practice agreement forms. Phreesia-delivered interviews occur in both English and Spanish.

Phreesia provides a secure web accessible Practice Portal that staff and providers can access from computer workstations. Through the portal, practices can access patient information, view and edit individual records, and print and export patient data. The printout is a paper and/or PDF record of a patient’s tablet interview that is automatically produced after a patient completes the interview. Phreesia also provides integration with many leading electronic medical record vendors.

For the CF-5As intervention, we will add two key innovations to the existing Phreesia system. First, tailored information on smoking cessation and resources will be created for patients. A patient printout will include 5As-tailored health messages and referral/treatment resources. Second, a separate printed decision tool and treatment guide will be produced to better inform the clinician on 5As use. Table 1 shows where the tablet will assist in 5As delivery.

Table 1.

Use of a Computer Tablet to Deliver the U.S. Public Health Service 5As Model for Smoking Cessation

5As Component As Currently Administered As Administered in the CF- 5As
Ask Provider Tablet
Advise Provider Tablet + Provider
Assess Provider Tablet
Assist Provider Tablet + Provider
Arrange Provider Tablet + Provider
Open in a new tab

CF-5As: computer-facilitated 5As

2.3 Study Settings and Populations

Initial program development and alpha-testing of the CF-5As delivery system will occur at the University of California San Francisco (UCSF) Screening and Acute Care Clinic (SACC), where Phreesia tablets have already been implemented for patient registration. Beta-testing and the RCT will be carried out at three sites: an academic internal medicine primary care clinic, a general medicine clinic at a safety-net community hospital, and a HIV primary care practice at a safety-net community hospital. The three beta-testing and RCT sites are not currently using Phreesia or other similar tablets. All of the study sites currently use electronic medical records for clinical documentation and placing orders. While each clinic may have a small number of physician tasks done on paper, the majority are done electronically.

2.3.1 SACC

SACC is an acute care clinic in San Francisco that operates on a same-day, walk-in appointment basis for adults. Approximately 60% of patients are from community-based, non-UCSF primary care practices. In FY2011, SACC provided care to 15,569 patients (60% were female, 76% were between 25–64 years old, 48% were racial/ethnic minorities, and 93% had commercial insurance or Medicare). SACC is operated by 16 physicians, 4 nurse practitioners or physician assistants, and 30 residents doing elective rotations.

2.3.2 University of California San Francisco Adult General Medicine Clinic (UCSF)

UCSF follows approximately 18,000 patients who make about 40,200 annual visits and are seen by 30 faculty physicians, 30 primary care residents, 33 categorical residents, 3 fellows, and 5 nurse practitioners. The clinic also has 13 clinical staff and 34 administrative staff. Patients are ethnically diverse: 49% White, 24% Asian/Pacific Islander, 15% Latino, 11% African-American, and 1% American Indian. The payer mix is also diverse: 32% Medicare, 34% managed care, 15% Medicaid, 13% contracts, 4% non-capitated insurance plans, and 2% self-pay. An estimated 12% of the patients are smokers.

2.3.3 San Francisco General Hospital General Medicine Clinic (SFGH GMC)

Operated by the City and County of San Francisco, SFGH GMC serves as the primary health care facility for San Francisco’s underserved and uninsured families: 32% of patients are uninsured, 36% have Medicaid, and 31% have Medicare. Patients are diverse: 28% are Latino, 19% African-American, 20% white, 29% Asian-American, and 4% other race or ethnicity. An estimated 25% of the patients are smokers. The clinic provides approximately 25,000 patient visits to more than 6,500 patients/yr. SFGH providers include 22 faculty, 51 residents, 2 fellows, and 7 NP’s. The clinic also has 7 clinical staff and 16 administrative staff.

2.3.4 The UCSF Positive Health Program at SFGH (PHP)

PHP provided 12,088 patient visits to 2,653 people living with HIV/AIDS in 2013–14. Primary care is provided by 30 part-time faculty members, 6 fellows, and 4 full-time nurse practitioners. The clinic also has 20 clinical staff and 5 administrative staff. Patients are ethnically diverse: 19% are Latino, 22.5% African-American, 5.4% Asian, 50.9% white and 2.2% other races. PHP serves a diverse array of publicly funded insurance, including Medicare, Medicaid, Community Health Network capitated plans and Healthy San Francisco, a health access program for San Francisco’s uninsured. An estimated 40% of the patients are smokers.

2.4 Phase 1: Program and Instrument Development

Project personnel will work closely with Phreesia programmers and clinic staff to develop a CF-5As service delivery model that is tailored to each clinic. When possible, clinical data will be directly integrated into the electronic medical record. Where integration is not available, copy-paste routines from the Phreesia web-based portal into the appropriate location in the electronic medical record is an alternative. While we chose the Phreesia system for both technical and logistical reasons, we intend our CF-5As application to be exportable to similar platforms.

2.4.1 Development of the Computerized 5As Module

Validated 5As interventions, training materials, and computer smoking cessation programs (Ajzen I & Fishbein M, 1980; Dillman DA & Bowker D, 2001; Fiore MC et al., May 2008; The EX Plan; Unrod et al., 2007) will inform the 5As content optimized for the tablet platform. Formative research with patients will inform the delivery, length, and appropriate messaging. Prototypes of the layout and tablet flow will be reviewed by individuals with 5As and computer expertise. These will be tailored further based on feedback from interviews with patients using tablet screen shots. The final design will draw from best practices and visual appeal for web-based survey administration, including simplicity of format (Vehovar, Manfreda, & Batagelj, 2000), screen-based navigation that allows skip patterns(C. C. Doak, Doak, Friedell, & Meade, 1998; Zukerberg, Nichols, & Tedesco, 1999), and instructions for touch screen usage (Vehovar et al., 2000).

The CF-5As intervention will deliver smoking cessation messages and create both patient and provider printouts. Patients will receive a summary of their responses, step-by-step instructions on enhancing motivation or promoting cessation, and instructions on follow-up care. Personalized risk and recommendation information will be provided to each patient. Visuals will accompany the messages(Doak CC, Doak LG, & Root JH, 1996), and a 6th-grade reading level will be used for all written patient materials (L. G. Doak & Doak, 1987). Figure 3 shows an example of a patient printout. For the Spanish language version, we will follow a rigorous translation and adaptation method: initially, the English version of the module will be translated into Spanish by Phreesia, then it will be reviewed by bilingual project staff, and finally materials will be independently back-translated into English (Brislin RW, 1976). Any discrepancies and non-idiomatic translations will be resolved by a team of bilingual research staff.

Figure 3.

Figure 3

Open in a new tab

Sample patient printout

On the provider printout, the steps of “assist” and “arrange” will be emphasized, because these are the steps that are overlooked most often. For patients interested in cessation, information regarding local resources, quitlines, and/or smoking cessation medications will be presented to the provider to discuss with the patient. For patients not ready to quit, suggestions to improve motivation will be provided. Concrete goals for the patient will be suggested and the provider will be instructed to “arrange” a follow-up appointment. Several mock provider handouts will be created and vetted through provider focus groups and refined during the alpha and beta-testing trials. Figure 4 shows a sample provider printout.

Figure 4.

Figure 4

Open in a new tab

Sample provider printout. There is another side to the printout that provides scripts for common scenarios, as well as a table showing available doses of stop-smoking medications, instructions for their use, and their approximate costs.

2.4.2 Alpha-testing

Alpha-testing will be done at the SACC to ensure that all items are understood and 5As fidelity is maximized. Direct observations of patients using the computer tablets and evaluation of tablet-collected data will help identify any problem areas. Suggestions for functional enhancements will also be solicited from the patients. Clinicians will be provided simulated printouts and we will conduct semi-structured interviews for feedback. Staff will be interviewed to assess their perspectives on clinic flow, patient reaction, and provider buy-in. Interviews with the clinicians and staff will be audiotaped, transcribed, and systematically analyzed.

2.4.3 Beta-Testing

Beta-testing will occur in the study clinics to best adapt the product to the local environment and population, improve design, develop clinic flow processes, and document potential problems. Since the study clinics are not already using computer tablets, beta-testing will help to develop strategies to minimize interruption to regular clinic flow. To improve “perceived usefulness” of the tablet, each clinic can add customized material to the tablet on topics of interest to the clinic and their population; examples include a survey on use of emergency room services, a questionnaire on food, exercise and safety, and a survey on patient satisfaction and safety net resources such as free food and shelter. Adapting an approach by Nielsen (Nielsen, 2000), beta-testing will take place two times using post-use interviews of patients (5 patients per round per clinic), staff (2 per round per clinic), and providers (3 per round per clinic) for a total of 60 brief interviews (30 patients, 12 staff, 18 providers) to assess the content, appeal, and ease of use of the tablet and the printouts. The CF-5As module will be delivered to 5 consecutive patients with minimal additional assistance from research staff. After module completion, each patient will be interviewed individually to assess comprehension of each module section and to obtain his or her reaction to the motivational messages. Patients will receive printouts and will be encouraged to discuss them with their providers.

2.4.4 Developing Evaluation Instruments and an Implementation Plan

Contextualization of TAM in health care settings requires an understanding of the factors comprising perceived usefulness, social norms and influence, and facilitating conditions in these settings, and how these factors could inform implementation strategies. We posit that an iterative process of beliefs elicitation with medical staff and clinicians and observational analysis of social and organizational factors will allow us to define these variables, design evaluation tools to measure them, and adapt our intervention to promote 5As implementation. To best understand implementation drivers, interviews, direct observations, and brief surveys will be triangulated and compared.

In an iterative pre-testing phase, we will conduct beliefs elicitation interviews with SACC personnel to construct TAM-based questionnaires contextualized for a health care setting. These newly developed questionnaires will then be used for pre-intervention data collection to revise the intervention as indicated. For example, staff may indicate that weekly performance feedback from the clinic director could motivate better compliance with distributing tablets to patients (social norms and influence). Providers might indicate that readily available smoking brochures (facilitating conditions) would improve 5As fidelity. After pre-intervention data collection, implementation plans will be tailored for each study clinic using interview, survey, and observational data to inform refinements during beta-testing.

The most commonly occurring themes from beliefs elicitation interviews with physicians and staff from each clinic will be converted into questionnaire items and pilot tested in the traditional Theory of Planned Behavior format (Francis et al., 2004). Revised questionnaires will be administered in the RCT to more quickly assess TAM variables over the course of the study. TAM suggests that buy-in from staff and providers is strongly predicted by perceived usefulness. We will design and tailor by study site strategies to increase perceived usefulness and make use of facilitating conditions and social influence around this new technology tailored to suit a particular clinical context.

2.5 Phase 2: Study Design, Recruitment, and Evaluation of the CF-5As Model and Implementation Factors

2.5.1 Study design

We will evaluate our hypothesis that CF-5As will improve the frequency and fidelity with which 5As are administered with a RCT in three primary care clinics. After pre-intervention assessment, providers will be randomized for an implementation trial.

2.5.2 Randomization

Providers will be randomly assigned to intervention groups, stratified by site. Within each clinic site, using restricted randomization (Hayes & Moulton, 2009), we will create two sets of providers matched on personal and practice characteristics (e.g., provider sex, percentage of clinical effort, provider type: attending, nurse practitioner, fellow, resident); one set per site will be randomized to intervention, the other to control. Randomization at the provider level was chosen due to a low number of study sites and to reduce the risk of bias from providers potentially altering their behaviors as a result of treating patients in both study arms. Enrolled patients who smoke will be assigned to a study arm based on whether the provider that they are seeing that day (who is not necessarily their primary care physician) is randomized to the intervention or control group. Smoking patients seeing any randomized provider will be eligible, regardless of the type of visit (e.g. acute visit, new patient visit, or routine follow-up).

2.5.3 Eligibility criteria

All providers will be invited and incentivized to participate in the pre-intervention assessments. Participation in the RCT by randomized providers involves standard clinical care (with intervention providers receiving a 5As printout and control providers receiving nothing). Participation in the pre-intervention assessments is not a prerequisite for provider inclusion in the study. Providers must be in internal or family medicine, actively engaged in patient care, and plan to continue practicing in their current location for at least one year. No education or training on the 5As will be done with providers as part of this study. Providers randomized to the CF-5As condition will be incentivized to participate in pre- and post-intervention interviews and surveys. All providers will be incentivized to participate in subsequent interviews and surveys. They will be compensated with a $25 gift card for their participation.

2.5.4 Intervention and control conditions

Smoking patients of providers assigned to the intervention group will receive the CF-5As module, and printouts will be generated and delivered to both patient and provider during the same clinic visit. The printouts are delivered by a clinical research coordinator or research assistant. The module is expected to take approximately 5–10 minutes to complete. Patients of control providers will be given the tablets for study enrollment and demographics only (neither patient nor provider will receive a printout). Measures of 5As utilization and fidelity will be obtained by calling patients after their primary care visit, i.e. requiring no provider effort. Patient surveys will be used rather than medical chart review or provider surveys as medical charts are often incomplete and providers often over-estimate their levels of adherence (Conroy et al., 2005).

2.5.5 Patient recruitment

During the pre- and post-intervention periods, all mentally-competent, English- and Spanish-speaking patients 18+ years of age will be provided a tablet by the front desk staff or a research assistant and screened for eligibility on the tablet. Patients reporting smoking at least one cigarette in the past 7 days and having smoked more than 100 cigarettes in their lifetime will be invited to participate in the study. Signed consent will be obtained on the tablet and two phone numbers and a mailing address where the patient can be reached after the primary care visit will be obtained. If a patient consents to the study but does not have time to initiate or complete the module prior to their visit with a provider, then the survey is deemed incomplete and the patient is not eligible for participation in the study.

2.5.6 Assessment and Outcome Measures

The main outcomes that we will assess are the impact of the CF-5As service delivery model on 5As fidelity and evaluation of innovative implementation strategies to facilitate both tablet and 5As adoption based on the Technology Acceptance Model. Prior to implementation of the 5As intervention, primary care providers in the study and their patients, as well as clinic staff, will complete a baseline assessment.

2.5.6.1 Patient survey measures

For patients, the assessments are comprised of a brief survey of demographic questions administered prior to the primary care visit and either a telephone or in-person interview after the visit about smoking habits and about discussions related to smoking that occurred during the clinical visit (Table 2). The post-visit survey must occur within 72 hours of the patient’s visit, and participants will be compensated with a $20 gift card once the post-visit call or interview is completed. Specifically, 5As fidelity will be assessed by asking patients whether they were asked about their tobacco use patterns, whether they were advised to quit, whether their willingness to quit was assessed, whether they were assisted in their quit attempt (if interested), and whether follow-up was arranged. Although some patients have multiple clinic visits during the time of the study, each patient can only participate in the study up to 3 times assuming that they still meet eligibility criteria. Previously enrolled participants will be identified either by the tablet after they enter their demographic information, by answering a question about how many times they participated (if 3 or more, they will be ineligible to continue), or through staff verification of eligibility through the Phreesia dashboard.

Table 2.

Post-Primary Care Visit Survey Questions

Patient smoking habits
 Duration of smoking
 Current smoking patterns (days/week, cigarettes/day)
 Time before first cigarette in the morning (> or ≤ 30 minutes)
 Interest in cessation
Smoking assessment during visit (by tablet, staff, or provider)
 Asked if currently smoke (Ask)
 Advised to quit smoking (Advise)
 Asked about readiness to quit smoking (Assess)
 Asked about a quit plan and/or acceptable next steps
 Asked about roadblocks to smoking cessation
 Provided resources such as educational materials, counseling, referrals (Assist)
 Offered pharmacotherapy (Assist)
 Set quit date (Assist)
 Arranged follow-up visit (Arrange)
Provider factors
 Quality of explanations
 Listening skills
 Discussion of health concerns, history
 Time spent
 Respect
Technology
 Preference for source of smoking cessation counseling
 Use of technology in addition to in-person counseling
 Preferred technology medium (e.g. website, tablet, phone)
 Location of counseling
 Use of technology for other health conditions/behaviors
Open in a new tab
2.5.6.2 Clinical team survey measures

For providers, administrative staff, and nursing staff, assessments at baseline and at the end of the study include semi-structured interviews and surveys about technology acceptance and tablet implementation based on TAM core variables (Table 3). Staff surveys will focus on factors such as leadership support, organization structure, workload, communication, and perceived social norms of the workplace. Provider surveys include the above and attitudes about smoking and smoking cessation. We will conduct semi-structured interviews with clinic staff and providers to assess TAM variables (PU, PEOU, SNI, FC) and to identify barriers, facilitators, and unintended consequences of tablet implementation at each site. Individuals will be compensated with a $25 gift card for each questionnaire and interview.

Table 3.

Provider and Clinical Staff Assessments as Related to the CF-5As at Different Study Times

Pre-intervention (baseline) During intervention Post-intervention
Primary Care Providers

  • Qualitative interviews

  • Survey 1: Facilitating Conditions, Social Norms and Influence

  • Survey 2: Attitudes*

  • Survey 3: Tablet Technology Implementation

  • Survey 4: New Jersey Health Care Provider Tobacco Survey

 Primary Care Providers

  • Qualitative interviews

 Primary Care Providers

  • Qualitative interviews

  • Survey 1: Facilitating Conditions, Social Norms and Influence

  • Survey 2: Attitudes*

  • Survey 3: Tablet Technology Implementation
Administrative Staff

  • Qualitative interviews

  • Survey 1: Facilitating Conditions, Social Norms and Influence

  • Survey 2: Attitudes*

  • Survey 3: Tablet Technology Implementation

 Administrative Staff

  • Qualitative interviews

  • Survey: Tablet Technology Implementation

 Administrative Staff

  • Qualitative interviews

  • Survey 1: Facilitating Conditions, Social Norms and Influence

  • Survey 2: Attitudes*

  • Survey 3: Tablet Technology Implementation
Clinical Staff

  • Qualitative interviews

  • Survey 1: Facilitating Conditions, Social Norms and Influence

  • Survey 2: Attitudes*

  • Survey 3: Tablet Technology Implementation

 Clinical Staff

  • Qualitative interviews

  • Survey: Tablet Technology Implementation

 Clinical Staff

  • Qualitative interviews

  • Survey 1: Facilitating Conditions, Social Norms and Influence

  • Survey 2: Attitudes*

  • Survey 3: Tablet Technology Implementation
Baseline Clinic Demographics Ending Clinic Demographics
Open in a new tab
*

Attitudes related to technology and smoking/smoking cessation

During baseline assessment, we will also examine provider factors that might influence 5As fidelity using the New Jersey Health Care Provider Tobacco Survey to assess provider attitudes and knowledge about smoking and smoking cessation (Gundersen DA et al., December 2008; Steinberg & Delnevo, 2007). This extensive, validated survey includes questions on tobacco dependence treatment practices, barriers to treatment, attitudes towards cessation, perceived effectiveness of treatment, self-efficacy, awareness of clinical guidelines, and awareness/referrals to local quitlines. The surveys will allow us to evaluate any associations between these factors and 5As fidelity.

2.5.6.3 Other measures

On the clinic level, we will measure tablet implementation by tablet saturation, defined as the percentage of English and Spanish-speaking patients seen in a clinic that are given a tablet upon check-in by front desk staff. Tablet saturation will be assessed by observational data collected over several days at each study site both towards the middle and the end of the recruitment period. Information about rates of tablet referral, patient use of the tablets, provider compliance with the 5As recommendations, in addition to information about institutional, social, and structural/environmental facilitators and barriers to implementation will be collected at each clinic. This will be done through surveys and interviews as well as direct observations of clinic flow and operations at each study site. By accounting for site-related contextual factors, a more thorough understanding of implementation can be obtained.

2.5.7 Quantitative Data Analysis Plan

For provider surveys, 3 comparisons are planned: post-test group comparison, group-by-time interaction, and pre-post comparison. For staff surveys, only a pre-post comparison is planned because staff will work with both intervention and control patients.

2.5.7.1 Analysis of 5As fidelity

The primary outcome is 5As fidelity, i.e. patient-reported receipt of each of the components of the 5As during their clinic visit. Thus, our primary analysis will focus on comparisons between intervention groups on 5As fidelity. Patient-level 5As measures will be regressed onto the intervention group and categorical time (baseline versus intervention phase) indicators, as well as their interaction. Significant intervention group differences will be interpreted and described. Secondary analyses will also be performed to explore potential intervention effect modifiers. Specifically, we will evaluate interactions between the intervention groups with recruitment site, provider sex, and provider clinical experience, as well as interactions between patient demographics (age, race/ethnicity, sex, and language) and intervention group.

2.5.7.1.1 Power analyses for 5As fidelity outcomes

By randomizing providers in this study, contamination could bias the results if providers assigned to opposing experimental groups discuss the study with each other; however, this would tend to bias toward the null. We have addressed this by increasing the planned sample size, which allows for detection of smaller intervention effects. We plan to recruit 624 patients during the baseline phase and 1526 patients during the intervention phase. Assumptions included intention-to-treat analyses, 80% power, two-tailed alpha = 0.05, 80% of enrolled patients completing a phone interview, case-wise deletion of missing data (for power analysis only), and logistic regression models testing group differences at follow-up as well as the group-by-time interaction. For the 5As outcomes, we expect baseline rates equal to 70% (Advise), 60% (Assess), 50% (Assist: broadly), 25% (Assist: quitline), and 10% (Arrange). The data have a 4-level structure: sites, providers, patients, and repeated assessments. Intra-site and intra-provider-and-site correlations were assumed to equal 0.05 and 0.10, respectively. The minimum detectable effect sizes for group differences were 70% vs. 77.9% (OR=1.51) for Advise, 60% vs. 68.7% (OR=1.46) for Assess, 50% vs. 59% (OR=1.44) for Assist: broadly, 25% vs. 33.2% (OR=1.49) for Assist: quitline, and 10% vs. 16.1% (OR=1.73) for Arrange. For group-by-time (baseline versus intervention phase) interaction effects, assuming no time effect in the control group, the minimum detectable group differences were 70% vs. 81.2% (OR=1.97) for Advise, 60% vs. 72.3% (OR=1.87) for Assess, 50% vs. 62.2% (OR=1.78) for Assist: broadly, 25% vs. 36% (OR=1.95) for Assist: quitline, and 10% vs. 21.5% (OR=2.73) for Arrange.

2.5.7.2 Effects of tablet implementation

The post-intervention phase surveys will allow evaluation of the effects of implementation on the clinic and staff/provider attitudes and knowledge including TAM variables of PU, PEOU, SNI, and FC. Responses to the staff and provider surveys are reported on Likert scales. For analysis of the provider data, 3 comparisons are planned: post-test group comparison, group-by-time interaction, and pre-post comparison within the intervention group. For the staff data, only a pre-post comparison is planned because staff will work with both intervention and control patients.

2.5.7.2.1 Power analyses for effects of tablet implementation

We assume 144 providers and 36 staff will provide both pre- and post-assessments, intra-person correlation of repeated measurements=0.50, unit-standardized continuous outcomes assessed via linear mixed models and binary outcomes with base rate=40% assessed via alternating logistic regression. The minimum detectable effect sizes, d and odds ratios (OR), for provider data equaled d=0.44 & OR=2.73 (post-test group effect), d=0.45 & OR=2.72 (group-by–time effect), and d=0.33 & OR=2.00 (pre-post effect). For the pre-post staff comparison, d=0.44 & OR=2.73 (all estimated by simulation).

2.5.8 Qualitative Data Analysis Plan

Formative, process and outcome evaluations through observations and semi-structured interviews with providers and staff will be conducted to assess TAM variables and to identify barriers, facilitators, and unintended consequences of tablet implementation at each site. Work patterns, institutional policies, physical layout, patient population characteristics, and other factors vary between study sites, and an in-depth understanding of these contingencies is essential to evaluating implementation. Careful documentation of contextual factors at each site will allow for a more thorough explanation of partial adoption, adaptation, or workarounds. Constant-comparative analysis will identify commonalities and differences in implementation barriers and facilitators across sites and over time.

In Year 3, an implementation (process) evaluation will use direct observations and interviews with providers, staff, and patients to gather detailed information about the implementation process, with particular attention to intervention activities and clinic processes and structures. Interviews with providers and staff will be conducted in Year 4 to assess perceived usefulness of the CF-5As and perspectives on the overall effectiveness of the program.

Qualitative findings will report on institutional, social, and structural/environmental facilitators and barriers to implementation, as well as any unintended consequences of the tablet intervention on clinic processes and patient care. Our analysis will be informed by the mixed methods approach “merging data” (Creswell, Klassen, Plano Clark, & Smith, 2011) in which the scope of triangulation is expanded such that one type of data is used to explain, support, refute, or add complexity and meaning to another. For example, analysis of interviews and clinic observations will help to explain differing rates of adherence to the 5As among clinicians.

3. Discussion

This study protocol plans to evaluate a novel technological tool to facilitate implementation of the evidence-based 5As for smoking cessation in primary care settings. By combining the data-gathering capability of a computer tablet with a provider’s counseling and treatment skills, adherence to the 5As may be improved, and subsequently promote successful tobacco cessation among more smokers. More broadly, we will study the various facilitators and barriers to successful technology implementation and continued use in outpatient settings as framed by the Technology Acceptance model and its focus on perceived usefulness, perceived ease of use, social norms and influence, and facilitating conditions. Results will be used to adapt our computer delivery system to our current study sites and to promote the efficient and effective use of digital health interventions for substance abuse and other behavioral disorders.

Computerized health screens have been used to screen for and provide brief intervention on health behaviors such as alcohol and drug use (Fiore MC et al., May 2008; Holtz, Landis, Nemes, & Hoffman, 2001; Schwartz et al., 2014). Despite initial concerns about privacy and compliance, patients are willing to share sensitive data with the computer and some studies suggest disclosure and accuracy may be superior to personal interviews (Bachman, 2003; Newman et al., 2002). If successful, the CF-5As will combine a tool for screening for tobacco use with a treatment plan that will extend after the visit. Adaptations to the intervention may also allow for screening for and intervening on other health behaviors and substance use in the future.

Both the CF-5As model and the TAM-informed implementation interventions are exportable products that could promote evidence-based practices for substance use in primary care. Upon study completion (i.e. in year 5), we hope to disseminate our work through both academic organizations and a national network of academic and community clinics. All community clinics currently using the Phreesia pad will be offered free access to the CF-5As program (via automatic downloads) and direct consultation to facilitate local implementation. We hope that the CF-5As can be incorporated into a variety of different platforms and electronic medical records, to increase accessibility and ease of implementation with pre-existing technological systems in clinics.

Our CF-5As evaluation will be guided by the RE-AIM framework (reach, efficacy, adoption, implementation, maintenance) (Glasgow, Vogt, & Boles, 1999). This will support our goal of evaluating individual and practice-level changes, and creating an intervention that can be disseminated to other clinical settings. Reach will be assessed by determining the percentage of eligible adult smokers recruited at each site and how many completed post-visit follow-up surveys. Provider adherence to the 5As after implementation of the intervention will help determine its efficacy. Direct observations, interviews, and questionnaires will allow for assessment of tablet/technology adoption and implementation at our clinical study sites, and 5As fidelity will continue to be assessed after trial conclusion to evaluate maintenance.

Strengths of the study include the diversity of the three study sites, two of which are located in a safety-net setting. This will allow for our findings to be generalizable to a broader audience, and allows us to evaluate the effects of a broader range of patient- and clinic-level factors on technology implementation. One limitation is that the study population only includes English and Spanish-speaking individuals; future efforts should include additional languages to be accessible to a larger audience. Another limitation is that our outcome, 5As fidelity by the provider, is only assessed by patient report rather than including directly-observed measures of 5As fidelity.

An increasing number of technological tools are being developed for use in health care settings. Using technology that engages patients, providers, and clinic staff to promote behavioral interventions such as smoking cessation may provide an innovative platform through which evidence-based practices can be implemented. The process of implementation can also examine the underlying implementation science of computer-aided service delivery models with important implications for the integration of other substance use or behavioral health interventions in primary care.

Supplementary Material

supplement

Supplementary Figure 1. CF-5As timeline

Highlights.

  • We will develop a computer facilitated 5As (CF-5As) model for smoking cessation

  • A randomized control trial in primary care clinics will test provider adherence to the 5As

  • Mixed methods will evaluate factors influencing 5As and technology adoption

Acknowledgments

This work was supported by NIH National Institute on Drug Abuse (R01DA034253) and NIH NRSA T32HP19025 (Kalkhoran). The funding sources had no role in the study design; in the collection, analysis, and interpretation of the data; in the writing of the report; and in the decision to submit the article for publication. The authors would like to thank Catrina Chambers, PhD, MSPH for her review of and feedback on the manuscript.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall, Inc; 1980. [Google Scholar]
  2. Bachman JW. The Patient-Computer Interview: A Neglected Tool That Can Aid the Clinician. Mayo Clin Proc. 2003;78(1):67–78. doi: 10.4065/78.1.67. http://dx.doi.org/10.4065/78.1.67. [DOI] [PubMed] [Google Scholar]
  3. Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall, Inc; 1986. [Google Scholar]
  4. Brislin RW. Translation: Applications and Research. New York, NY: Gardner; 1976. [Google Scholar]
  5. Civljak M, Stead LF, Hartmann-Boyce J, Sheikh A, Car J. Internet-based interventions for smoking cessation. Cochrane Database Syst Rev. 2013;7:Cd007078. doi: 10.1002/14651858.CD007078.pub4. [DOI] [PubMed] [Google Scholar]
  6. Conroy MB, Majchrzak NE, Silverman CB, Chang Y, Regan S, Schneider LI, Rigotti NA. Measuring provider adherence to tobacco treatment guidelines: a comparison of electronic medical record review, patient survey, and provider survey. Nicotine Tob Res. 2005;7(Suppl 1):S35–43. doi: 10.1080/14622200500078089. [DOI] [PubMed] [Google Scholar]
  7. Creswell JW, Klassen AC, Plano Clark VL, Smith KC. Best practices for mixed methods research in the health sciences. Bethesda (Maryland): National Institutes of Health; 2011. [Google Scholar]
  8. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly. 1989;13:319–339. [Google Scholar]
  9. Davis FD. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International journal of man-machine studies. 1993;38(3):475–487. [Google Scholar]
  10. Dillman DA, Bowker D. The web questionnaire challenge to survey methodologists. In: Reips Ulf-Dietrich, Bosnjak Michael., editors. Dimensions of Internet Science. Germany: Pabst Science Publishers; 2001. [Google Scholar]
  11. Doak CC, Doak LG, Root JH. Teaching Patients with Low Literacy Skills. Philadelphia, PA: J.B. Lippincott Company; 1996. [Google Scholar]
  12. Doak CC, Doak LG, Friedell GH, Meade CD. Improving comprehension for cancer patients with low literacy skills: strategies for clinicians. CA: A Cancer Journal for Clinicians. 1998;48(3):151–162. doi: 10.3322/canjclin.48.3.151. [DOI] [PubMed] [Google Scholar]
  13. Doak LG, Doak CC. Lowering the silent barriers to compliance for patients with low literacy skills. Promot Health. 1987;8(4):6–8. [PubMed] [Google Scholar]
  14. Fiore MC, Jaen CR, Baker TB, et al. Treating Tobacco Use and Dependence: 2008 Update. Rockville, MD: U.S. Department of Health and Human Services. Public Health Service; May, 2008. [Google Scholar]
  15. Fishbein M, Ajzen I. Predicting and Changing Behavior: The Reasoned Action Approach. New York, NY: Psychology Press; 2010. [Google Scholar]
  16. Francis JJ, Eccles MP, Johnston M, Walker A, Grimshaw J, Foy R, Bonetti D. Constructing questionnaires based on the theory of planned behaviour. A manual for health services researchers. 2004;2010:2–12. [Google Scholar]
  17. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–1327. doi: 10.2105/ajph.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gundersen DA, Steinberg MB, Jordan HM, Delnevo CD, Berger HF, Hrywna M. The 2008 New Jersey Health Care Provider Tobacco Survey: A Statewide Report for the New Jersey Department of Health and Senior Services. New Brunswick, NJ: University of Medicine and Dentistry of New Jersey - School of Public Health; Dec, 2008. [Google Scholar]
  19. Hayes RJ, Moulton LH. Cluster randomized trials. Boca Raton, Fla.; London: Chapman & Hall/CRC; 2009. [Google Scholar]
  20. Holtz K, Landis R, Nemes S, Hoffman J. Development of a computerized screening system to identify substance abuse in primary care. J Healthc Qual. 2001;23(3):34–37. 45. doi: 10.1111/j.1945-1474.2001.tb00346.x. [DOI] [PubMed] [Google Scholar]
  21. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. Jama. 1998;280(15):1339–1346. doi: 10.1001/jama.280.15.1339. [DOI] [PubMed] [Google Scholar]
  22. Jamal A, Agaku IT, O’Connor E, King BA, Kenemer JB, Neff L. Current cigarette smoking among adults - United States, 2005–2013. MMWR Morb Mortal Wkly Rep. 2014;63(47):1108–1112. [PMC free article] [PubMed] [Google Scholar]
  23. King BA, Dube SR, Babb SD, McAfee TA. Patient-reported recall of smoking cessation interventions from a health professional. Prev Med. 2013;57(5):715–717. doi: 10.1016/j.ypmed.2013.07.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Marcy TW, Kaplan B, Connolly SW, Michel G, Shiffman RN, Flynn BS. Developing a decision support system for tobacco use counselling using primary care physicians. Inform Prim Care. 2008;16(2):101–109. doi: 10.14236/jhi.v16i2.681. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Munoz RF, Barrera AZ, Delucchi K, Penilla C, Torres LD, Perez-Stable EJ. International Spanish/English Internet smoking cessation trial yields 20% abstinence rates at 1 year. Nicotine Tob Res. 2009;11(9):1025–1034. doi: 10.1093/ntr/ntp090. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Newman JC, Des Jarlais DC, Turner CF, Gribble J, Cooley P, Paone D. The differential effects of face-to-face and computer interview modes. Am J Public Health. 2002;92(2):294–297. doi: 10.2105/ajph.92.2.294. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Nielsen J. Why you only need to test with 5 users: Alertbox 2000 [Google Scholar]
  28. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24–34. doi: 10.1007/s10488-008-0197-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Quinn VP, Stevens VJ, Hollis JF, Rigotti NA, Solberg LI, Gordon N, Zapka J. Tobacco-cessation services and patient satisfaction in nine nonprofit HMOs. Am J Prev Med. 2005;29(2):77–84. doi: 10.1016/j.amepre.2005.04.006. [DOI] [PubMed] [Google Scholar]
  30. Schroeder SA. What to do with a patient who smokes. Jama. 2005;294(4):482–487. doi: 10.1001/jama.294.4.482. [DOI] [PubMed] [Google Scholar]
  31. Schwartz RP, Gryczynski J, Mitchell SG, Gonzales A, Moseley A, Peterson TR, O’Grady KE. Computerized versus in-person brief intervention for drug misuse: a randomized clinical trial. Addiction. 2014;109(7):1091–1098. doi: 10.1111/add.12502. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Sciamanna CN, Marcus BH, Goldstein MG, Lawrence K, Swartz S, Bock B, Ahern DK. Feasibility of incorporating computer-tailored health behaviour communications in primary care settings. Inform Prim Care. 2004;12(1):40–48. doi: 10.14236/jhi.v12i1.107. [DOI] [PubMed] [Google Scholar]
  33. Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, Haynes RB. Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6:87. doi: 10.1186/1748-5908-6-87. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Stead LF, Buitrago D, Preciado N, Sanchez G, Hartmann-Boyce J, Lancaster T. Physician advice for smoking cessation. Cochrane Database Syst Rev. 2013;5:Cd000165. doi: 10.1002/14651858.CD000165.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Stead LF, Hartmann-Boyce J, Perera R, Lancaster T. Telephone counselling for smoking cessation. Cochrane Database Syst Rev. 2013;8:Cd002850. doi: 10.1002/14651858.CD002850.pub3. [DOI] [PubMed] [Google Scholar]
  36. Steinberg MB, Delnevo CD. Physician beliefs regarding effectiveness of tobacco dependence treatments: results from the NJ Health Care Provider Tobacco Survey. J Gen Intern Med. 2007;22(10):1459–1462. doi: 10.1007/s11606-007-0282-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. The EX Plan. Retrieved January 23, 2015, from http://www.becomeanex.org.
  38. Tong EK, Strouse R, Hall J, Kovac M, Schroeder SA. National survey of U.S. health professionals’ smoking prevalence, cessation practices, and beliefs. Nicotine Tob Res. 2010;12(7):724–733. doi: 10.1093/ntr/ntq071. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. U.S. Department of Health and Human Services. The Health Consequences of Smoking: 50 Years of Progress. A Report of the Surgeon General. Atlanta, GA: Centers for Disease Control and Prevention, National Center on Chronic Disease Prevention and Health Promotion, Office on Smoking and Health; 2014. [PubMed] [Google Scholar]
  40. Unrod M, Smith M, Spring B, DePue J, Redd W, Winkel G. Randomized controlled trial of a computer-based, tailored intervention to increase smoking cessation counseling by primary care physicians. J Gen Intern Med. 2007;22(4):478–484. doi: 10.1007/s11606-006-0069-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Vehovar V, Manfreda KL, Batagelj Z. Sensitivity of e-commerce measurement to survey instrument; Paper presented at the 13th International Bled Electronic Commerce Conference.2000. [Google Scholar]
  42. Venkatesh V, Bala H. Technology acceptance model 3 and a research agenda on interventions. Decision sciences. 2008;39(2):273–315. [Google Scholar]
  43. Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: four longitudinal field studies. Management science. 2000;46(2):186–204. [Google Scholar]
  44. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: Toward a unified view. MIS Quarterly. 2003:425–478. [Google Scholar]
  45. Yarbrough AK, Smith TB. Technology acceptance among physicians: a new take on TAM. Medical Care Research and Review. 2007 doi: 10.1177/1077558707305942. [DOI] [PubMed] [Google Scholar]
  46. Zukerberg A, Nichols E, Tedesco H. Designing surveys for the next millennium: Internet questionnaire design issues. National center for education statistics. 1999:79. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement

Supplementary Figure 1. CF-5As timeline

NIHMS697921-supplement.pptx (57.6KB, pptx)

RESOURCES