Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Jan 1.
Published in final edited form as: Am Indian Alsk Native Ment Health Res. 2022;29(3):71–89. doi: 10.5820/aian.2903.2022.71

RETENTION IN A 6-MONTH SMOKING CESSATION STUDY AMONG ALASKA NATIVE AND AMERICAN INDIAN PEOPLE

Dawson E Mills 1, Krista R Schaefer 2, Julie A Beans 3, Michael R Todd 4, Renee F Robinson 5, Kenneth E Thummel 6, Denise A Dillard 7, Jaedon P Avey 8
PMCID: PMC9552566  NIHMSID: NIHMS1840363  PMID: 36178748

Abstract

Participant retention in longitudinal health research is necessary for generalizable results. Understanding factors that correlate with increased retention could improve retention in future studies. Here, we describe how participant and study process measures are associated with retention in a longitudinal tobacco cessation research study performed in Anchorage, Alaska. Specifically, we conducted a secondary analysis exploring retention among 151 Alaska Native and American Indian (ANAI) people and described our study processes using study retention categories from a recent meta-analysis. We found that our study processes influence retention among ANAI urban residents more than measures collected about the participant. For study process measures, calls where a participant answered and calls participants placed to the study team were associated with higher retention. Calls where the participant did not answer were associated with lower retention. For participant measures, only lower annual income was associated with lower retention at 6 weeks. Promoting communication from participants to the study team could improve retention, and alternative communication methods could be used after unsuccessful calls. Finally, categorizing our study retention strategies demonstrated that additional barrier-reduction strategies might be warranted.

INTRODUCTION

High participant retention between study enrollment and conclusion is often needed to ensure sufficient power and unbiased results (Young et al., 2006; Redwood et al., 2010; Robinson et al., 2007; Singh et al., 2018; Teague et al., 2018). Characteristics of those who participate consistently and are retained in longitudinal cohort studies often differ in important ways from those who miss follow-up appointments, leading to lower generalizability of findings (Young et al., 2006; Redwood, 2010; Odierna & Schmidt, 2009; Robinson et al., 2007).

Strategies to maximize participant retention, especially among underserved and minority populations, must be considered when researchers design and implement longitudinal studies. Doing so will ensure generalizability of findings, account for population-specific differences, and promote equitable provision of health services—including health research. A tumultuous relationship with the United States government and health care systems, as well as health research misconduct, have led to a legacy of mistrust in health research among many minority communities (Kennedy et al., 2007; Dillard et al., 2019). This, along with other factors, may contribute to the reported difficulty of retaining study participants from diverse and often under-represented populations (Clark, 2012; Nicholson et al., 2015). The effectiveness and reproducibility of retention strategies that work well for one population might not be as effective in another. Importantly, geographic considerations, such as those found in Alaska, may necessitate different retention strategies and support efforts (Beans et al., 2019; Clark, 2012).

One area where retention is of consistently great concern is tobacco cessation research. Research in this area is critically needed because tobacco cessation is extremely difficult for most people, with low annual quit rates and many attempts needed, on average, before tobacco cessation success (Chaiton et al., 2016; Kotz et al., 2014; Hensel et al., 1995; Borland et al., 2011; Kealey et al., 2007). Research on such programs is often impaired by relatively high numbers of participants leaving the tobacco cessation program before the program end date. The proportion of participants who fully complete these programs is often low; for instance, Daley et al. (2018) report that slightly less than three-quarters of participants who began a tobacco cessation program implementation study completed a 24-week follow-up. This proportion is actually quite good, as Smith et al. (2017) report that the proportion of participants who complete studies is “often in the low-to-moderate range, from under half up to three-quarters of the original sample” (p. 171). Improving tobacco cessation research by adopting strategies that increase retention could increase the generalizability of results and accordingly improve tobacco cessation program outcomes.

There is also a strong need for dissemination of information regarding retention practices used in longitudinal studies, as the retention literature is limited and sometimes contradictory in its findings (Robinson et al., 2007; Teague et al., 2018). For instance, a systematic review from 2007 and an updated review in 2015 found that a higher number of retention strategies (e.g., alternative contacts or having a dedicated research phone number) correlated with higher overall retention, while a recent meta-analysis found the association non-significant (Robinson et al, 2015; Robinson et al., 2007; Teague et. al, 2018). Varying recruitment settings and populations may contribute to these seemingly contradictory findings. Recent work by Teague et al. (2018) recommended using a systematic approach to report retention strategies and resulting outcomes. The Teague classification system has four domains of study retention strategies: barrier-reduction, community-building, follow up and reminder, and tracing (Teague et al., 2018).

In this paper we identify participant and study process measures associated with participant retention in a six-month, observational study of urban Alaska Native and American Indian people (ANAI) who smoke, a group rarely described in the retention literature (Redwood et al., 2010; Teague et al., 2018). In addition, we used Teague et al.’s categorization scheme to classify the retention strategies employed in this longitudinal study and identify areas for improvement. Framing our study processes in a standardized way, evaluating which measures correlate with study retention, and identifying which retention strategies were helpful could improve participant retention and increase the generalizability of longitudinal tobacco cessation research.

METHODS

Setting

The study was conducted at Southcentral Foundation (SCF), a nonprofit, tribally-owned and operated health care organization that provides pre-paid primary care services to over 65,000 ANAI people living in southcentral Alaska. SCF’s vision is “a Native Community that enjoys physical, mental, emotional and spiritual wellness”; its mission is “working together with the Native Community to achieve wellness through health and related services” (Gottlieb, 2013).

Participants were recruited directly from SCF’s Quit Tobacco Program, which started in 2004 in response to the relatively high rate of smoking among ANAI people (Fenn et al., 2007; Hensel et al., 1995; Dilley et al., 2013). The Quit Tobacco Program is a comprehensive, multidisciplinary health service option that provides tobacco cessation, pharmacological, and counseling services to eligible ANAI individuals according to evidence-based guidelines, including the U.S. Public Health Service Clinical Practice Guideline (Fenn et al., 2007).

Study Design

This exploratory, secondary data analysis focused on measures that correlated with participant retention in a tobacco cessation study. The six-month, observational tobacco cessation study was conducted to identify the relative contributions of sociodemographic, clinical, and genetic factors affecting pharmacological interventions to help ANAI people successfully quit smoking tobacco. The cessation study consisted of three data collection study visits: baseline (before quitting tobacco), 6-weeks post quit date, and 26-weeks post quit date. At baseline, participants were asked to answer a series of surveys related to tobacco use and sociodemographic characteristics. The 6- and 26-week study visits involved surveys about tobacco use. Blood draws, an expelled carbon monoxide test, and urine sample collection were conducted at each study visit to measure several genetic and metabolic parameters pertaining to smoking cessation. Participants were given a $20 incentive at the completion of each study visit, for a potential total of $60.

At the end of each of the first two study visits, participants scheduled their next visit, unless participants asked to schedule it later. The study team provided standard business hours during which participants could schedule a study visit (8:00 A.M. to 5:00 P.M., Monday to Friday). The project also provided a specific telephone number that study participants could call or text to reach the study team. The team scheduled two participants simultaneously if the participants requested it. A reminder letter was sent approximately three weeks before the scheduled study visit. Letters were also sent to participants if they missed a study visit. Two reminder calls were made to participants, one week before and one day before the scheduled study visit. Study staff would either confirm the study visit, reschedule, or leave a voicemail. If the participant could not be reached, study staff called an alternative contact person, provided by most participants at baseline. Up to three phone call reminders were made for each scheduled study visit. When a participant missed a study visit, study staff would attempt to reschedule the visit with up to three separate contacts (two calls to the participant and one to the alternative contact). In total, study staff contacted participants up to 6 times and alternative contacts up to 2 times between each study visit. Each contact was logged in a contact database. Emails and texts were only sent to participants if they requested these communication methods.

An advisory group helped ensure that the project was acceptable to the ANAI community prior to any data collection. Regular feedback reports were sent to research partners to keep track of study data such as the number of people scheduled for study visits in the upcoming week. This project was reviewed and approved by the Alaska Area Institutional Review Board (AAIRB# 2015-10-037). In addition, this study and manuscript was reviewed and approved by SCF and the Alaska Native Tribal Health Consortium Research Review Boards prior to journal submission (Hiratsuka et al., 2017).

Participants

Individuals were eligible to participate in the tobacco cessation study if they were enrolled in SCF’s Quit Tobacco Program, self-identified as ANAI, were 18 years of age or older, smoked at least 100 cigarettes to date, and smoked in the last 24 hours. Exclusion criteria were simultaneous use of other tobacco products which included chew, electronic cigarettes, pipes, cigars, or iqmik—a smokeless tobacco used among AN people (Renner et al., 2005). Additionally, potential participants were ineligible to participate if they were being treated for cancer, had hemophilia, or were pregnant.

Informed Consent

Participants in the tobacco cessation study provided informed consent before any data were collected. The informed consent included an overview of the study and potential risks and benefits of participation in the study. Participants were informed throughout the study that participation was voluntary.

Measures and Materials

Participant measures were collected via in-person surveys and from a de-identified dataset of electronic medical record information. Study process and retention measures were recorded by study staff in a Microsoft Access® database.

Participant Measures

Participant measures included gender, age, residence type, number of people in residence, number of children in residence, number of tobacco users in residence who are trying to quit tobacco, highest level of education, annual household income, number of cigarettes smoked per day, Fagerstrom test for tobacco dependence, reported readiness to stop smoking, and the reported main reason to stop smoking. A proxy variable for employment was constructed from multiple self-report variables (i.e., ‘Yes’ to ‘do you or others smoke at work’, ‘Yes’ to ‘works nights or swing shift’, or ‘Yes’ to ‘current member of the armed forces’).

Study Process Measures

Study process measures included the number of contact attempts made to a participant or alternative contact after baseline; the number of successful calls made by research staff to a participant or alternative contact (i.e., if a researcher successfully conversed with the participant); and the number of unsuccessful call attempts made by research staff to a participant or alternative contact (i.e., participant missed call, a voicemail was left, or phone number was no longer working). Each of these types of calls were categorized to increase the statistical power by grouping the larger number of calls together (i.e., categories are 0 calls, 1 call, 2 calls, 3 calls, or 4 or more calls). Other variables included whether emails, letters, or texts were sent to a participant and whether the participant contacted the study team.

Retention Measures

The primary outcome of interest was whether a participant attended or missed follow-up study visits. In each of three intervals (e.g., baseline to 6-weeks, 6-weeks to 26-weeks, and baseline to 26-weeks), participants were stratified into ‘retained’ or ‘not-retained’ groups before tests of comparison were conducted. The baseline to 26-week interval reflects retention irrespective of attendance of the 6-week study visits.

Data Collection

Data collection for the tobacco cessation study took place between July 2016 and October 2018. Participant measures were collected by surveys during the study visit and an electronic medical record query. Study process measures were queried from the Microsoft Access® database and reviewed manually to ensure accurate data extraction for texts, calls, emails, and mailing letters. For number of calls, and their sub-distinctions, data were created through an iterative manual process by reviewing the call logs. Calls were documented for each participant, and a random sample of 20% of the extracted call data were reviewed by a second study team member to ensure inter-reviewer consistency and definitional agreement. In cases of ambiguity, two team members discussed the data until consensus was reached. A final review was conducted on the call logs to reduce the possibility of error.

Analysis

Descriptive statistics on participant and study process measures were calculated as percentages for categorical variables and median with interquartile range (IQR) for continuous variables.

We assessed the association of participant and study process measures with retention between the three intervals: baseline to 6-weeks, 6-weeks to 26-weeks, and baseline to 26-weeks. We fit separate regression models for each independent variable. Significance of association was assessed with a chi-square test for categorical data and a t-test for continuous data. Odds ratio (OR) are reported with 95% confidence intervals (CI). We emphasize ORs and CIs, rather than p-values, for interpretation of results since this is an exploratory analysis (Goodman, 2008). All analyses were conducted using R software version 3.5.3 (R Core Team, 2020).

Participant retention strategies used in this study were categorized in accordance with Teague et al. (2018). This method of reporting processes adds to the transparency in research methodology, may provide a useful tool with pragmatic examples for others to incorporate, and allowed us to identify retention strategies that worked well and areas needing improvement. Retention strategies were grouped into four main domains: barrier-reduction, community-building, follow-up and reminder, and tracing.

RESULTS

Participant Description

Participants enrolled at baseline (N = 151) included more women (58%), and the median age was 45 (Table 1). Most participants lived in a single-family home (45%) or an apartment/condominium (49%). Within a participants’ residence, over half had 3 or more people and around 20% lived by themselves. Over half did not have a child living in the residence, and around 30% of participants had another person in the residence trying to quit smoking. The majority (83%) of participants were currently employed. Most graduated high school/earned a GED (37%) or had some college education (44%). Over half (52%) had an annual household income below $30,000.

Table 1.

Distribution of participant and study process measures

Baseline
N = 151		 Baseline to 6-
weeks
Retained
N = 86		 6-weeks to 26-
weeks
Retained
N = 48		 Baseline to
26-weeks
Retained
N = 54
Participant Measures n (%) n (%) n (%) n (%)
Gender
 Female 87 (58) 50 (58) 27 (56) 30 (56)
 Male 64 (42) 36 (42) 21 (44) 24 (44)
Age, median (IQR) 45 (35-54) 47 (35-54) 48 (36-54) 48 (35-54)
Residence
 Single family home 68 (45) 45 (52) 22 (46) 23 (43)
 Apartment/Condo 74 (49) 38 (44) 24 (50) 29 (54)
 Othera 8 (5) 2 (2) 2 (4) 2 (4)
People in residence
 1 29 (19) 14 (16) 10 (21) 11 (20)
 2 40 (26) 24 (28) 14 (29) 15 (28)
 3+ 82 (54) 48 (56) 24 (50) 28 (52)
Children in residence
 0 86 (57) 45 (52) 29 (60) 32 (59)
 1+ 65 (43) 41 (48) 19 (40) 22 (41)
Residents trying to quit
 1 (self) 107 (71) 58 (67) 35 (73) 41 (76)
 2+ 44 (29) 28 (33) 13 (27) 13 (24)
Employment status
 Working 125 (83) 68 (79) 38 (79) 44 (81)
Education
 Some high school 19 (13) 9 (10) 5 (10) 5 (9)
 High school graduate/GED 56 (37) 32 (37) 18 (38) 22 (41)
 Some college 66 (44) 40 (47) 23 (48) 24 (44)
 College graduate or more 10 (7) 5 (6) 2 (4) 3 (6)
Annual household income
 <$9,999 40 (26) 16 (19) 8 (17) 10 (19)
 $10,000-29,999 40 (26) 21 (24) 15 (31) 16 (30)
 $30,000 – 49,999 25 (17) 19 (22) 10 (21) 11 (20)
 $50,000 – 69,999 23 (15) 12 (14) 8 (17) 10 (19)
 $70,000+ 23 (15) 18 (21) 7 (15) 7 (13)
Cigarettesb per day, median (IQR) 10 (7-15) 10 (7-15) 10 (7-15) 10 (7-15)
Fagerström test for nicotine dependence
 Low (0-4) 88 (58) 53 (62) 34 (71) 35 (65)
 Moderate (5-6) 37 (25) 20 (23) 8 (17) 11 (20)
 High (7-10) 26 (17) 13 (15) 6 (13) 8 (15)
Reason to stop smoking
 General health 69 (46) 39 (45) 23 (48) 26 (48)
 Cost 30 (20) 16 (19) 10 (21) 12 (22)
 Risk of cancer 12 (8) 5 (6) 2 (4) 3 (6)
 Otherc 30 (20) 26 (30) 13 (27) 13 (24)
Study Process Measures n (%) n (%) n (%) n (%)
Successful calls made to participant
 0 29 (19) 5 (6) 2 (4) 2 (4)
 1 26 (17) 11 (13) 4 (8) 6 (11)
 2 33 (22) 23 (27) 12 (25) 13 (24)
 3 22 (15) 18 (21) 12 (25) 12 (22)
 4+ 41 (27) 29 (34) 18 (38) 21 (39)
Unsuccessful calls to participant
 0 17 (11) 11 (13) 9 (19) 10 (19)
 1 27 (18) 17 (20) 11 (23) 13 (24)
 2 28 (19) 17 (20) 11 (23) 12 (22)
 3 24 (16) 15 (17) 8 (17) 10 (19)
 4+ 55 (36) 26 (30) 9 (19) 9 (17)
Participant contacted study team
 0 94 (62) 44 (51) 23 (48) 27 (50)
 1 35 (23) 28 (33) 15 (31) 16 (30)
 2+ 22 (15) 14 (16) 10 (21) 11 (20)
Study team emailed participant 21 (14) 9 (10) 2 (4) 2 (4)
Study team texted participant 5 (3) 2 (2) 1 (2) 1 (2)
Study team mailed letter to participant 115 (76) 65 (76) 32 (67) 37 (69)
Open in a new tab

Note. Numbers rounded to nearest whole number.

a

Other = halfway house, mobile home, shelter, rooming, or boarding house

b

Menthol and non-menthol cigarettes

c

Bad breath, ability to be physically active, reduce risk of heart disease, role model for family, health of family in the home

The median number of cigarettes smoked per day was 10. Fagerstrom scores for nicotine dependence were ‘Low’ (58%), ‘Moderate’ (25%), and ‘High’ (17%). The most common self-reported reason to stop smoking was ‘general health’ (46%), followed by ‘cost’ (20%), ‘reduce risk of cancer’ (8%), among other reasons.

From baseline to the next study appointment 6 weeks after the quit date, 86 (57%) participants attended. Forty-eight attended both the 6- and 26-week appointments; accordingly, complete study retention was 32%. Independent of attending the 6-week appointment, 54 (36%) participants attended the 26-week appointment.

Study Process Description

The study team made many calls (Table 1); 80% of participants spoke with the study team, with 27% having four or more calls. Only 11% of participants had zero unsuccessful calls – meaning the study team connected with the participant on every call attempt. Over a third (36%) had four or more unsuccessful calls, meaning the participant did not answer, the phone number was not a working number, or the study team left a voicemail. While most participants (62%) did not call the study team, 23% called once, and 15% called twice or more. The study team emailed 14% of participants, texted 3%, and sent letters to 76%.

Participant Measures and Retention

There were very few statistically significant participant measures associated with retention across different time intervals of the study (Table 2). Given this caution to interpretation, some trends appeared in this exploratory analysis. A positive trend in attending the 6-week and a negative trend in attending the 26-week appointments can be seen with number of children in the residence and number of tobacco users trying to quit.

Table 2.

Bivariate logistic regression for the odds ratio of retention between baseline and 6-week appointment, 6-week appointment and 26-week appointment, and baseline and 26-week appointment

Participant Measures OR a (95% CI) p OR a (95% CI) p OR a (95% CI) p
Gender 0.88 0.69 0.70
 Female 1.05 (0.55, 2.02) 0.84 (0.35, 1.99) 0.88 (0.45, 1.72)
 Male Ref Ref Ref
Age 0.14 0.17 0.10
 1 year increase 1.02 (0.99, 1.05) 1.02 (0.99, 1.06) 1.02 (1.00, 1.05)
People in residence 0.57 0.36 0.90
 1 Ref Ref Ref
 2 1.61 (0.61, 4.27) 0.56 (0.12, 2.22) 0.98 (0.37, 2.67)
 3+ 1.51 (0.65, 3.57) 0.40 (0.10, 1.38) 0.85 (0.36, 2.08)
Children in residence 0.19 0.10 0.67
 0 Ref Ref Ref
 1+ 1.56 (0.81, 3.03) 0.48 (0.20, 1.12) 0.86 (0.44, 1.69)
Residents trying to quit 0.29 0.23 0.31
 1 Ref Ref Ref
 2+ 1.48 (0.72, 3.09) 0.57 (0.23, 1.41) 0.68 (0.31, 1.42)
Employment status 0.17 0.98 0.75
 Working 0.53 (0.20, 1.27) 1.01 (0.35, 2.88) 0.87 (0.37, 2.13)
 Not working Ref Ref Ref
Education 0.74 0.91 0.76
 Some high school 0.67 (0.23, 1.92) 0.97 (0.22, 4.57) 0.55 (0.16, 1.67)
 High school/GED Ref Ref Ref
 Some college 1.15 (0.56, 2.39) 1.05 (0.41, 2.70) 0.88 (0.42, 1.85)
 College or more 0.75 (0.19, 2.98) 0.52 (0.06, 3.54) 0.66 (0.13, 2.66)
Annual household income 0.02 0.31 0.43
 <$9,999 0.19 (0.05, 0.57) 1.57 (0.40, 6.33) 0.76 (0.24, 2.45)
 $10,000-29,999 0.31 (0.09, 0.94) 3.93 (1.07, 16.0) 1.52 (0.52, 4.73)
 $30,000 – 49,999 0.88 (0.22, 3.42) 1.75 (0.48, 6.67) 1.80 (0.55, 6.11)
 $50,000 – 69,999 0.30 (0.08, 1.05) 3.14 (0.71, 15.8) 1.76 (0.53, 6.11)
 $70,000+ Ref Ref Ref
Cigarettes per day 0.76 0.57 0.85
 1 cig increase 0.99 (0.95, 1.03) 0.98 (0.93, 1.04) 1.00 (0.95, 1.04)
Fagerström test for nicotine dependence 0.60 0.15 0.48
 Low 1.51 (0.63, 3.68) 2.09 (0.61, 7.37) 1.49 (0.60, 3.96)
 Moderate 1.18 (0.43, 3.24) 0.78 (0.19, 3.24) 0.95 (0.32, 2.90)
 High Ref Ref Ref
Reason to stop smoking 0.51 0.73 0.77
 General health Ref Ref Ref
 Cost 0.88 (0.37, 2.09) 1.16 (0.36, 4.01) 1.10 (0.45, 2.64)
 Risk of cancer 0.55 (0.15, 1.89) 0.46 (0.06, 3.10) 0.55 (0.11, 2.04)
 Other 1.43 (0.64, 3.25) 0.70 (0.25, 1.89) 0.80 (0.34, 1.79)
Successful calls made to participant <0.01 0.48 <0.01
 0 0.28 (0.08, 0.94) 1.17 (0.11, 10.5) 0.25 (0.03, 1.20)
 1 Ref Ref Ref
 2 3.14 (1.09, 9.50) 1.91 (0.45, 9.03) 2.17 (0.71, 7.23)
 3 6.14 (1.73, 26.1) 3.50 (0.76, 18.4) 4.00 (1.20, 14.7)
 4+ 3.30 (1.20, 9.49) 2.86 (0.70, 13.2) 3.50 (1.21, 11.2)
Unsuccessful calls to participant 0.50 0.10 0.01
 0 1.08 (0.31, 3.96) 2.45 (0.44, 19.7) 1.54 (0.45, 5.41)
 1 Ref Ref Ref
 2 0.91 (0.30, 2.71) 1.00 (0.24, 4.16) 0.81 (0.28, 2.34)
 3 0.98 (0.31, 3.09) 0.62 (0.15, 2.58) 0.77 (0.25, 2.33)
 4+ 0.53 (0.20, 1.34) 0.29 (0.08, 1.01) 0.21 (0.07, 0.59)
Participant contacted study team <0.01 0.45 0.07
 0 Ref Ref Ref
 1 4.55 (1.89, 12.2) 1.05 (0.41, 2,75) 2.09 (0.93, 4.67)
 2+ 1.99 (0.78, 5.40) 2.28 (0.65, 9.34) 2.48 (0.96, 6.48)
Study team emailed participant b 0.16 0.05 0.02
0.52 (0.20, 1.30) 0.19 (0.03, 0.86) 0.16 (0.02, 0.58)
Study team texted participant b 0.44 0.87 0.47
0.49 (0.06, 3.05) 0.79 (0.03, 20.4) 0.44 (0.02, 3.06)
Study team mailed letter to participant b 0.85 0.04 0.10
0.93 (0.43, 1.97) 0.30 (0.09, 0.88) 0.53 (0.25, 1.14)
Open in a new tab

Note. OR = odds ratio; CI = confidence interval.

a

Each OR is for a model with one independent variable included

b

Reference group: no emails, texts, or letters

An annual household income of <$9,999 had a decreased odds of retention with an OR of 0.19 (95% CI [0.05, 0.57]) for baseline to 6-weeks compared to those in the highest income group (>$70,000). All low-income groups were less likely to attend the 6-week study visit compared to the highest income group. In contrast, almost all the low-income groups had higher odds of attending the 26-week study visit, though this was not significant, suggesting those with an annual income in this range were more likely to be retained in the later stage of the study compared to the highest income group.

Study Process Measures and Retention

Successful calls made by the study team to participants significantly correlated with increased retention, with two successful calls having an OR of 3.14 (95% CI [1.09, 9.50]) compared to one successful call (Table 2). This trend appeared between all study visit intervals. Correspondingly, if the call was unsuccessful, participants were less likely to attend follow-up study visits, but this was not significant. With more unsuccessful calls, the participant was even less likely to attend the study visit (OR = 0.81 for 2 vs. 1 unsuccessful call; OR = 0.77 for 3 vs. 1 unsuccessful call; and OR = 0.21 for 4+ vs. 1 unsuccessful call).

When participants contacted the study team for any reason, they were more likely to attend follow-up study visits (OR = 4.55, 95% CI [1.89, 12.2] for baseline to 6-weeks for 1 participant call compared to none; OR = 2.09, 95% CI [0.93, 4.67] for baseline to 26-weeks for 1 participant call compared to none).

Sending any email to a participant had a very low odds for the participant attending follow-up study visits (OR = 0.52, 0.19, and 0.16 for baseline to 6-weeks, 6-weeks to 26-weeks, and baseline to 26-weeks, respectively), though only 21 participants were sent emails. Sending any text or letter to a participant had low odds of them attending follow-up study visits, but these results were not significant.

DISCUSSION

This study assessed associations between participant and study process measures with retention in a tobacco cessation study among urban ANAI people. We found that study process measures were associated with study retention more than participant measures. One participant measure, annual income, significantly correlated with differential retention from baseline to 6-week appointment. Several study process measures correlated with differential retention, including successful calls to participants, unsuccessful calls to participants, a participant contacting the study team, and sending emails. These findings suggest that the characteristics of the participants in this tobacco cessation study were less important in determining retention than the specific retention strategies used. This emphasizes the importance of considering techniques and strategies to improve retention when designing study procedures.

We retrospectively organized the retention strategies used in this study according to the retention domains used by Teague et al. (2018).

Barrier-Reduction Strategies

First, encouraging participants to schedule follow-up study visits in-person at the baseline visit reduced the burden of scheduling on both participant and study team. Flexible scheduling from the study team, during business hours, allowed participants to fit study visit times into their personal schedule. When possible, the team scheduled two participants within the same household simultaneously if requested by the participants. Finally, the project had a specific telephone number that study participants could call or text to reach the study team during business hours, though it was not toll free.

Social determinants of health may influence tobacco cessation efforts, as transportation barriers or the fact that study visits were conducted during normal business hours may have limited access for many individuals, especially those with fewer resources or inflexibility associated with lower wage employment. Reducing these barriers to participation might help, especially as Teague et al. (2018) reported that the cumulation of barrier-reduction strategies correlated with improved study retention. Further efforts to reduce barriers to participation could be explored. Hirchak et al. (2018) recently described the popularity of gasoline vouchers and bus passes in ANAI populations during a series of focus groups with ANAI participants. Implementing barrier-reduction strategies such as providing transportation vouchers may increase retention in future studies.

Community-Building Strategies

The research study was conducted at SCF, a tribally-owned and operated health care organization familiar to community members. SCF has a strong and consistent organizational brand that was used throughout the study. The study team collaborated with an existing program, the Quit Tobacco Program, to recruit for the study. In addition, the study addressed a community-identified health priority, tobacco use. Last, an advisory group ensured project acceptability to the ANAI community before and during data collection.

Follow-Up and Reminder Strategies

The tobacco cessation study planned for multiple contact attempts from the inception of the study. Phone call reminders were placed one week before and the business day prior to each scheduled study visit (or at another time based on a communicated participant preference). Email and text reminders were used in a small number of cases if the participant requested them. Study team limited the number of calls made to each participant based on participant preference. Financial incentives were given ($20 for each appointment).

We found that increasing the number of calls to participants did not significantly correlate with higher or lower retention. This is in contrast to a recent meta-analysis examining call frequency, which found a negative association between reminder calls and study retention (Teague et al., 2018). In fact, as the number of unsuccessful calls to participants increased, the odds of a participant attending a study appointment decreased. From this finding, we might infer that continuing to call participants after an unsuccessful call may not be a worthwhile effort and different communication strategies should be employed. Higher retention was noted amongst participants who contacted the study team, suggesting study teams could explore other types of incentives for responding to calls or texts or how to make Teague et al.’s community building and tracing strategies for retention a more engaging process. For example, for longitudinal studies providing a one-page newsletter, or similar type of communication updating enrolled participants between study visits, may help to keep participants engaged in the study and, in turn, increase retention. Moreover, this strategy addresses ANAI community members requests for engagement throughout research studies (Beans et al., 2018).

Tracing Strategies

Participants were encouraged to list an alternative contact, who was called if contact to the participant was unsuccessful. Overall, to increase the success of tracing using alternative contacts, participants could be asked to provide more individuals at the outset and these alternative contacts could be called earlier after unsuccessful calls, texts, or emails.

Strengths

This study had several strengths. First, various communication strategies were used, including calls, texts, emails, and letters. This allowed for more in-depth exploration of which communication strategies were associated with increased retention. Second, we explored measures associated with retention in an understudied population and improving processes could lead to increased participation among ANAI people in future research. In turn, this could lead to more statistical power and generalizable longitudinal study findings, making research more meaningful to the broader ANAI community. Finally, we identified several actionable suggestions by categorizing retention strategies into domains that may improve future research efforts, such as utilizing alternative communication methods after an unsuccessful call to a participant, increasing communication to participants via study update newsletters, encouraging participants to call study staff, or attempting to reduce barriers to participation via transportation assistance. Even though our findings with alternative communication strategies (i.e., texts, emails, and letters) were inconclusive, the fact that additional unsuccessful calls are correlated with lower study retention suggests a need to explore innovative contact strategies, with further research on their efficacy.

Limitations

There were several limitations to this study. First, the study process measures have inherent limitations. Many staff were involved in data recording during the study, which made extracting data from communication logs challenging. Second, the study was observational and used data not intended for the examination of strategies of retention, so ascertaining causal relationships is challenging. This is especially true for many participant measures like employment. Third, the study did not recruit or conduct follow-up visits during Saturday clinic hours, which may have prevented participants who work during the workweek (M-F) from participating as most study participants were employed. Fourth, participants were not asked why they were unable to attend or what resources they would need in order to keep an appointment.

CONCLUSION

These findings suggests that study teams should carefully consider strategies to improve retention in future longitudinal research studies, especially when it involves ANAI participants. Employing the four domains in Teague’s classification system should not just be a retrospective assessment, but a prospective tool to design study processes. Our findings indicate that when participants contact the study team, retention was significantly higher. Improving participant engagement, such as incorporating study update communication, could increase participant-driven contact with the study team and should be explored. Overall, these findings could inform future tobacco cessation research strategies, increasing retention and making studies more generalizable to the broader ANAI community.

ACKNOWLEDGEMENTS

The authors would like to thank all of the study participants, the health educators working in the SCF Quit Tobacco Program, and pharmacy and medical assistant staff at the Alaska Native Primary Care clinic. The authors are grateful to the members of the Southcentral Foundation and Alaska Native Tribal Health Consortium research review committees for their continued review of research and dissemination at the Alaska Native Medical Center campus.

FUNDING INFORMATION

This work was supported by the National Institutes of Health (S06 GM106351, P01 GM11669, U26 1IHS0079, S06 GM123545). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Footnotes

CONFLICT OF INTEREST

The authors declare that they have no conflicts of interest.

Contributor Information

Dawson E. Mills, Research Department at Southcentral Foundation in Anchorage, AK..

Krista R. Schaefer, Research Department at Southcentral Foundation in Anchorage, AK..

Julie A. Beans, Research Department at Southcentral Foundation in Anchorage, AK..

Michael R. Todd, Research Department at Southcentral Foundation in Anchorage, AK..

Renee F. Robinson, College of Pharmacy at Idaho State University in Pocatello, ID..

Kenneth E. Thummel, College of Pharmacy at the University of Washington in Seattle, WA..

Denise A. Dillard, Research Department at Southcentral Foundation in Anchorage, AK..

Jaedon P. Avey, Data Service Department at Southcentral Foundation in Anchorage, AK..

REFERENCES

  1. Beans JA, Hiratsuka VY, Shane AL, Day GE, Redwood DG, Flanagan CA, Wilson AS, Howard BV, Umans JG, & Koller KR (2019). Follow-up study methods for a longitudinal cohort of Alaska Native and American Indian People living within urban south central Alaska: The EARTH Study. Journal of Community Health, 44(5), 903–911. 10.1007/s10900-019-00630-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Beans JA, Hiratsuka VY, Apok CR, Caindec K, Dillard DA, & Robinson RF (2018). Community dissemination in a tribal health setting: A pharmacogenetics case study. American Indian and Alaska Native Mental Health Research, 25(1), 80–94. 10.5820/aian.2501.2018.80 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Borland R, Partos TR, Yong H-H, Cummings KM, & Hyland A (2011). How much unsuccessful quitting activity is going on among adult smokers? Data from the International Tobacco Control Four Country cohort survey. Addiction, 107(1), 673–682. 10.1111/j.1360-0443.2011.03685.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Chaiton M, Diemert L, Cohen JE, Bondy SJ, Selby P, Philipneri A, & Schwartz R (2016). Estimating the number of quit attempts it takes to quit smoking successfully in a longitudinal cohort of smokers. BMJ Open, 6(e011045). 10.1136/bmjopen-2016-011045 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Clark MJ (2012). Cross-cultural research: Challenge and competence. International Journal of Nursing Practice, 18(2), 28–37. 10.1111/j.1440-172x.2012.02026.x [DOI] [PubMed] [Google Scholar]
  6. Daley CM, Daley SM, Pacheco CM, Smith TE, Talawyma M, McCloskey C, Choi WS, Nazir N, Filippi MK, McKinney D, Gunville J, & Greiner KA (2018). Feasibility of implementing the All Nations Breath of Life culturally tailored smoking cessation program for American Indians in multi-tribal urban communities. Nicotine & Tobacco Research, 20(5), 552–560. 10.1093/ntr/ntx030 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dillard DA, Caindec K, Dirks LG, & Hiratsuka VY (2018). Challenges in engaging and disseminating health research results among Alaska Native and American Indian people in southcentral Alaska. American Indian and Alaska Native Mental Health Research, 25(1), 3–18. 10.5820/aian.2501.2018.3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Dilley JA, Patterson E, Hiratsuka VY, & Rhode K (2013). Discovering unique tobacco use patterns among Alaska Native people. International Journal of Circumpolar Health, 72(1). 10.3402/ijch.v72i0.21208 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Fenn DC, Beiergrohslein M, & Ambrosia J (2007). Southcentral Foundation Tobacco Cessation Initiative. International Journal of Circumpolar Health, 66(Suppl 1), 23–28. https://pubmed.ncbi.nlm.nih.gov/18154229/ [PubMed] [Google Scholar]
  10. Goodman S (2008). A dirty dozen: Twelve P-value misconceptions. Seminars in Hematology, 45(3), 135–140. 10.1053/j.seminhematol.2008.04.003 [DOI] [PubMed] [Google Scholar]
  11. Gottlieb K (2013). The Nuka System of Care: Improving health through ownership and relationships. International Journal of Circumpolar Health, 72(1), 93–98. 10.3402/ijch.v72i0.21118 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hensel MR, Cavanagh T, Lanier AP, Gleason T, Bouwens B, Tanttila H, Reimer A, Dinwiddie RL, & Hayes JC (1995). Quit rates at one year follow-up of Alaska Native Medical Center Tobacco Cessation Program. Alaska Medicine, 37(2), 43–47. https://pubmed.ncbi.nlm.nih.gov/7661325/ [PubMed] [Google Scholar]
  13. Hiratsuka VY, Beans JA, Robinson RF, Shaw JL, Sylvester I, & Dillard DA (2017). Self-determination in health research: An Alaska Native example of Tribal ownership and research regulation. International Journal of Environmental Research and Public Health, 74(11), 1324. 10.3390/ijerph14111324 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Hirchak KA, Leickly E, Herron J, Shaw J, Skalisky J, Dirks LG, Avey JP, McPherson S, Nepom J, Donovan D, Buchwald D, McDonell MG, & the HONOR Study Team. (2018). Focus groups to increase the cultural acceptability of a contingency management intervention for American Indian and Alaska Native communities. The Journal of Substance Abuse Treatment, 90, 57–63. 10.1016/j.jsat.2018.04.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kealey KA, Ludman EJ, Mann SL, Marek PM, Phares MM, Riggs KR, & Peterson AV (2007). Overcoming barriers to recruitment and retention in adolescent smoking cessation. Nicotine & Tobacco Research, 9(2), 257–270. 10.1080/14622200601080315 [DOI] [PubMed] [Google Scholar]
  16. Kennedy BR, Mathis CC, & Woods AK (2007). African Americans and their distrust of the health care system: Healthcare for diverse populations. The Journal of Cultural Diversity, 14(2), 56–60. https://pubmed.ncbi.nlm.nih.gov/19175244/ [PubMed] [Google Scholar]
  17. Kotz D, Brown J, & West R (2014). Prospective cohort study of the effectiveness of smoking cessation treatments used in the “real world”. Mayo Clinic Proceedings, 89(10), 1360–1367. 10.1016/j.mayocp.2014.07.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Nicholson LM, Schwirian PM, & Groner JA (2015). Recruitment and retention strategies in clinical studies with low-income and minority populations: Progress from 2004-2014. Contemporary Clinical Trials, 45(Pt A), 34–40. 10.1016/j.cct.2015.07.008 [DOI] [PubMed] [Google Scholar]
  19. Odierna DH, & Schmidt LA (2009). The effects of failing to include hard-to-reach respondents in longitudinal surveys. American Journal of Public Health, 99(8), 1515–1521. 10.2105/ajph.2007.111138 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. R Core Team. (2020). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.r-project.org/ [Google Scholar]
  21. Redwood D, Leston J, Asay E, Ferucci E, Etzel R, & Lanier AP (2011). Strategies for successful retention of Alaska Native and American Indian study participants. The Journal of Primary Prevention, 32(1), 43–52. 10.1007/s10935-010-0209-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Renner CC, Enoch C, Patten CA, Ebbert JO, Hurt RD, Moyer TP, & Provost EM (2005). Iqmik: A form of smokeless tobacco used among Alaska Natives. American Journal of Health Behavior, 29(6), 588–594. 10.5993/ajhb.29.6.13 3 [DOI] [PubMed] [Google Scholar]
  23. Robinson KA, Dennison CR, Wayman DM, Pronovost PJ, & Needham DM (2007). Systematic review identifies number of strategies important for retaining study participants. Journal of Clinical Epidemiology, 60(8), 757.e1–765.e19. 10.1016/j.jclinepi.2006.11.023 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Robinson KA, Dinglas VD, Sukrithan V, Yalamanchilli R, Mendez-Tellez PA, Dennison-Himmelfarb C, & Needham DM (2015). Updated systematic review identifies substantial number of retention strategies: Using more strategies retains more study participants. Journal of Clinical Epidemiology, 68(12), 1481–1487. 10.1016/j.jclinepi.2015.04.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Singh P, Ens T, Hayden KA, Sinclair S, LeBlanc P, Chohan M, & King-Shier KM (2017). Retention of ethnic participants in longitudinal studies. Journal of Immigrant and Minority Health, 20, 1011–1024. 10.1007/s10903-017-0618-0 [DOI] [PubMed] [Google Scholar]
  26. Smith LJ, McNamara PJ, & King AC (2017). Optimizing follow-up and study retention in the 21st century: Advances from the front line in alcohol and tobacco research. Drug and Alcohol Dependence, 175, 171–178. 10.1016/j.drugalcdep.2017.01.045 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Teague S, Youssef GJ, Macdonald JA, Sciberras E, Shatte A, Fuller-Tyszkiewicz M, Greenwood C, McIntosh J, Olsson CA, Hutchinson D, & the SEED Lifecourses Science Theme. (2018). Retention strategies in longitudinal cohort studies: A systematic review and meta-analysis. BMC Medical Research Methodology, 18(1), 1–22. 10.1186/s12874-018-0586-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Young AF, Powers JR, & Bell SL (2006). Attrition in longitudinal studies: Who do you lose? Australian and New Zealand Journal of Public Health, 30(4), 353–361. 10.1111/j.1467-842x.2006.tb00849 [DOI] [PubMed] [Google Scholar]

RESOURCES