Multi-source recruitment strategies for advancing addiction recovery research beyond treated samples

Abstract

Background

The lack of established sampling frames makes reaching individuals in recovery from substance problems difficult. Although general population studies are most generalizable, the low prevalence of individuals in recovery makes this strategy costly and inefficient. Though more efficient, treatment samples are biased.

Aims

To describe multi-source recruitment for capturing participants from heterogeneous pathways to recovery; assess which sources produced the most respondents within subgroups; and compare treatment and non-treatment samples to address generalizability.

Results

Family/friends, Craigslist, social media and non-12-step groups produced the most respondents from hard-to-reach groups, such as racial minorities and treatment-naïve individuals. Recovery organizations yielded twice as many African-Americans and more rural dwellers, while social media yielded twice as many young people than other sources. Treatment samples had proportionally fewer females and older individuals compared to non-treated samples.

Conclusions

Future research on recovery should utilize previously neglected recruiting strategies to maximize the representativeness of samples.

1. Introduction

Substance abuse policy and services are increasingly embracing a recovery focus, which acknowledges the need for ongoing supports for recovery (Laudet & Humphreys, 2013). However, we currently lack an empirical understanding of persons in recovery to adequately inform and evaluate recovery-oriented efforts. A first and challenging step towards building the science of recovery is to identify strategies to engage a population that is, for many reasons—e.g., stigma or other barriers to professional services— often “hidden” though in plain sight. Here we describe a multi-source recruiting strategy designed to maximize the generalizability of participants in research on addiction recovery.

For optimal generalizability of any sample, proper sampling frames must be used. Generally speaking, the ideal sampling frame would provide access to every single individual within a specified target population (Kalton, 1983); for studies of recovery from substance use disorders (SUDs), the ideal sampling frame would include every individual in recovery without duplicates and exclude all individuals who are not in recovery. Since such a comprehensive list does not exist, researchers must rely on individual sampling frames that yield slices of the recovering community. For example, treatment alumni and recovery organization lists are convenient and efficient ways to reach individuals in recovery who have registered with the groups for events and newsletters. Longitudinal treatment studies offer another slice of the population, although reliance on treatment samples for understanding recovery limits the external validity of findings since most treatment studies use multiple strict eligibility and exclusion criteria that can lead to biased conclusions (Humphreys, Harris, & Weingardt, 2008). Since different studies focus on distinct research questions and may require specific types of individuals (based, say, on drug of choice), the resultant samples inherently vary; thus, no single treatment study can be representative of the entire population of treatment-seekers. Finally, the majority of people who recover from SUDs do so without specialized professional treatment: the National Longitudinal Alcohol Epidemiologic Survey (NLAES) showed that only 28% of individuals with a past diagnosis of alcohol dependence reported ever receiving any kind of alcohol treatment (Russell et al., 2001). Results from the 2001–2002 National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) were similar, with 26% of those with prior-to-past-year alcohol dependence (Dawson et al., 2006) and 20% of those with past-year drug dependence having sought treatment (Compton, Thomas, Stinson, & Grant, 2007). In addition, lower rates of psychiatric comorbidity have been reported among treatment-naïve individuals with SUD compared to those who have received treatment (Compton, Thomas, Stinson, & Grant, 2007; Di Sclafani, Finn, & Fein, 2008; Hasin, Stinson, Ogburn, & Grant, 2007; Kessler et al., 1996; Wang et al., 2005b). Clinical samples often yield lower recovery rates than general population samples because they consist of more severely dependent individuals and exclude those who can recover without treatment (Dawson, et al., 2006).

A third approach to reaching individuals in recovery is the general population survey which draws a probability sample from which individuals in recovery can be identified. General population studies such as NLAES and NESARC are considered the gold standard for generalizable samples. However, the low prevalence of individuals in recovery makes general population surveys prohibitively costly and inefficient for obtaining a large sample of recovering individuals. For example, only 4.9% of the individuals interviewed for NESARC's Wave 1 survey, were in remission from alcohol dependence; thus obtaining a large sample of individuals in recovery through general population sampling techniques would require screening at least 20 people for every one eligible participant.

We know of no large, national study that has successfully and efficiently recruited individuals across treated, treatment-naïve, and informally treated (i.e., those who have attended self-help) groups. The smaller studies that do exist are limited to a single self-help group and/or are regionally circumscribed. Consequently, our knowledge about recovery processes and outcomes does not necessarily represent the broad yet un-quantified and under-investigated recovery population. This knowledge gap has hindered the development of recovery-oriented services and supports (Laudet, 2008, Laudet and Humphreys, 2013).

Study objectives

The current study was conducted in the context of a national project, the What Is Recovery? study, which sought to develop a psychometrically sound Recovery Definition reflecting the heterogeneity of recovery experiences associated with different pathways to recovery (e.g., treatment, 12-step, pharmacotherapy, natural recovery). Here, we examine the feasibility of recruiting strategies that may contribute to engage in research, individuals in SUD recovery who are typically under-represented in SUD research (e.g., untreated individuals, individuals not participating in 12-step groups).

The three overarching goals of this study are 1) to describe multi-source recruitment sources for reaching a large, diverse sample of persons in recovery from SUD, 2) to determine demographic differences across recruitment sources to assess which sources yielded the most respondents from traditionally hard-to-reach subgroups (e.g., racial minorities), and 3) to compare the demographics across several treatment and non-treatment study samples to understand what subgroups are missed with single-source recruiting methods.

2. Materials and Methods

Recruitment methodology

To maximize our reach to various segments of the recovery community with regard to race/ethnicity, other sociodemographics, and recovery path (treated and non-treated; 12-step and non-12-step mutual aid participation; medication-assisted; etc.), we worked in collaboration with grassroots organizations, treatment programs. Research partner organizations were recruited into the project several months prior to survey dissemination and asked to help with outreach in three key ways: 1) posting announcements about the study on their websites, 2) posting the study link on their websites, and 3) disseminating emails voicing their support for the study with a link to the survey to their constituents. Depending on technological savvy, partners also tweeted about the study, posted information on their Facebook pages, “liked” our study's Facebook page, and made announcements at events, conferences and meetings.

Many of our partners could be categorized as recovery organizations (involved in advocacy, education, prevention, and referrals and included governmental units and non-governmental organizations) or treatment and alumni organizations (whether 12-step based, non-12-step based, medically assisted or specifically targeted groups such as Christian recovery or minority focused). Partnering organizations included the National Council on Alcoholism and Drug Dependence, the National Alliance for Medication Assisted Recovery, Harm Reduction for Alcohol, dozens of 12-step treatment and sober living alumni groups, several state and regional departments of behavioral health across the country, Faces & Voices of Recovery, and popular non-12-step mutual aid groups such as Women For Sobriety, LifeRing, and SOS (Secular Organizations for Sobriety). The full list of partnering organizations is posted at www.whatisrecovery.org.

We also collaborated with informal partners that included physician groups, addictions magazines, websites and radio programs, friends of the study team, the study's expert panel of physicians and researchers, and other personal contacts. We sent emails to all partners several times before and during the three-month recruitment period, updating about recruitment progress and asking them to again please reach out to their constituents and audiences. Because so many formal and informal partners contacted other organizations and individuals that also posted notices about the survey, we cannot specify the actual number of recruitment sources or potentially eligible individuals who were exposed to our study link or informational materials. Furthermore, most of the WIR study team had added signature lines on their emails that referenced the survey, and many people who saw information about the survey and passed it along to others.

Over 155 treatment-oriented organizations and individuals were emailed with offers to use WIR-prepared recruitment materials (e.g., emails and fliers) for spreading the word. One of our partners, CRC Health Services, has 133 clinics and material was re-written and sent to each clinic for its particular clientele. Links to the study website were posted on many of these websites, not just our partnering organizations. Many sites requested posters with tear-strips providing the URL, announcements for staff meetings, blog-posts and newsletter items. In addition to recruiting through these partner organizations, we also posted ads with links to the survey on Craigslist. Thirty-three cities were selected for posting in 4 categories: Community Groups, Community Local News, Community Volunteers and Jobs, Etc.

Individuals in Alcoholics Anonymous (AA) who were familiar with the study team chose to hand out cards at meetings, picnics, after meeting announcements, AA retreats, symposiums, and clearinghouses. Because of the strict 12-step traditions against aligning with any organization or research or cause (e.g., the 6^th Tradition), individuals were able to post information in some central offices on a case-to-case basis. After discussing the issue with two central office managers, we mailed information to 300 Alano Club addresses and half of the AA Central Directory of state offices of 400 locations. The remaining AA offices were contacted via email.

Finally, we employed a variety of ancillary communication strategies that fall into our “Other” recruitment category: electronic media (e.g., blogs, webcasts, magazine articles); print media (e.g., the San Francisco Chronicle ran a story on study in its weekly Health section); and television and radio (e.g., a 30-minute interview on the Hispanic Univision channel in Northern California, public service announcements on a Christian radio program in the South, and radio interviews throughout the recruitment period). We distributed 15,000 cards printed with the website and basic information about the survey during Recovery Month events (September), workshops, conferences, and in front of theaters playing the movie Bill W. We had a strong presence at National Recovery Month walks in Washington DC, Detroit, Philadelphia, Las Vegas, Charlotte and at regional walks such as Contra Costa County (CA) and in Vermont.

Analyses

Demographic comparisons across recruitment sources

In addition to describing the sample, we compared demographics (e.g., sex, age, education, and race/ethnicity) and recovery characteristics (e.g., attended treatment, non-abstainer) across recruitment source subgroups. We then ranked the 17 recruitment sources according to their respective respondent yield within specific subgroups (e.g., non-treated, minorities). Rankings were determined by calculating frequencies of respondents from each subgroup within each recruitment source, and then ordering sources according to which yielded the most (1) to least (17) respondents from each subgroup.

Demographic comparisons across recovery samples

We then compared the demographics of the WIR sample to four other samples of individuals who consider themselves in recovery or as having had a problem with alcohol or drugs. The first comparison sample was collected by the New York State Office of Alcoholism and Substance Abuse Services (OASAS), who conducted a nationally representative random-digit-dial survey in 2011 of 2,526 adults (ages 18 and older). Both unlisted and listed landline telephone numbers were sampled. The data were weighted to US Census data for age, gender, geographic region, education and race. Our analyses focused on those who answered, “yes” to the OASAS question, “Did you used to have a problem with drugs or alcohol, but no longer do?” (N =239, or 10% of the original OASAS sample). Demographic frequencies for this group came from the OASAS group (Feliz, 2013).

The second comparison sample came from the 2010 National Alcohol Survey (NAS N12). The NAS N12 is a multistage-area probability sample of 7,969 individuals aged 18 and older in households within all 50 states and DC. Blacks and Hispanics were over-sampled. We focused on those in NAS N12 who answered, “yes” to the question, “Whether or not you have ever talked to anyone or whether you have ever been treated, do you consider yourself to be in recovery from an alcohol problem?” (N = 462, or 6% of the original NAS N12 sample). Demographic frequencies for this group were run in Stata (v11) using data obtained from the Alcohol Research Group; data were weighted to the US census age-by-gender distributions.

The third comparison sample comes from Faces and Voices of Recovery's Life in Recovery (LIR) survey (Hill & Laudet, 2013; Laudet 2013). The brief anonymous online survey conducted between November 1 and December 31, 2012, was a first step in quantifying the costs of addictions and the benefits of recovery to individuals and to the nation's health and economy. The LIR survey link was disseminated via Faces & Voices' website and social media sites, and shared (e.g., re-“tweeted”) by interested parties. To maximize the representation of Spanish-speaking individuals, who are typically underrepresented in research, the Life in Recovery Survey was translated into Spanish and administered in Spanish through a separate web link. A total of 3,208 surveys were completed in English and 10 in Spanish.

The final comparison sample is the US general population (age 18 years and over only). We used 2010 Census data for sex, age, and race/ethnicity, and 2009 Current Population Survey Data on Educational Attainment for education. Demographic frequencies for this group were either directly from Census data or required minor calculations using Census tables.

Demographic comparisons across treatment samples

As a final step in understanding differences between treated and untreated samples, we compared those within WIR who had sought/received formal treatment (N=6,649) to three other treatment samples. The first comes from the COMBINE Study, a large (N=1,383), multi-site NIAAA-funded randomized control trial of medications and behavioral treatments for alcohol dependence. The second comes from Project MATCH study sample, a large (N=1,726), multi-site trial of behavioral interventions. The third treatment comparison sample consisted of those who answered yes to the question “Have you sought formal treatment for alcohol dependence?” in the 2010 National Alcohol Survey (N=143).

3. Results

Sample descriptives

Overall, the WIR sample was diverse in gender (54% female), age (range: 18 to >65), education (12% high school only, 39% some college or vocational school, 50% college graduates), and residential setting (36% urban, 44% suburban, 20% rural dwellers- Table 1, first column). Most respondents were White, 8% were African American, 5% Hispanic, and 2% Native American. Almost all (98%) had met criteria for lifetime substance use dependence disorder as defined by the Mini International Neuropsychiatric Interview (MINI) (Lecrubier et al., 1997; Sheehan et al., 1997); recovery duration ranged from <1 to >20 years. More than half (56%) had been in recovery for more than 5 years, and 59% of the sample reported that alcohol had been their primary problem. The majority (87%) considered total abstinence from all substance as a part of recovery. In terms of recovery paths, 71% had received addiction treatment, 95% had attended (or are currently attending) mutual help group (e.g., 12-step) meetings; 70% had gone to treatment and mutual help group meetings, 25% had gone to mutual help group meetings only, 1% had gone to treatment only, and 4% reported neither. All 50 US states were represented (not shown).

Table 1. WIR sample descriptives: Overall and by recruitment source.

	WIR Overall (N=9.341) %	Recovery Organizations (N=1,113) %	Social Media (N=2,154) %	Treatment alumni/Recovery services (N=1,084) %	Word of mouth (N=1,473) %	Self-help groups (N=1,109) %	Other Sources (N=2,168) %
Gender
Female	54	52	58	46	54	62	53
Race
African American	8	13	8	8	7	3	8
White	88	83	85	90	91	95	89
Other	4	4	7	2	2	2	3
Ethnicity
Hispanic	5	4	8	4	3	3	8
Age
<35	18	12	34	16	14	13	13
36-50	32	33	38	32	29	30	31
>50	49	54	28	52	57	58	56
Education
No college	11	12	16	11	8	9	9
Some college	38	35	49	30	35	34	38
College grad or more	51	53	36	59	57	57	53
Residential Setting
Urban	36	36	45	33	34	28	35
Suburban	44	41	40	47	46	52	41
Rural	20	22	16	20	20	20	24
Primary Substance
Alcohol Only	59	53	47	66	63	79	58
One Drug Only	36	43	47	32	32	18	36
Polydrug	5	5	6	2	5	3	6
Length of Recovery
<1 year	15	10	20	19	6	25	12
1-2 years	10	7	15	12	5	14	8
2-5 years	19	16	22	23	16	21	16
> 5 years	56	68	43	46	73	40	64
Recovery Path
Neither treatment nor 12-step	5	3	9	1	1	9	4
12-Step only	24	22	21	8	33	29	25
Treatment only	2	1	2	2	1	3	1
Treatment + 12-step	70	74	68	89	65	59	70
Recovery Definition
No use of any substance	86	90	77	90	93	84	88
Options other than abstinence	14	10	23	10	7	16	12

Comparisons across recruitment sources

Respondents could select one of 16 sources provided when asked about how they heard about the study (Table 2); an “Other” category, which captures blogs, webcasts, magazine articles, TV and radio, was provided as well. Together, the five largest yielding sources produced almost two-thirds of the entire sample: Other (24%), family and friends (15%), Craigslist (12%), social media (7%) and non-12-step self-help groups (7%).

Table 2. Percentage of WIR sample from each recruitment source.

Recruitment Source	Individual Source %
Other	24
Social/Electronic Media
Craigslist	12
Social Media	7
Advertisements	4
Word of Mouth
Family & Friends	15
Conferences	1
Recovery Organizations
Faces & Voices	6
Recovery Community Organizations	5
National Assn for Medicated Assisted	<1
White Bison	<1
NorthStar Community Recovery	<1
Self-Help Groups
Non 12-Step Groups	7
12-Step Groups	5
Alano Clubs	<1
Treatment & Alumni Groups
Treatment Facilities	6
Alumni Groups	5
Veteran's Administration	<1

Key differences in demographics and recovery characteristics were noted across recruitment sources (Table 1). Notably, media sources were more successful than other sources in reaching typically untapped recovery subgroups, i.e., those who are younger, less educated, and/or earlier in recovery; fewer of the media-recruited participants selected total abstinence as their recovery definition (vs. more moderate goals); media also yielded more persons in natural recovery (i.e., no exposure to either treatment or mutual aid/self-help groups). Recovery organizations (12%) yielded nearly twice as many African-Americans and more rural dwellers than other sources. Perhaps not surprisingly, social/electronic media (34%) yielded twice as many young people as did other sources.

Table 3 ranks the five recruitment sources that yielded the most respondents from traditionally hard-to-reach groups, such as racial minorities and treatment-naïve individuals. With some small variation, the five largest yielding recruitment sources (family and friends, CraigsList, social media, non-12-step self-help groups, Other sources) were the most important recruitment sources for minority respondents, non-abstainers who moderate their substance use but do not necessarily abstain, and natural recovery (i.e., no treatment or help-seeking) populations.

Table 3. Rankings of success of recruitment sources in recruiting hard-to-reach groups.

Group	Recruitment Source
	Other Sources	Family and friends	CraigsList	Social Media	Non-12-step self-help
Help-seeking & substance use
Never treated	2a	4	1	6	3
No 12-step exposure	1	4	3	7	2
Not abstaining (harm reducer)	2	4	1	5	3
Race
Hispanic	2	3	1	4	10
Asian	3	-	1	2	4
Black	1	3	2	9	8
Hawaiian/Pacific Islander	2/3	2/3	1	-	-

^aFrequencies of respondents from each Group were cross-tabbed with the Recruitment Source to produce rankings for sources that yield most (1) to least (5) respondents in each Group.

Demographic comparisons across recovery samples

The WIR and LIR samples were more than half female (54% and 57%, respectively), while the OASAS (37%) and NAS (41%) had fewer women. The WIR and LIR samples were somewhat racially diverse, but with more Native Americans and whites than the US general population. The general population OASAS and NAS samples had about the same proportion of Hispanics as the US general population. All four recovery samples contained proportionally more persons aged 35-65 (64-73%) than the US general population (52%). In summary, the Internet survey samples (WIR and LIR) matched very closely yet produced different demographic profiles than the non-Internet general population samples (OASAS and NAS). None of the four recovery samples matched the demographic breakdown of the general population.

Demographic comparisons across treated samples

Among those who had sought treatment, more than 50% of WIR participants were female, compared with about a third for the other samples (Table 4). The WIR and NAS treated samples were the oldest, while treatment samples COMBINE and MATCH had proportionally more individuals aged 21-35 and fewer aged 65 or over. The COMBINE, MATCH outpatient and NAS treated samples were slightly more than 10% Hispanic, twice the number in the WIR treated sample (5% Hispanic); however, the MATCH aftercare sample was only 3% Hispanic. Both the WIR and COMBINE samples were more than 40% college graduates. Other samples were all similarly educated, with about three-quarters or more having at least a high school education. The NAS sample was the most racially diverse, with a fifth African American and slightly more than half White; WIR, COMBINE, and MATCH had similar racial breakdowns to one another. Importantly, the treatment groups differed from one another; there are gender and age differences between those recruited in treatment (COMBINE, MATCH) and those reporting treatment in national surveys (WIR, NAS). Furthermore, COMBINE and MATCH, which both recruited directly from treatment sites, differed in terms of education.

Table 4. Demographic comparisons between What Is Recovery? treated subsample and other treated samples.

	WIR TREATED (N = 6,649) %	COMBINE (N=1,383) %	Project MATCH (N=1,726)		NAS TREATED a (N =143) %
			Outpatient (N=952) %	Aftercare (N=774) %
Sex
Female	52	31	28	20	36
Male	49	69	72	80	64
Age
18-20	<1	<1	1	<1	1
21-35	18	19	43	31	8
36-50	34	53	42	49	33
51-65	40	26	14	16	46
65+	8	2	1	4	11
Education
<HS	3	6	11	12	12
HS	9	23	34	37	32
Some college/voc school	41	30	32	34	32
College or more	47	41	24	17	24
Ethnicityb
Hispanic	5	11	12	3	16
Non-Hispanic	95	89	88	97	84
Raceb
Native American/Alaskan	2	1	2	1	6
Asian	1	<1	<1	0	--
Native Hawaiian/Pacific	<1	--	--	--	--
Islander	9	8	6	15	21
Black	88	76 (NH) c	80 (NH)	80 (NH)	57
White

^aSample comes from nationally representative survey (weighted to U.S. Census data for age, gender, geographic region, education and race), and consists of those who answered yes to both of the following questions: “Whether or not you have ever talked to anyone or whether you have ever been treated, do you consider yourself to be in recovery from an alcohol problem?” and “Did you ever receive treatment from a chemical dependency or substance abuse program for either alcohol or drugs?”

^bRace/ethnicity categories not consistent across samples

^cNH= Non-Hispanic

Treated vs. recovery samples

Compared to the recovery (non-treated) samples shown in Table 5, the treatment samples shown in Table 4 consistently had proportionally fewer females and fewer individuals age 65 years and over. While all of the recovery samples were about half female, all of the treated samples except WIR were 20-36% female. In terms of age groups, the COMBINE and MATCH samples were only 1-4% aged 65 or over, while the age breakdowns for the WIR (8% aged 65 or over) and NAS (11% aged 65 or over) treated samples more closely resembled those of the recovery samples (8-18%). Comparisons across education, race, and ethnicity categories were less consistent because the proportions of individuals within each category varied considerably across both the treated and recovery samples.

Table 5. Comparison of the What Is Recovery? sample with other recovery samples.

	WIR: OVERALL (N = 9341) %	OASAS (N = 239)a %	NAS (N = 462)b %	Life in Recovery (N=3228)	US GEN POPc %
Sex
Female	54	37	41	57	51
Male	46	63	59	43	49
Aged
18-34/35	18	25	12	15	31
34/35-64/65	73	64	70	79	52
65/66+	9	10	18	7	17
Education
<HS	3	30	18	2	14
HS	8	37	33	8	32
Some college	31	21	26	28	20
Vocational school	8	--	--	8	4
College or more	50	11	23	56	29
Ethnicityd
Hispanic	5	15	20	5	17
Non-Hispanic	95	85	80	95	83
Raced
Native	2	--	--	10	<1
American/Alaskan	1	--	1	0.4	5
Asian	<1	--	--	0.3	<1
Native
Hawaiian/Pacific	8	12 (NH)e	22 (NH)	8	12
Islander	88	73 (NH)	55 (NH)	82	81
Black
White

^aSample comes from nationally representative survey (weighted to U.S. Census data for age, gender, geographic region, education and race), and consists of those who answered yes to “Did you used to have a problem with drugs or alcohol, but no longer do?”

^bSample comes from nationally representative survey (weighted to U.S. Census data for age, gender, geographic region, education and race), and consists of those who answered yes to “Whether or not you have ever talked to anyone or whether you have ever been treated, do you consider yourself to be in recovery from an alcohol problem?”

^c2010 Census data for residents18 years and older; 24.4% of US general population was age 0-17 in 2010

^dAge, race/ethnicity categories not consistent across samples

^eNH= Non-Hispanic

4. Discussion

The need for research among persons in recovery is growing as federal agencies, services and policy dealing with substance use disorders are increasingly shifting to a recovery-oriented paradigm (Laudet & Humphreys, 2013). Findings suggest that the broad outreach strategy used in the WIR study yielded a diverse sample of individuals in terms of both individual characteristics and recovery experiences. Media recruiting, which does not rely on a specific recovery path (e.g., formal treatment or 12-step membership), appeared particularly effective at engaging traditionally untapped segments of the recovery community in research, such as non-abstainers and minorities.

In contrast, treatment studies usually do not employ broad recruiting strategies, thus limiting the generalizability of their results. For example, we found that treatment samples (e.g., COMBINE, MATCH) may under-represent women and individuals aged 65 or over. In terms of our comparison between the WIR and other recovery samples on the one hand, and treated samples on the other hand, we did expect demographic differences: WIR, NAS, and LIR come from large national surveys, while the treatment samples are more geographically constrained. Most notably, WIR demographics were almost identical to those of the LIR Internet survey sample; although we have no way of knowing whether either of these samples could represent the recovery population as a whole, the consistency between the WIR and LIR samples suggests that Internet-based recruiting is reliable.

Importantly, the online population does not represent the US general population because of upward bias in socioeconomic status among Internet users. Similarly, not all racial/ethnic groups access the Internet at similar rates, which may preclude the use of web-based surveys for research on populations that are not well-represented online (Sue & Ritter, 2012). As with our survey, the Internet-based LIR survey reached fewer Hispanics and individuals with lower education levels. A study of web-based respondent driven sampling in young adults similarly yielded fewer individuals with lower education or who self-reported as minority race (Bauermeister et al., 2012). Importantly, Bauermeister et al. (2012) did find comparable rates of drug and alcohol use to those found in NSDUH. The authors concluded that online research is a growing trend, and that researchers must identify suitable strategies to reach these under-represented subpopulations when conducting web-based data collection.

The NAS and OASAS employed population-based sampling and telephone interviews, which may yield a sample more representative of the general population. However, population-based approaches of studying natural remitters have been shown to include higher proportions of less severe problem drinkers than media-solicited samples (Rumpf, Bischof, Hapke, Meyer, & John, 2000). A very high percentage of those in natural recovery in our WIR sample met DSM-V “severe” criteria (90%), a bit lower than for the sample overall (98%) (Witbrodt, forthcoming). The pros and cons of Internet- vs. population-based approaches highlight the importance of using multiple recruitment strategies when sampling persons in recovery.

Somewhat unexpectedly, demographics across treatment samples differed in some respects. For example, there were more women in WIR than in both other treatment and other recovery samples (except NAS, which recruited from the general population rather than treatment settings). Therefore findings imply that results from any one treatment study are not necessarily generalizable to all treatment populations. Comparisons across treatment studies should be interpreted cautiously.

Limitations

Two inconsistencies in data collection across samples warrant explanation. First, OASAS and N12 did not ask about vocational school, which make the education categories slightly less comparable. Similarly, we do not know how many people from each sample (other than WIR) have post-college graduate degrees; 25% of WIR respondents had graduate degrees, suggesting that the WIR sample may be more highly educated than other recovery samples. However, 41% of the COMBINE sample had at least a college degree, which is similar to the 47% seen in the WIR sample. Future analyses of the recovery scale will examine whether the definition of recovery varies according to holding a graduate degree. The second inconsistency regards race categories: OASAS and N12 included “Hispanic” (considered ethnicity, not race, in WIR and the US Census) but were missing other specific race categories. Race categories across samples may not be comparable because, for example, the 88% of Whites from the WIR sample included Hispanic Whites, while the 65% of Whites in OASAS does not include Hispanic Whites; thus, racial breakdowns may be more similar across samples than they seem, because the OASAS and NAS “White” category are likely underestimated due to exclusion of Hispanics.

We were unable to calculate non-response rates in WIR because invitations to participate were open to any individual (18 years or older) who considers him/herself to be in recovery and the size of the population of eligible respondents is unknown. As in all studies, those who participate in research studies and surveys are often self-selected and may not represent the target population. Finally, only 5% of the WIR sample could be categorized as “naturally recovered,” i.e., never sought treatment or self-help for substance use. Our results show that media recruiting was most effective at reaching this group; future studies should consider other techniques for reaching naturally recovered individuals.

Lessons learned

Reaching a representative sample of individuals that reflects the heterogeneity of the population in recovery remains a challenge, but future efforts might benefit from our experience. For example, researchers who plan to use Craigslist should be aware that the site's rules constantly change, especially regarding payment. Other problems included having to change the wording in each posting so as not to be considered duplicates, postings disappearing from some sites, and not being able to rely upon the account management tools within Craigslist itself. Another valuable lesson is the necessity of using different wording about the targeted audience in all recruitment efforts (including Craigslist). Researchers cannot assume that the individuals they want to reach in a recovery study will consider themselves as being “in recovery”; other popular terms are “recovered”, “in medication-assisted recovery”, or “used to have a problem with alcohol and drugs.” Outreach must reflect this heterogeneity in self-definition, if we are to increasingly reach less biased, more representative samples.

We also learned that our research partners appreciated updates of the study's progress, and were willing to re-contact their constituents and run new stories about the project. These included announcing milestones, such as the number of respondents so far, or a “countdown” to the final days of the survey period, as well as problems, such as needing to reach more African Americans or Hispanics in recovery. We provided our study partners with a recruitment “toolkit” containing sample wordings for emails, websites and newsletters at the beginning of the study and as we approached these milestones or encountered specific challenges with recruitment progress.

Four additional techniques also may help yield a larger, more representative sample than obtained here. The first is to expand the data collection method to include telephone interviews and possibly paper-and-pencil surveys. The former can be done efficiently using an automated telephone survey system, but both approaches carry costs not required of web-based approaches. However, alternate data collection approaches could make the study accessible to those without Internet access, and may yield more minorities and more individuals in lower educational brackets. Secondly, a longer recruitment period, or perhaps a second round of recruitment after disseminating the current survey results, might generate sufficient word-of-mouth referrals to reach more ethnic minorities, more individuals in natural recovery, and individuals otherwise not reached by the study team's own recruitment efforts. Third, researchers could provide computers with Internet connections at places where people in recovery come together, e.g., recovery month events. Finally, translating the survey and recruitment materials into Spanish might help recruit more Hispanics.

Conclusion

The WIR study is the largest nationwide survey of persons in addiction recovery ever conducted. The multi-source recruiting strategy described here is unprecedented in its breadth; our sample shows great diversity in terms of both individual characteristics and recovery experiences, and highlights potential frameworks for recruiting outside of treatment sites. Findings support the notions that specific subgroups respond to various recruitment strategies differently, and that recovery-oriented research should include both treated and treatment-naïve individuals if results are to be generalized to the recovery population as a whole.