Qualitative insights in item development for a comprehensive and inclusive measure of recovery capital

Abstract

Background:

Item specification is foundational to measurement development but rarely reported in depth. We address this gap by explicating our use of qualitative methods to ground and develop items for a new recovery capital measure, the Multidimensional Inventory of Recovery Capital.

Method:

We recruited a diverse sample of service providers (n = 9) and people in recovery from alcohol problems (n = 23) to provide feedback on an item pool assessing social, human, physical, community, and cultural capital. Using applied qualitative analysis, we coded findings from interviews and focus groups and made final decisions by consensus regarding item elimination, retention, or revision. This process yielded a 49-item draft measure.

Results:

Only nine items from an initial 90-item list were retained in their original form. Participant feedback guided item elimination, addition, and revision for linguistic or conceptual clarity. We detected little systematic variation in feedback based on income or race; however, there were stark divergences on particular items based on recovery pathway (i.e. 12-step versus other approaches).

Conclusions:

The high degree of alteration to the item pool highlights the importance of establishing validity with respondents. Response variation based on recovery pathway suggests the need for broad heterogeneity in respondents. Measures that are sensitive, psychometrically sound, and aligned with theory are critical for advancing research on recovery capital and related disparities for diverse populations.

Introduction

Since its debut more than two decades ago, the concept of recovery capital (Granfield and Cloud 1999; Cloud and Granfield 2001) has captured the interest of addiction researchers, service providers, and grassroots recovery movements. Although researchers have stressed the importance of developing psychometrically sound and culturally inclusive measures of recovery capital, current measures are hampered by significant limitations (Hennessy 2017; Bowen et al. 2020). This paper reports results from the first phase of a study to develop a comprehensive, inclusive, and theory-based measure of recovery capital. Our aim is to explicate our use of qualitative methods to ground the development of measurement items in the experiences and realities of the measure’s target population, in order to avoid common pit-falls in item specification that ultimately hinder utility in research and practice (Gilgun 2004; LeCroy 2019).

Recovery capital: evolution of a theory

From earlier definitions focused on abstinence, the definition of recovery from substance use problems has evolved to include multidimensional elements such as well-being, optimism, and overall life improvements (Kaskutas et al. 2014; Kelly et al. 2018; National Institute on Alcohol Abuse and Alcoholism 2020). Relatedly, Cloud and Granfield (2001) initially proposed recovery capital as a theoretical framework to measure the resources outside of formal treatment that aid in one’s recovery across social, physical, human, and cultural domains (Cloud and Granfield 2001). Social capital captures supportive relationships and connections within social networks. Physical capital includes material resources, such as transportation, housing, and financial assets. Human capital consists of one’s personal qualities, skills, attitudes, and education and training. Finally, cultural capital is one’s beliefs, values, spirituality, and connections to the larger environment. Cloud and Granfield (2008) subsequently expanded recovery capital theory to include negative forms of capital in these same categories that deter recovery success. Since its initial conceptualization, researchers have theorized that recovery capital may act as a buffer to psychological stress, with higher levels of recovery capital resulting in corresponding stress reductions that promote sustained recovery (Kelly and Hoeppner 2015).

Research has examined recovery capital in a variety of populations and groups, including people experiencing homelessness (Neale and Stevenson 2015), formerly incarcerated individuals (Lyons and Lurigio 2010; Connolly and Granfield 2017), ethnic minorities (Pouille et al. 2020), young adults (Demant and Järvinen 2011), and adolescents (Hennessy et al. 2019; Nash et al. 2019). However, gaps remain in terms of the representativeness of populations included in recovery capital research samples. In a systematic review of 38 articles on recovery capital spanning from 1999 to 2016, Hennessy (2017) found that on average, study samples were 72.1% white and 60.9% male. Although some studies did include greater racial and gender diversity, Hennessy noted the need for research to address disparities in recovery capital and their structural drivers for marginalized populations.

Recovery capital is likely an underlying factor in persistent disparities in recovery outcomes. People with fewer economic resources and people of color encounter historical trauma and ongoing systemic racial and economic oppression that impact both vulnerability to substance use problems and treatment and recovery prospects (Collins et al. 2016; Zemore et al. 2018; Wagner and Baldwin 2020). Barriers such as limited access to housing, loans, and employment opportunities for formerly incarcerated people stifle the development of recovery capital and disproportionately impact people of color (Kelly et al. 2017). In addition, factors including identifying as cisfemale, transgender, non-binary, and/or lesbian, gay, bisexual, or queer are associated with poorer recovery outcomes and reduced access to recovery resources, reflecting the predominance of a cismale and heteronormative paradigm of recovery (Glaser 2014; Elm et al. 2016; Kelly et al. 2017; Mericle et al. 2020; Wagner and Baldwin 2020). There is also a prevailing need to improve understanding and measurement of cultural, community, and social dimensions of recovery across populations (Best and Ivers 2021).

Measuring recovery capital

Advancement of research on recovery capital, its role in recovery disparities, and the development of interventions to bolster positive capital and reduce negative capital sources benefits from the availability of measures that are grounded in theory and psychometrically reliable and valid for diverse populations (Hennessy 2017). Currently, the predominant recovery capital measure is the Assessment of Recovery Capital (ARC; Groshkova et al. 2013) and its condensed 10-item version, termed the Brief Assessment of Recovery Capital (BARC-10; Vilsaint et al. 2017). The full version of the ARC is a 50-item instrument covering ten domains, including substance use and sobriety, social support, and meaningful activities (Groshkova et al. 2013). The abbreviated BARC-10 contains one item derived from each domain (Vilsaint et al. 2017).

Though initial testing found each of these measures to have reasonable psychometric strength, more recent analyses have indicated theoretical and empirical limitations. The ARC/BARC-10 includes domains not articulated in the theoretical literature (Cloud and Granfield 2008), such as sobriety (interpreted as abstinence) as a component of recovery capital, and does not assess key components of the theory, including cultural capital and negative capital (Bowen et al. 2020). In addition, the ARC and BARC-10 were evaluated with predominantly white samples that were engaged in treatment or self-help groups. Although Hanauer et al. (2019) noted invariance across racial and ethnic groups, genders, and sexual orientations for a 10-item Short Recovery Capital Scale (SRCS) consisting of items intentionally paralleling the BARC-10, the study did not report on participants’ socioeconomic status and the sample consisted solely of participants in residential treatment.

The psychometric features of the ARC/BARC-10 and related measures such as the SCRS are largely unknown in populations that are diverse with regard to socioeconomic status, race and ethnicity, and recovery pathway (e.g. inclusive of those recovering outside of formal treatment systems). Recovery capital is not manifested uniformly among populations; therefore, measurement items that are demonstrated to be reliable and valid in primarily white and middle-class samples may not necessarily remain so in other groups. LeCroy (2019) notes that there are numerous dangers to the proliferation of measures that are not reliable, valid, or comprehensible to the populations responding to them, including the inability to detect effects in intervention research.

The development and availability of recovery capital measures that are psychometrically sound for varied groups is critical to the advancement of recovery science and practice and especially recovery intervention research. Given entrenched disparities, it is particularly important that any measure of recovery capital is reliable and valid for groups that are under-served and under-represented in prior research, including low-income individuals, people of color, women, sexual and gender minorities, and people not engaged in treatment. While measures such as the ARC, BARC-10, and SCRS have helped to advance understanding of recovery capital, the shortcomings of these instruments suggest the need for further development of inclusive and theory-aligned measures of recovery capital.

Current study

In this paper, we report results from the first phase of a study to develop a new and comprehensive measure of recovery capital, the Multidimensional Inventory of Recovery Capital (MIRC). Our primary aim is to describe the methodological processes and results for developing and refining measurement items. Although item specification is foundational in measurement development and validation, developers of new measures often neglect to adequately describe procedures for specifying constructs, generating items, and ensuring content validity (Holmbeck and Devine 2009). Within healthcare, the movement toward creating and implementing patient-reported outcome measures has reinforced the importance of grounding item generation in patient perspectives through the use of qualitative methods (U.S. Food and Drug Administration 2009; Pilkonis et al. 2013; McCarrier et al. 2016; Williams et al. 2019). Concurrently, policymakers and funders including NIAAA (2020) as well as addiction researchers have articulated the need for new measures that capture nuanced aspects of the recovery process (Hennessy 2017; Witkiewitz and Tucker 2020). However, very little prior research has explicated the use of qualitative methods to generate items for recovery measurement that reflect the viewpoints of primary stake-holders, such as service providers and people in recovery (Neale et al. 2014, 2016).

Particularly critical, though often overlooked, is eliciting input on potential items from varied members of a target population to attain cultural validity (Holmbeck and Devine 2009; LeCroy 2019). With regard to recovery, extant disparities underscore the urgency of ensuring that samples sufficiently include populations historically under-served in practice and under-represented in addiction and recovery research, including people of color, low-income individuals, and sexual and gender minorities (Collins et al. 2016; Zemore et al. 2018; Wagner and Baldwin 2020). Though evidence suggests that 12-step group attendance correlates with higher levels of recovery capital through the promotion of abstinence-focused social supports (Majer et al. 2021), recovery capital measures have not assessed differences by recovery pathway. Therefore, heterogeneity in recovery samples must also encompass variations in recovery pathway and be inclusive of individuals with varying levels of engagement (or non-engagement) with self-help groups and formal treatment systems (Subbaraman et al. 2015; Kelly et al. 2017). To address these gaps, this paper provides an in-depth view of our process of item development, with a focus on how we elicited and incorporated input from a heterogeneous sample of service providers and individuals in recovery from alcohol problems.

Methods

Initial item generation

To generate the initial item pool, the research team reviewed empirical research that described elements of recovery capital. We utilized Hennessy’s (2017) systematic review of recovery capital research and conducted a literature search to find additional relevant studies. Our review focused on studies that analyzed components or elements of recovery capital, and excluded studies on other aspects of recovery capital, such as those analyzing correlates of recovery capital as a whole. Through this search, we identified 15 relevant studies published between 2001 and 2019.

We used Cloud and Granfield (2008) theoretical framework to organize the empirical elements of recovery capital described in these studies. Studies identified a vast range of recovery capital elements, such as participation in recreational groups (Zschau et al. 2016), meaningful employment (Neale et al. 2014), and financial problems as a component of negative recovery capital (Duffy and Baldwin 2013). Next, we classified each element as positive or negative under the categories of social, physical, human, and cultural capital. Further discussion concerning cultural capital resulted in splitting this category into cultural (e.g. values, tastes, and traditions) and community (e.g. community-level resources) capital (White and Cloud 2008; Hennessy 2017). Study team members then wrote at least one item per element. The team discussed the items developed through this process and generated additional items, yielding a list of 90 original items.

Following these initial steps, nine recovery research experts who were not part of the research team reviewed and provided feedback on the list. Based on this feedback, the research team revised many of the items and added additional items suggested by the experts, resulting in an expanded set of 102 items. To reduce the number of items for review in focus groups, we grouped items that assessed a central construct and chose one item to represent the construct. For example, ‘I feel supported by my family’ represented the construct of family support, with more specific items assessing support from romantic partners and children retained in an item bank for further consideration (referred to as ‘banked items’ throughout the research process). Through this process, we identified 53 items to present to participants. Figure 1 summarizes changes made to the item list across stages of the item development process, including initial item generation, expert review, consolidation around central constructs, and vetting with service providers and people in recovery.

Figure 1.

Summary of item changes across steps of the item development process.

Item vetting with service providers and people in recovery

Eligibility and quota sampling criteria

To vet and refine the proposed items, we conducted focus groups and interviews with two types of participants: services providers and people in recovery from alcohol problems. Data collection took place from October 2020-March 2021. All study activities were carried out with the approval of the Institutional Review Board at the principal investigator’s university. Participants were age 18 or older and living in the United States. The eligibility criterion for service providers was working full or part-time in any role and in any aspect of the substance use treatment and recovery field. For people in recovery, the eligibility criterion consisted of identifying as having resolved a prior problem with alcohol (either alcohol alone or alcohol and other drugs) for at least 30 days. Similar criteria have been used in prior recovery studies (Kaskutas et al. 2014; Kelly et al. 2018). Our criterion centered on alcohol due to our funding source as well as recognition of the unique recovery challenges associated with alcohol as a legal and widely available substance.

To ensure economic and racial and ethnic diversity in the recovery sample, we formulated a quota sampling plan based on income and racial and ethnic identity. We used house-hold income under 200% of the federal poverty level (FPL) as the threshold for low-income status and asked participants to self-report how they defined their race and ethnicity, subsequently grouping responses as white or people of color. We defined ‘people of color’ to include identifying as Black or African American, Native American, Asian American, Latinx or Hispanic ethnicity, or more than one race or ethnicity. The plan specified recruiting five to eight people in recovery in each of the following categories: white and low-income; people of color and low-income; white and middle/high-income; people of color and middle/high-income. Although not part of the quota sampling criteria, we also monitored our recruitment for heterogeneity with regard to gender and sexual orientation. We did not use quota sampling for the service provider sample, given feasibility constraints (e.g. the likelihood of limited income variability among people employed in the same sector) and the choice to limit the provider sample size relative to the recovery sample size, in order to highlight the voices of people in recovery in the research.

Recruitment

We used a multisource recruitment strategy to recruit both service providers and people in recovery (Subbaraman et al. 2015). Our strategy relied on online communication, given the timing of the study during the COVID-19 pandemic. This included disseminating the study information via email to personal contacts and institutional listservs, posting study information on social media (Facebook, Instagram, Twitter), sharing information with staff at social service agencies, and informal snowball sampling through participants telling their contacts about the study. For the recovery sample only, we also utilized a university-based research registry, through which we notified community members who had indicated potential interest in participating in substance use-related research. Using these varied methods enabled us to recruit a heterogenous sample in accordance with our sampling plan (Subbaraman et al. 2015).

Data collection procedures

Potential participants were screened for eligibility via phone, text, or email, then scheduled a time to attend a focus group or individual interview via videoconferencing using Zoom. After completing consent procedures, participants filled out an online survey assessing demographics and gathering benchmark data on treatment history, prior problem severity (assessed with the lifetime version of the Short Inventory of Problems; Miller et al. 1995), and current alcohol and drug use (assessed using questions from the Alcohol Use Disorders Test and Drug Use Disorders Identification Test; Berman et al. 2005; Saunders et al. 1993).

Focus group versus individual interview format was determined based on participant preference and availability, ensuring respondents could participate in the format in which they felt most comfortable. Seven focus groups comprised of two to four participants and 13 individual interviews were conducted, using the same structured format. A facilitator from the study team displayed the potential items in a PowerPoint presentation. The facilitator read each item aloud and asked participants to share feedback, using probing questions such as, ‘Does this item reflect something that is important to your own recovery?’ ‘What don’t you like about this item?’ ‘Is anything in this item unclear or confusing?’ The facilitator also asked participants to describe other aspects of recovery capital that were missing and should be included on the measure. Focus groups and interviews took approximately 90 minutes (inclusive of all procedures) and were recorded and auto-transcribed in Zoom for analysis. Participants received $30 electronic gift cards as compensation for their time.

Data analysis and coding

We used the framework approach for applied qualitative analysis (Ritchie and Spencer 1994), which enables comparison both within and across cases. The research team created a deductive thematic framework that categorized responses from the five respondent groups (e.g. service providers and the four quota sampling categories of people in recovery) under each measurement item. Following an interview or focus group, a member of the research team reviewed the transcript and completed primary coding by summarizing the participants’ perspective and feedback under each item, using verbatim quotes and paraphrasing.

The first and second authors periodically reviewed the primary coding, writing memos to note emerging themes and questions. Simultaneous with data collection, the first and second authors engaged in secondary coding by reviewing the primary codes and memos to draft recommendations regarding if a particular item should be eliminated, replaced with a banked item, retained as written, or revised. The authors continued to revisit these draft recommendations as additional interviews and focus groups and corresponding primary coding were completed. As the last stage, the first and second author met with two study co-investigators and a research consultant, whose role was to provide expertise and guidance on recruitment and data collection, measurement development processes, and interpretation of results. The co-investigators and consultant are senior researchers with several decades of experience in recovery research. All members of the team reviewed the primary and secondary coding and memos and made final decisions by consensus regarding item elimination, retention, and revision. This process yielded the first version of the MIRC with 49 items, which was further evaluated through cognitive interviewing and psychometric pilot testing (not described in this paper).

Results

Sample description

The sample consisted of 32 participants, including nine service providers and 23 people in recovery. Although there was overlap between the two subsamples—with some providers disclosing a recovery history and some people in recovery working in the field—for the analysis we enumerated participants according to the subgroup in which they were recruited and screened. The sample was majority cis-gender female (58.1%). Around one quarter of respondents (25.8%) identified their sexual orientation as lesbian, gay, bisexual, or ‘other.’ People of color comprised 43.4% of the recovery sample and 38.7% of the total sample. Low-income individuals were nearly half (47.8%) of the recovery sample and 35.5% of the total sample. A majority of the recovery sample (60.9%) identified as having resolved their alcohol problems for two years or more, while 13% had resolved their problems for between 1 and 2 years. About one quarter of the sample (26.1%) indicated they had been in recovery for less than one year. Many participants reported ever having had problems with other drugs in addition to alcohol including marijuana (39.1%), cocaine, crack, or meth amphetamine (34.8%), heroin (13%), prescription opioids (21.7%), other prescription medications (13.0%), or other drugs not specified (8.7%).

There was also substantial variation in pathways of recovery. In the recovery subsample, 34.8% of participants had ever attended inpatient treatment for alcohol problems and 52.2% had ever participated in outpatient treatment. Just under half (47.8%) had attended self-help groups (in-person or online) in the past 30 days, with 21.7% having attended previously and 30.4% never having attended. Around one-fifth of recovering participants (21.7%) had attended both inpatient and outpatient treatment in addition to self-help groups. In contrast, 17.4% had no experience with inpatient or outpatient treatment or self-help groups. Finally, a substantial minority of recovering participants used alcohol occasionally (43.5%) and the same percentage used other drugs (Table 1).

Table 1.

Description of the sample (N = 31^*).

	Service Providers	People in Recovery	Total Sample
	(n = 8)	(n = 23)	(N = 31)
Characteristic	n (%)	n (%)	n (%)
Gender
Cisgender female	5 (62.5%)	13 (56.5%)	18 (58.1%)
Cisgender male	3 (37.5%)	8 (34.8%)	11 (35.5%)
Other gender identities	0	2 (8.7%)	2 (6.5%)
Race
Black	1 (12.5%)	7 (30.4%)	8 (25.8%)
White	6 (75.0%)	13 (56.5%)	19 (61.3%)
More than one race	1 (12.5%)	2 (8.7%)	3 (9.7%)
Other race	0	1 (4.3%)	1 (3.2%)
Hispanic/Latinx ethnicity	0	1 (4.3%)	1 (3.2%)
Sexual orientation
Gay or lesbian	0	2 (8.7%)	2 (6.5%)
Bisexual	0	5 (21.7%)	5 (16.1%)
Heterosexual	8 (100%)	15 (65.2%)	23 (74.2%)
Other	0	1 (4.3%)	1 (3.2%)
Age in years (M)	50.0	41.7	43.8
Education – bachelors or greater	7 (87.5%)	14 (60.8%)	21 (67.7%)
Low-income (<200% federal poverty level)	0	11 (47.8%)	11 (35.5%)
Urbanicity
Urban	3 (37.4%)	12 (52.2%)	15 (48.4%)
Suburban	5 (62.5%)	8 (34.8%)	13 (41.9%)
Rural	0	3 (13.0%)	3 (9.7%)
Time since resolving alcohol problems
2 years or more		14 (60.9%)
1–2 years		3 (13.0%)
Less than 1 year		6 (26.1%)
Treatment and self-help group history
Ever received inpatient treatment		8 (34.8%)
Ever received outpatient treatment		12 (52.2%)
Attended self-help groups in past 30 days (in person or online)		11 (47.8%)
Attended self-help groups previously but not in past 30 days		5 (21.7%)
Never attended self-help		7 (30.4%)
SIP Score (range 0–15) – Alcohol (M)		11.8
SIP Score (range 0–15) – Other drugs (M)		8.2
Current alcohol use
None		13 (56.5%)
Monthly or less		9 (39.1%)
2–4 times/month		1 (4.3%)
Current other drug use
None		13 (56.5%)
Monthly or less		8 (34.8%)
More than monthly		2 (8.7%)
Problem drug use (lifetime)
Marijuana		9 (39.1%)
Cocaine, crack, or methamphetamine		8 (34.8%)
Heroin		3 (13.0%)
Prescription opioids		5 (21.7%)
Other prescription medications		3 (13.0%)
Other drugs		2 (8.7%)

^*The total sample size was 32 participants; one service provider participant did not complete the demographic and background survey measures and thus is not included in the table.

Respondent feedback on the MIRC

A review of the initial 90-item list (see Figure 1) revealed a high degree of modification. Only nine (10%) of the original items remained unchanged in any form in the final 49-item version. As an example, Table 2 illustrates the evolution of items for the category of negative human capital, in which three items were retained in their original form, four were eliminated, two were revised, and one was added. Below, we elaborate on themes in participant feedback guiding choices about item retention, elimination, and revision as well as generation of new items. For context, in Table 3 we summarize demographic and recovery characteristics of participants whose interviews were excerpted.

Table 2.

Examples of item changes in the negative human capital category.

Original item	Action	Reason for change	Transformed item
I find it hard to have fun	Retained	N/A	N/A
I have too much time on my hands	Eliminated	Low relevance	N/A
I have problems with my physical health	Retained	N/A	N/A
I have problems with my mental health	Retained	N/A	N/A
I am dealing with a lot of stress	Revised	Linguistic clarity	I am experiencing a lot of stress
I have shame related to my actions while drinking or using	Revised	Conceptual clarity	I am struggling with guilt or shame
I often feel depressed	Eliminated	Captured by mental health item	N/A
I often feel anxious	Eliminated	Captured by mental health item	N/A
I have thoughts of harming myself or ending my life	Eliminated	Captured by mental health item	N/A
–	Added	New concept suggested	I have trouble sleeping

Table 3.

Characteristics of participants with excerpted interviews (n = 16).

ID Number	Recruitment Category	Gender	Race	Sexual Orientation	Income Group	Self-Help Group Attendance	Current Alcohol Use
1	Service provider	Cisgender female	White	Heterosexual	High	–	Monthly or less
2	Service provider	Cisgender male	White	Heterosexual	High	–	Monthly or less
3	Service provider	Cisgender male	White	Heterosexual	High	–	2–4 times/ month
4	Person in recovery	Cisgender male	White	Bisexual	Low	Currently attending	Never
5	Service provider	Cisgender female	Black	Heterosexual	High	–	Never
6	Person in recovery	Cisgender female	White	Heterosexual	Low	Currently attending	Never
7	Person in recovery	Cisgender male	Other race	Heterosexual	Low	Attended in past	Never
8	Person in recovery	Agender	Black	Asexual	Low	Never attended	Never
9	Service provider	Cisgender female	White	Heterosexual	High	–	Monthly or less
10	Person in recovery	Cisgender female	White	Bisexual	High	Attended in past	Monthly or less
11	Person in recovery	Cisgender female	White	Heterosexual	High	Currently attending	Never
12	Person in recovery	Cisgender female	White	Heterosexual	High	Currently attending	Never
13	Person in recovery	Cisgender male	Black	Heterosexual	High	Currently attending	Never
14	Person in recovery	Cisgender female	White	Heterosexual	High	Currently attending	Monthly or less
15	Person in recovery	Cisgender male	More than one race	Heterosexual	High	Never attended	Monthly or less
16	Person in recovery	Nonbinary	White	Bisexual	Low	Attended in past	Monthly or less

Unchanged items

Unchanged items tended to measure concepts widely understood to be key facilitators of recovery and were linguistically simple, such as ‘I feel physically healthy most days.’ The small set of unchanged items were viewed by respondents as important to their recovery and to recovery in general. For example, regarding an item about being part of a recovery community, service providers in one focus group expressed the following thoughts sequentially: ‘I think it’s totally, totally important and that’s a good way to ask’ (Participant 1), ‘I say that I agree, you know, being a part of our community and a sense of recovery is so essential’ (Participant 2), and ‘I agree with them. It’s good’ (Participant 3).

Item elimination

Outright item elimination was common, in response to participant critique. The item, ‘I am getting help for my recovery from doctors, nurses, counselors, social workers, peer recovery specialists, or other professionals’ was eliminated after critical feedback including concerns about the following: passivity versus assertiveness and self-accountability in recovery (Participant 4); that ‘informal supports’ are not included and may be just as relevant (Participant 5); that ‘there can be a lot of variance in between all the different types’ of support (Participant 6); and that the item was too aligned with a medical model of recovery, which generated responses such as, ‘the whole thing with labeling somebody an addict for the rest of their life is not productive at all’ (Participant 7). Another item that was eliminated was, ‘Drug courts or the criminal justice system have been a source of motivation and support for my recovery.’ Although some respondents found this relevant and reasonable, others found the question difficult to answer because drug courts versus the standard court system were so different, and because of painful prior experiences. ‘Look, for a lot of people they’ve had negative experiences in those systems, so they don’t try,’ Participant 8 commented.

Item revision

Feedback on item revision took two main forms: suggestions to improve linguistic clarity and simplicity, and recommendations to refine conceptual clarity. Some items were slightly reworded to capture nuances not present in the original framing. One example was replacing the word ‘good’ with ‘effective’ for the item ‘I have good coping skills that I use in my recovery,’ based primarily on feedback from one participant who noted:

I don’t like ‘good’ coping skills, because like, I have shitty coping skills that still keep me sober. I feel like I’ve gained some weight because I like sugar, but it’s kept me sober for going on six years now.

(Participant 4)

Other items were changed for greater conceptual rather than linguistic clarity. Two items, ‘I have people close to me who are still having problems with alcohol or drugs’ and ‘Friends or family members use alcohol or drugs around me’ were merged into ‘People using alcohol or drugs around me makes my recovery more difficult.’ The reworking removed the specificity of ‘friends or family members,’ expanding the scope. Further, it removed the concept of ‘problems,’ which elicited a negative response in that people were reticent to judge whether someone else had a problem. In sum, participants felt that the primary detriment to recovery was others using substances in their presence, regardless of if substance use was problematic for those individuals, and regardless of if they were friends or family. As Participant 9 explained:

I feel like it has this judgment of people we don’t know … the ‘still having problems with’ implies that, we just think that they shouldn’t be drinking or whatever … ‘ whereas it could just be, ‘are the people around you sober or not, yes or no.’

Another example is the transformation of the item ‘I am living where I want’ into ‘My housing situation is helpful for my recovery.’ Many participants, and especially service providers, rejected the initial formulation on the grounds that what people want is often far afield from what is beneficial in recovery. A provider (Participant 9) noted with wit, ‘I could be living in the place I want because it’s a trap house and it’s free, but that’s not a good thing for my recovery.’

New items

A few novel items were introduced via the feedback process. Two participants suggested adding an item under negative social capital to assess isolation and loneliness, with one alluding to feeling ‘socially outcast’ in recovery (Participant 10) and another elaborating on the difficulty of maintaining social connections in recovery when one’s peers are engaged in activities that involve alcohol (Participant 11). As a result, we added the item ‘I feel alone’ as negative social capital. Participant 12 discussed the importance of sleep as human capital, noting ‘there are a lot of people with addiction who have sleep disorders.’ The research team thus formulated the item, ‘I have trouble sleeping’ under negative human capital.

Participant 13 talked extensively about how racism and sexism impact recovery, such as through shaping one’s experience of treatment and how people are treated in the criminal justice system. At the end of the interview, the interviewer asked the participant about his thoughts regarding adding an item to assess discrimination and bias as negative cultural capital. ‘I think we need to start talking about that,’ the participant offered, elaborating that the current social climate ‘might create a space for someone to be able to answer that question.’ In response, we added the item ‘Discrimination related to my race, gender, or sexual orientation has made my recovery harder.’

Overarching themes and global survey experience feedback

In synthesizing participants’ feedback, we noted little systematic variation according to characteristics such as racial and ethnic identity, gender identity, and socioeconomic status. No items were consistently endorsed or rejected by members of one group and not by another. However, we detected subtle variations. One illustration is that for the item ‘It’s hard to avoid the alcohol or drug use in my neighborhood or town,’ participants across income levels agreed on the item’s salience as negative community capital, but the item appeared to particularly resonate with low-income people. One participant vividly recalled going to an AA meeting in a space that used to be a bar: ‘I mean, you walk into this meeting and … The whole bar is still set up … the first thing I think of is ‘Damn, I want to drink” (Participant 7). This participant also discussed seeing people selling drugs and ‘liquor everywhere’ in the neighborhood where he went for counseling appointments. Although higher income participants agreed that this item was important, one suggested that this concern might be mitigated via participation in online communities and Zoom-based meetings, a possibility not overtly recognized by participants with lower incomes.

We observed that systemic variation in responses to particular items appeared to occur primarily on the basis of recovery pathway. For instance, most participants affirmed the salience of the item ‘I have a spiritual practice that helps me in recovery,’ with some linking it to spirituality in a 12-step context and others situating spirituality more broadly. ‘The 12-step program, that belief in the higher powers is a big deal,’ Participant 14 remarked. Others appreciated that the item was formulated in a broad way, with some, such as Participant 15, linking it to mindfulness and meditation practices. However, a few participants who articulated having negative experiences in 12-step groups expressed concern that the item was too aligned with a 12-step view: ‘You don’t ask a diabetic if they needed a higher power to recover,’ Participant 16 stated. Given that the large majority of participants affirmed the item and were able to interpret it in nuanced ways according to their own notions of spiritual practice, we retained this item without revision, but noted that future research using the MIRC might explore varying response patterns to this item depending on respondents’ experience with 12-step groups.

Additionally, a few participants expressed feeling that the survey questions were judging the quality of their recovery or blaming them for any deficits in recovery capital. For example, one participant stated that answering affirmatively to the item ‘I have problems with my mental health’ might feel ‘harsh,’ particularly ‘if they’ve been labeled as trouble-some in some way, shape, or form’ (Participant 11). Another participant remarked that the item ‘I feel supported by my family’ (eventually revised to ‘My family supports my recovery’) could trigger negative emotions in people who don’t have family support, including people who have experienced family rejection because of their sexual orientation, which could be an underlying reason for their substance use problems (Participant 4). Although the potential perception of judgment seemed difficult to completely avoid, the research team weighed this consideration and added instructions aiming to convey a nonjudgmental tone to the overall measure.

A final noteworthy point from participant feedback was the limitations posed by closed-ended questions. ‘My fantasy … is that there would be a place for people to write-… the qualitative details,’ Participant 12 offered. Other participants expressed a similar desire to qualify or elaborate on their responses to the closed-ended survey questions. Although our focus remains on developing the MIRC as a traditional close-ended survey measure, we note this as a consideration for future clinical use of the finalized measure, as completing the measure could serve as a starting point to open-ended conversations about recovery capital.

Discussion

This study describes the evolution of the item pool for a new measure of recovery capital, shaped through a process of eliciting the perspectives of service providers and people in recovery. Although the formulation of items is foundational to the development—and ultimate utility—of any psychometric instrument, this step typically receives little attention in the research literature (Holmbeck and Devine 2009; Williams et al. 2019). Our intent in this paper was to open the black box of item development and illuminate at a granular level how respondent feedback shapes decisions regarding item retention, elimination, revision, and generation.

For some measures, such as Neale et al. (2014, 2016) work to develop a comprehensive measure of recovery, constructs are inherently murky and extensive qualitative work is needed to flesh them out. In our case, recovery capital theory provided a backbone for the measure, specifying the sub-constructs of social, physical, human, and cultural capital in positive and negative forms (Cloud and Granfield 2008). However, clarity of theoretical concept and elucidating measurement items are quite different affairs. The qualitative insights gained in this process strongly informed the latter. Despite the vetting of potential items with research experts and subsequent revision, we drastically modified the item pool based on participant feedback, leading to further item revision, item elimination, and new item generation. Only a small proportion (10%) of items from the original item list remained unchanged on the final 49-item pilot MIRC, due to consistently positive feedback and near universal endorsement. The process repeatedly demonstrated the ways in which the majority of initial measurement items had construct and content validity from the perspective of research experts, but lacked face validity from target users prior to their revision. Based on our results as well as a growing number of studies on item development for patient-reported outcome measures in health care (Pilkonis et al. 2013; McCarrier et al. 2016; Williams et al. 2019), we advocate that target population feedback on potential items become a standard part of the measurement development process, alongside expert feedback on items and direct and indirect target user input in subsequent stages of cognitive interviewing and psychometric testing.

Furthermore, this study affirms the value of eliciting feedback from target users who are heterogeneous with regard to demographics and recovery characteristics. Because sample sizes for qualitative research are typically small, there is a danger in gathering feedback from homogenous samples when the goal is to build a measure that can be used widely with heterogeneous groups. We were especially struck in our study by the variability in feedback on particular items according to recovery pathway. This finding underscored the importance of attempting to write items in neutral ways that neither endorsed nor repudiated a 12-step model of recovery. We also took care to ensure that no items on the measure implied a view of recovery centered on abstinence. As our sample included a substantial proportion of recovering participants (43.4%) who reported at least occasional alcohol use, we conclude that the measurement items appear to have credibility with people on both abstinent and non-abstinent pathways with regard to alcohol. Future research should build on recent work by Majer et al. (2021) by exploring variations in recovery capital for people on different recovery pathways, including those that include 12-step support and/or an abstinence orientation and those that do not. Future work should also address variation in recovery pathways with regard to differing substances, as a person may abstain from some substances and continue to use others, as well as explore the intersection of medication-assisted treatment with recovery capital.

Additionally, a criticism of recovery research is that much of it relies primarily on samples of people in inpatient or outpatient substance use treatment (Subbaraman et al. 2015). This excludes people in natural recovery, as well as many people in long-term recovery who have not been in treatment recently. Although a measure of recovery capital could be useful in treatment settings, we see its utility as extending beyond these bounds, encompassing possibilities such as people in natural recovery using the tool for self-assessments and researchers using the measure in studies. While it can be more challenging to locate and recruit participants who are not engaged in formal treatment, we believe it is critical to do so when eliciting feedback on recovery-related measures.

Several limitations exist regarding this study and the measure we are developing. Despite our best efforts to recruit a diverse sample, our sample is not a perfect representation of the larger population of people in recovery from alcohol problems. No participants in the sample identified as Native American or Asian American and only one participant identified as having Latinx or Hispanic ethnicity. Regarding recruitment, we did not formally track the number of participants recruited by each recruitment method we used, limiting sampling transparency. It is also critical to note that our choice to focus on people in recovery from alcohol problems limits the generalizability of the MIRC. Although we formulated MIRC items to refer broadly to ‘recovery’ or ‘alcohol or drugs’ and not alcohol alone, further evaluation would be needed to establish the face validity of items with people whose recoveries do not involve alcohol problems. In addition, the phrasing of our inclusion criterion asked participants if they had ‘resolved a prior problem with alcohol.’ Although not our intention, this phrasing may have implied that recovery is a finite task, rather than a dynamic and ongoing process of change (Witkiewitz et al. 2020). Thus, this wording could have excluded potential participants who identify as in recovery but do not consider their problem resolved.

A methodological limitation is that we collected data via both individual interviews and focus groups. These options were advantageous in terms of accommodating participants’ schedules and preferences, though our analysis did not account for the possibility that individual versus group format may have influenced participants’ responses. It is also possible that we would have received different feedback had we conducted data collection in person. Participants appeared comfortable engaging in the interview or focus group process via Zoom, but we did not gather direct feedback on how participants felt about the technology. In addition, some people in the target population may not have pursued participation due to the online format. Although there is great potential for the ongoing use of videoconferencing technology in qualitative research to overcome participation barriers based on public health concerns and/or geographic, mobility, and scheduling limitations, participant preferences for online versus face-to-face opportunities should be considered (Archibald et al. 2019).

Conclusion

Measures that are sensitive to change, reliable, and valid for diverse populations are essential to advancing recovery research (Hennessy 2017; LeCroy 2019). Specifying items is merely the first step in building and fully validating a measure of recovery capital that can be used to study and address the root causes of disparities in recovery capital and recovery-related outcomes for marginalized populations. We encourage researchers who are developing new measures for a variety of constructs to explicate their methods for ensuring that items are comprehensible and resonant across varied segments of a target population. There is no substitute in the later stages of scale development for the foundational vetting of measurement items with the stakeholders who hold the deepest from of expertise: the heterogeneous target users of the future measure.