Prioritizing Needs and Outcomes for Adolescent Substance Use Treatment Planning: An Online Modified-Delphi Process

Objectives:

Key stakeholders can have differing views about which information is essential to inform placement decisions for all patients. This study examined consensus across stakeholder groups on the most important individual needs and treatment outcomes for informing decisions specifically about the level of care for an adolescent in substance use treatment.

Methods:

We conducted an online modified-Delphi process with treatment providers, policymakers, researchers, and parents of adolescents who have received substance use treatment. Participants rated 48 individual needs from the Global Appraisal of Individual Needs–Initial that were mapped onto the 6 dimensions of the American Society of Addiction Medicine Criteria. In addition, participants rated 10 treatment outcomes from the Substance Abuse and Mental Health Services Administration's National Outcome Measures. We assessed consensus within stakeholder groups using the RAND/UCLA Appropriateness Method. We considered the items reaching consensus with the highest ratings across stakeholder groups as the most important individual needs and treatment outcomes.

Results:

We recruited 194 participants (81 providers, 54 policymakers, 32 researchers, 27 parents). Participants identified suicidality and severity of substance use disorder symptoms as the most important individual needs, and reduction in substance use as the most important treatment outcome.

Conclusions:

Standardized procedures for matching adolescents to levels of care for substance use treatment should at a minimum be based on assessments of suicidality and severity of substance use disorder symptoms, and consider reduction in substance use as a primary treatment outcome. These findings can progress the development of “level-of-care” decision rules specifically for adolescents.

Approximately 1 million adolescents in the United States meet substance use disorder (SUD) criteria (SAMHSA, 2018). While many treatments for adolescent substance use exist (Das et al., 2016), individual adolescents respond differentially to treatment based on their unique medical, psychological, and social needs, as well as the outcomes used to measure progress and success (National Institute on Drug Abuse, 2014). Much attention has therefore recently focused on developing measurement instruments and placement criteria that match each adolescent to their most appropriate level of care (Fishman, 2014). However, a pragmatic barrier to integrating these multidimensional patient assessments into the treatment planning process is the substantial number of needs and outcomes to potentially consider (Winters et al., 2014). For example, treatment providers often consider the time it takes to administer an assessment tool, with longer tools less likely to be integrated into routine clinical practice (Hamilton et al., 2008). To address this barrier, developers of prominent instruments offer briefer versions that assess the most important individual needs and treatment outcomes (Dennis et al., 2003).

Key stakeholders can have differing views about which individual needs and treatment outcomes are most important for treatment planning, based on their diverse experiences and perspectives. Policymakers prefer clinical indicators and outcomes that best fit the resources, personnel, and financing of their clinics and systems (Sterling et al., 2010). Adolescents and their parents frequently identify a range of non-clinical indicators and outcomes as essential to long-term recovery (Acri et al., 2012). Researchers are likely to focus on items with more robust evidence and high-quality measurement instruments (McGovern and Carroll, 2003). Identifying the individual needs and treatment outcomes that all key stakeholder groups consider of higher importance could serve as an important step toward improving the development and implementation of patient placement criteria for adolescents entering substance use treatment.

METHODS

This study is part of a larger project that combines stakeholder engagement with modern statistical methods to develop well-operationalized, empirically-supported sequences of decision rules for placing adolescents with SUDs in appropriate settings (Grant et al., 2017). Study procedures were given exempt status by the RAND Human Subjects Protection Committee (ID 2015-0268-AM01), and the protocol and materials can be found on the Open Science Framework (https://osf.io/bqbrw/).

Participants

We conducted an online modified-Delphi process consisting of 4 concurrent panels (1 for each stakeholder group): providers of adolescent substance use treatment; policymakers overseeing program planning at the clinic, health-system, state, or federal levels; researchers of adolescent substance use treatment; and parents of adolescents who have received substance use treatment. We approached stakeholders identified through published research; member lists of relevant organizations; treatment centers listed in the Substance Abuse and Mental Health Services Administration (SAMHSA) Treatment Locator; and suggestions from the wider project team, advisory board, and stakeholders we approached via the above mechanisms. We sent identified stakeholders an email with a link to our recruitment survey and informed them that they would receive a $200 Amazon gift card for completing the online panel. We aimed to recruit 20 to 40 participants per panel—a manageable size that included a diverse and significant group of stakeholders (Khodyakov et al., 2011).

Procedures

Our recruitment survey asked stakeholders to indicate their interest in participating in the study and complete a demographics questionnaire. We then sent interested stakeholders a link to RAND's ExpertLens web-based system for a three-round, online Delphi process (Dalal et al., 2011). We developed a list of specific individual needs and treatment outcomes using the Global Appraisal of Individual Needs–Initial (GAIN-I) instrument (Dennis et al., 2003). Participants rated 48 individual needs from the GAIN-I that map onto the 6 dimensions of the American Society of Addiction Medicine Criteria and 10 treatment outcomes from the GAIN-I that map onto the SAMHSA National Outcome Measures (NOMs). In Round One, participants used a 9-point Likert scale (1 = lower importance to 9 = higher importance) to rate the importance of assessing the 48 individual needs and 10 treatment outcomes for deciding the appropriate level of care (outpatient, intensive outpatient, residential, or inpatient) for an adolescent's substance use treatment. In Round Two, participants received feedback on Round One results: each participant saw graphed results (histogram with the median rating and their own rating for each item) and decisions about the group's consensus on the importance of items. In Round Three, participants re-rated each item in light of Round Two feedback.

Data Analysis

We first conducted a descriptive analysis of participant demographics and item ratings. We then assessed consensus within each panel using the inter-percentile range adjusted for symmetry (IPRAS) technique from the RAND/UCLA Appropriateness Method (Fitch et al., 2001). IPRAS initially determines whether there is disagreement (ie, uncertain importance) based on the dispersion of ratings; if there is no disagreement, the median rating indicates group consensus (ie, a median between 1-to-3 indicates “lower importance”, 4-to-6 “moderate importance”, 7-to-9 “higher importance”). We considered items of “higher importance” with a median of 8 or 9 in every panel as the most important individual needs and treatment outcomes.

RESULTS

Of 281 invited stakeholders, 194 (69%) participated (see Table 1). Participants predominantly were female (71%), were 35 to 54 years old (57%), had a bachelor's or master's degree (73%), were Caucasian/White (80%), lived in urban communities (71%), and had experience with outpatient settings (78%) for either cannabis (88%) or alcohol (87%). No items had uncertain importance within a panel (see Table 2). Two individual needs were identified as the most important: suicidality (“thoughts of, plans for, actions toward, or attempted suicide”) and severity of SUD symptoms (“extent of physiological, psychological, and social problems related to substance use”). One treatment outcome was identified as most important: reduction in substance use (“the degree to which the adolescent has reduced the quantity and frequency of their substance use by the end of treatment”). Moreover, every stakeholder group considered an additional 15 individual needs and seven treatment outcomes of higher importance. The 2 highest-rated of the 7 additional treatment outcomes match the most important needs: that is, reduction in substance use disorder symptoms (“the degree to which the adolescent has experienced a reduction in their symptoms of substance use disorder by the end of treatment”) and improved mental health (“the degree to which the adolescent has experienced an improvement in their mental health symptoms by the end of treatment”). Of the remaining items, 7 individual needs and 1 treatment outcome were considered of higher importance in only 3 panels, 5 individual needs and 1 treatment outcome of higher importance in only 2 panels, 7 individual needs of higher importance in only 1 panel, and 12 individual needs of higher importance in no panels. Only 1 item—the adolescent client's spiritual environment—was considered of lower importance by any panel.

TABLE 1.

Participant Demographics

TABLE 2.

Rating Results From Round Three

DISCUSSION

We engaged 194 stakeholders in an online modified-Delphi process to examine consensus on the relative importance of individual needs and treatment outcomes for informing level of care recommendations for adolescents entering substance use treatment. Stakeholders agreed that suicidality and severity of SUD symptoms were the most important of 48 individual needs, and that reduction in substance use was the most important of 10 treatment outcomes. Every stakeholder group also agreed on an additional 15 individual needs and 7 treatment outcomes as highly important, and the 2 highest-rated of these 7 additional treatment outcomes match the most important needs: that is, “reduction in substance use disorder symptoms” and “improved mental health” (which includes suicide ideation, planning, and attempts). These findings reflect the items included in even the briefer versions of measurement instruments linked to patient placement criteria, which professionals involved in adolescent substance use treatment planning can use to assess the individual needs and treatment outcomes deemed highly important by stakeholders in this study (4,8,5,Dennis et al., 2003; Hamilton et al., 2008; Fishman, 2014; Winters et al., 2014).

Several limitations are worth noting in interpreting the results of this study. First, as with all Delphi processes, our participants are not a representative sample of all relevant stakeholders. While we recruited a much larger sample than traditional online panels, the results of this project may be biased towards stakeholder sub-samples with Internet access, available time to respond, and stakeholders who identify as White and non-Hispanic. Second, this project was focused on a US-context; while results may have relevance to other countries and healthcare systems, these findings are more applicable to a US population—particularly those with experience in urban and outpatient settings. Lastly, Delphi processes involve self-report surveys designed to explore and develop consensus, leaving results subject to the facets, challenges, and limitations inherent to this study design and measurement format.

Matching individual adolescents to the most appropriate level of care is challenging, particularly due to the substantial range of individual needs and treatment outcomes to potentially consider. Findings suggest that multiple stakeholder groups consider suicidality, severity of SUD symptoms, and reduction in substance use as essential to inform level of care decisions for every adolescent entering substance use treatment. These findings can progress the development of “level-of-care” decision rules specifically for adolescents. For example, the individual needs and treatment goals identified as important by all stakeholder groups will be prioritized during later stages of our project for the development and testing of well-operationalized, empirically-supported sequences of decision rules for placing adolescent clients with SUDs in appropriate settings (Grant et al., 2017). In the interim, standardized procedures for matching adolescents to levels of care for substance use treatment should at a minimum be based on assessments of suicidality and severity of substance use disorder symptoms, and consider reduction in substance use as a primary treatment outcome.