Developing a Tailored Texting Preventive Intervention: A Card Sort Methodology

Abstract

Purpose

Intervention content written by adults for youth can result in miscommunication due to generational and cultural differences. Inviting at-risk youth to participate in the creation of intervention material can augment acceptability for their peers.

Methods

To improve intervention messaging, the present study examines the utility of a card sort technique when creating cellular phone text messages to be used in a preventive substance use intervention. During focus groups with 24 youth who are exiting the foster care system – a population with distinct cultural attributes – participants were asked to rate stage of change-specific health messages rooted in Motivational Interviewing and the Transtheoretical Model.

Results

Participants unanimously favored content that encouraged autonomy and choice. Statements that invited a “look to the future” were also rated favorably. Messages that referenced the past were not rated well, as were suggestions for professional assistance. Finally, encouragement to receive social support for change was met with ambivalence. While some participants regarded support as helpful, many others felt a severe lack of support in their lives, possibly prompting further substance use.

Conclusions

Youth exiting foster care constitute a unique population whose voice is paramount in the development of interventions. The content present in traditional approaches to substance use prevention (e.g., increasing social support) may not apply to this group of vulnerable youth. The card sort technique has strong potential to evoke youth-specific intervention content that is more readily understood and accepted by target audiences.

Introduction

Fewer than 10% of adolescents and young adults in need of treatment for substance use disorders receive care (Substance Abuse and Mental Health Services Administration, 2009). Many young people do not perceive the need to obtain such services (Wu & Ringwalt, 2006), reporting that substance use is normative for their age group (Colby, Swanton, & Colby, 2012; D’Amico, 2005; Jacobs & Johnston, 2005). However, a significant number of adolescents and young adults are interested in obtaining assistance, but are met with a different set of barriers including concerns about stigma (Corrigan, 2004; Rickwood, Deane, & Wison, 2007), difficulty articulating the extent of their problems or the need for help (Feldstein Ewing, Hendrickson, & Payne, 2008), and plainly not knowing where to turn for such resources (Klein, McNulty, & Flatau, 1998).

Although many efforts are being made to address prevention service access issues (D’Amico et al., 2012), the acceptability of substance use treatment and other mental health services remains an obstacle for young people. New approaches are therefore needed to improve acceptance of interventions for this population. Creating a novel and acceptable intervention can present a number of challenges, with the inherent differences between the researchers creating the intervention and their target population amplifying these difficulties. For instance, when adult researchers strive to create an intervention for young people, disconnects may exist in critical domains such as age, language, and perspective. D’Amico, McCarthy, Metrik, and Brown (2004) noted that most treatment services for adolescents have been adapted from adult models, which may not be particularly well-suited for adolescents (Brown, 1993; Brown & D’Amico, 2001).

Current literature on the development of substance use interventions for adolescents stresses the importance of comprehending the culture of the specific target population. Drury (2003) highlights differences in adult and adolescent perceptions of the other as a common concern. Specifically, adults often perceive adolescents as having “problems” in communication, leading youth to appear uncommunicative and even hostile (MacDonald, 1998), whereas adolescents often express the feeling that adults do not consider their perspectives. Rice and Dolgin (2008) suggest that adolescents frequently feel that adults, particularly adult practitioners, do not understand, listen to, or give credence to their own thoughts. Frameworks such as motivational interviewing, which emphasize conveying empathy, supporting autonomy, and encouraging a collaborative approach to prevention and treatment services (D’Amico et al., 2005; Naar-King & Suarez, 2011) can help bridge the communication gap. Integrating interactive and motivational techniques may be an important way of narrowing the gap between adult researchers and adolescent participants (D’Amico et al., 2005).

Not surprisingly, current literature suggests that programs developed with youth input are more likely to appeal to diverse groups of young people (D’Amico & Edelen, 2007). Nastasi (1998) suggests that when working with adolescents, it is important for researchers to recognize that the culture of adolescents is different from that of adults. Researchers should, thus, seek to more deeply understand adolescents’ culture(s) and apply this knowledge to program development. Eliciting youth input into prevention communications is also consistent with the recent Patient Centered Outcomes Research Initiative (PCORI; Gabriel & Normand, 2012), which promotes listening to the “voice of the patient” in interventions and outcome assessment. Creating culturally-consistent interventions, making use of the population’s vernacular, and reflecting the values and beliefs of group members can enhance service acceptability and efficacy (Nastasi et al., 2000), which ultimately should improve intervention effectiveness.

The card sort task is an approach which may mitigate these concerns, as it allows researchers to better understand the thought processes of their group of interest. It is a qualitative data collection technique often used to explore how participants organize and label concepts (Neufeld et al., 2004), to determine participant priorities (Lang & Carstensen, 2002), test theoretical models (Jahrami, Marnoch, & Gray, 2009), and to identify group and individual needs (Luniewski, Reigle, & White, 1999). It is based on Kelly’s personal construct theory (PCT), which posits that people make sense of the world by categorizing it and that people are able to describe their own categorization of the world (Rugg & McGeorge, 1997). Its advantages, including ease of administration, low susceptibility to demand biases, ability to handle large numbers and types of stimuli, and grounding within a theoretical framework (i.e., PCT), make it a popular technique (Whaley & Longoria, 2009). Numerous studies have utilized the card sort task to understand how individuals group or label concepts such as affect regulation strategies (Niven, Totterdell, & Holman, 2009) and coping behaviors under various stressful life events (Schwartz, Peng, Lester, Daltroy, & Goldberger, 1998). Other researchers have extended this use by asking participants to rank order concepts in order of importance or priority (e.g., Lang & Carstensen, 2002; Periyakoil, Noda, & Chmura Kraemer, 2010).

Increasingly, card sort methodology has been adapted to aid in the design of interventions by guiding decisions regarding service administration, challenges to anticipate, and how to best address implementation concerns (Nastasi & Berg, 1999). For such work, researchers have combined the goals of labeling and ranking stimuli by asking participants to group intervention materials into categories based on preference. For example, in their endeavors to develop text-messaging reminders for immunizations, Ahlers-Schmidt and colleagues (2013; 2012) asked participants to categorize text messages into: 1) critical information, 2) appropriate additional or clarifying information, or 3) unnecessary information.

Others have utilized a similar approach within a group setting, asking focus group participants to vote on presented intervention content (Hingle, Nichter, Medeiros, & Grace, 2013; Woolford et al., 2011; Yan et al., 2015). In each of these studies, participants indicated whether they liked or disliked a particular message [or liked it, but with caveats (Hingle et al., 2013)] and, either based on thresholds for voting or in all cases, group discussion would ensue as appropriate. These strategies capitalize on population-specific consensus of intervention content, thereby receiving a preliminary “stamp of approval” by the group of interest.

The current study adapts the card sort technique further, working with youth who are aging out of the foster care system – a population both at particular risk for substance use problems (for a review, see Braciszewski & Stout, 2012) and with distinct cultural attributes. Concerns about trusting health professionals, direct consequences for admitting substance use, and perceived lack of empathy make traditional forms of substance use prevention untenable with this group of young people (Braciszewski, Moore, & Stout, 2014). In response, we are engaging youth using a participatory approach to develop and test an indicated prevention program. Focus groups were conducted to elicit feedback on proposed intervention strategies and content. The current study reports the results of a card sort technique used in each focus group. These results will be used to modify content delivered in a future open trial of the intervention.

Method

Design

A series of focus groups was conducted as the first part of a three-phase project to iteratively develop and test a preventive substance use intervention for youth aging out of the foster care system. Focus groups covered a host of issues regarding intervention acceptability (described below). The current study describes the results from one segment of these groups, the card sort technique.

Participants

Young adults were recruited from a large New England agency serving individuals receiving transitional foster care services. To be included in the study, participants had to: (1) be 18–19 years old; (2) be no more than 2 years removed from state custody/foster care; (3) score “moderate” or “high” risk on the Alcohol, Smoking, and Substance Involvement Screening Test (WHO ASSIST Working Group, 2002); (4) not currently be in or seeking substance use treatment; (5) own a mobile phone; and (6) use text messaging at least weekly. Eligible young adults were then invited to take part in a focus group on the development of iHeLP, for which they would receive a $50 gift card.

In all, 89 young adults were screened, of whom 41 (46%) were eligible. The most commons reasons for ineligibility were low risk use of substances (n = 32) and no lifetime use of substances (n = 12). Seventeen youth did not participate in a focus group because of scheduling issues. Of the 24 who participated in the focus groups (n = 6, 3–6 participants per group), 62% were female and 54% of the sample indicated a Hispanic/Latino ethnicity. Twenty-nine percent reported their race as Black/African American, 62% as Caucasian, 4% as American Indian/Alaskan Native, and 4% as Asian. Within these categories, 17% reported more than one race. Almost two-thirds (62%) were currently employed and participants averaged roughly 11 years of education (SD = 2.02). Length of time in foster care ranged from 1 to 15 years (M = 4.04, SD = 3.42), with an average of 6.04 placements (SD = 8.02).

Creation of intervention content

To address substance use among youth aging out of foster care, we have created iHeLP (Interactive Healthy Lifestyle Preparation), which combines a 20-minute computerized screening and brief preventive intervention followed by dynamically-tailored text messaging. iHeLP addresses alcohol and illicit drug use by using an approach consistent with Motivational Interviewing (MI; Miller & Rollnick, 2013), a client-centered, directive method to enhance motivation for behavior change.

While a full description is available elsewhere (Braciszewski et al., in press), iHeLP tailors text messages to each participant’s current level of readiness to change their substance use. Categories, or “stages,” of change were taken from the Transtheoretical Model (TTM; Prochaska & DiClemente, 1992), which posits that behavior changes often involves a continuum where individuals may not think that a behavior is a problem (termed “Precontemplation”), are ambivalent about making a change (“Contemplation”), want to make a change (“Preparation”), start changing (“Action”), and continue those changes long-term (“Maintenance”). As such, communication with an individual who does not believe that he or she has a problem would be different than someone who is ready to change. Often, incorrect assumptions are made about readiness, particularly regarding substance use, which leads to breakdowns in communication between client and health professional.

The TTM is complementary to MI, as MI is a method of conversation that is particularly effective at the beginning TTM stages of readiness for change (DiClemente & Velasquez, 2002). Accordingly, communication with individuals at these beginning stages focuses more on engaging and instilling discrepancy in their current behavior, attention to why the individual might consider change, and an emphasis on autonomy (Miller & Rollnick, 2013). On the other hand, those “on the fence” can be more directly engaged in a conversation about discrepancy, weighing pros and cons, and encouraging self-efficacy. Finally, individuals who are ready to make a change can be invited to create their own strategies while clinicians support their ability to do so. These guidelines for communication can, thus, be translated into individual messages framed for each stage of change.

Indeed, prior to running the focus groups, the study team created a “text bank” of 90 messages, evenly split among the Precontemplation, Contemplation, and Preparation stages of change.¹ Message content was developed in accordance with MI (Miller & Rollnick, 2013) and the TTM (Prochaska & DiClemente, 1992) theories by the first and fourth authors, who have extensive clinical, research, and training experience using MI. Content was also adapted from previous successful trials of text messaging for behavior change which have utilized MI and TTM themes (Braciszewski et al., in press). Generally, messages were derived from the core interviewing skills of MI: open-ended questions, affirmations, reflections, and summaries (OARS; Miller & Rollnick, 2013). Thus, phrases that might be reflections of a client statement in session were turned into statements; evocative open-ended questions for participants to consider were also created. While OARS provide a manner in which to deliver information in an MI framework, the content of that message can be themed strategically to emphasize autonomy and personal choice, develop discrepancy, evoke change talk/thoughts, enhance self-efficacy, provide support, and (in some instances) offer advice or information, all of which influence change behavior.

At the heart of this strategy, however, is meeting the person where they are in the change process; thus, our text bank was designed as such. Thematic content for individuals in Precontemplation relied heavily on principles of autonomy (e.g., “The most important thing when it comes to change is making sure you’re ready for it. If you’re not, it’s ok.”) and reasons for change (e.g., “What concerns do you have about changing your alcohol use?”). Contemplation messages focused more on developing discrepancy (e.g., “What are your goals over the next year? How does drinking fit with those goals?”) and entertaining the possibility of change (e.g., “How would your relationships be different if you changed your drinking?”). Finally, Preparation content was much more direct, offering suggestions (e.g., “Get rid of temptations and reminders of smoking - throw out pipes and get rid of lighters at home, work, and in the car.”) and support (e.g., “Tell supportive friends and family about your decision. They can help encourage you.”). MI and TTM theories, however, acknowledge that motivation for behavior change is fluid and strategies (e.g., building autonomy, offering social support, developing discrepancy) are not contained to specific stages. Thus, some approaches – especially autonomy – were purposely implemented across stages as we created the text bank.

Procedure

Focus groups

Participants attended one focus group each, the overall goal of which was to provide qualitative feedback on the acceptability of iHeLP, its perceived utility, likelihood of use by other foster youth, and preferences for the content, interface, and functionality of the components. Specifically, each focus group had 5 major parts: (1) rationale for the project; (2) open-ended discussion on participant ideas for prevention development; (3) in-vivo presentation of the computerized portion of iHeLP; (4) the card sort task (described below); and (5) feedback on the logistical aspects of text message delivery (e.g., frequency and timing of messages, intervention duration). Groups were co-facilitated by the first two authors and lasted between 1 and 1.5 hours each. Participants were briefed with regard to confidentiality in a group setting, were informed that the group would be audio-recorded, and provided written, informed consent for participation. Recordings were later transcribed for thematic analysis.

Card sort

While logistical information and acceptability of the overall approach and the computer-specific content was gathered in the other parts of each focus group, feedback on text message content was collected via the card sort technique. Prior to beginning the task, participants were provided with a basic overview of the TTM: that individuals can be at different stages of readiness to change a behavior, the names given to those stages, and that conversations with a person may look different depending on that person’s stage. Group moderators then, for the first time, introduced the idea of text messaging as a component of iHeLP, indicating that daily messages would be sent, each according to an individual participant’s stage of change.

Each participant was then given one stack of 18 index cards; each card contained one sample text message (randomly distributed from the text bank of 90 messages). Six sample messages were given for each of the three stages of change investigated in this study. For ease of interpretation, the three stages were renamed “stay the same” (Precontemplation), “on the fence” (Contemplation), and “ready to change” (Preparation). Stage names that accompanied each message were printed on each card. Individuals were asked to sort their messages into two piles: “good” and “bad.” Messages were collected and deposited into large envelopes for later analysis. Formal discussion of ratings did not take place; rather, after index cards were fully collected, discussion turned to the practical aspects of message delivery. However, participants did offer some opinions spontaneously after the card sort ended (noted below, where appropriate). Appropriate ethical human subjects guidelines were followed and the study procedures were reviewed and approved by the Pacific Institute for Research and Evaluation Institutional Review Board, according to NIH regulations.

Data Analytic Strategy

The overall goal of the analytical process was to delineate themes of message content representing major stages of change that would be acceptable to youth aging out of foster care. Influenced by previous methods used with card sort-related data (Woolford et al., 2011; Yan et al., 2015), we first tallied the number of times each message illustrating a stage of change was rated as “good” or “bad” by the participants. While text message content initially fell under the broad themes outlined by MI and TTM (e.g., autonomy, support), messages were coded for further detail (i.e., more descriptive labels/themes) using a theoretical thematic approach (Boyatzis, 1998), within each stage of change. Once these detailed themes were delineated, we calculated total theme-based rating scores, allowing us to examine approval/disapproval of a theme. The totals reported below reflect the majority rating; that is, if a message/theme received 2 good ratings and 3 bad ratings, we reported it as being viewed negatively at a rate of 60% (3 bad ratings divided by 5 total ratings). A message/theme receiving 5 good ratings and 1 bad rating would be reported as positively viewed at 83%. We refer separately, below, to message-versus theme-level (e.g., “retrospective messages”) ratings.

Results

The ninety messages were rated an average of 3.53 times (range: 1 – 7; SD = 1.59) by the 24 raters. Almost all messages (92%) were given at least one “good” designation and 64% were received more good than bad ratings (i.e., were categorized as good more times than bad by multiple raters). Roughly 13% were rated neutrally (i.e., equal numbers of good and bad ratings) and the remaining 22% were given more bad than good ratings. Given the variance in number of ratings, only messages receiving three or more ratings were analyzed for thematic content. Messages in this reduced set (n = 57) were rated an average of 4.53 times (range: 3–7, SD = 1.10).

Stay the Same/Precontemplation

Within this category of text messages, 87% were rated good at least once, while 63% were considered more good than bad (17% neutral, 20% more bad than good). Retrospective messages (e.g., “What got in the way of a previous change and how can you make it different?”) and facts/warnings regarding harm (e.g., “Like cocaine, heroin, and marijuana, alcohol affects chemical pathways in the brain that control reward and pleasure.”) were viewed negatively by most raters (100% and 67%, respectively). Messages that invited a “look back” combined with an invitation to change may have been interpreted as accusatory:

“…the reason why I thought they were bad was if they were given to me, not as an option but more as a, ‘why haven’t you done this?’ or pressured upon me, so that didn’t help me at all…so when those options are given to you, as ‘why haven’t you done it?’ or ‘you could try reading or you could try going for a walks or doing something,’ it’s annoying to hear. ‘Have you tried this, have you tried that?’ If I found something by now, I would be doing it.”

Another stated:

“We don’t want to have to keep being reminded of what we did wrong in the past.”

Future-oriented questions that did not imply previous failures (e.g., “How would your life be different if you reduced your alcohol use?”) and statements that focused on autonomy/independence of choice (e.g., “No one can make you change. It’s completely up to you.”) were generally perceived as favorable (67% and 78%, respectively). Choice and options were mentioned often:

“If someone completely didn’t want to change and then they’re having a bad day and it’s one of those self-destructive days where they just go off and they just do a little too much and they get the text, and it’s like, ‘Listen, if you want to change, you can, but the choice is yours,’ then it might get them to actually start thinking.”

Reflecting on the importance of choices, another participant stated:

“Yeah, giving them options without making them feel like they have to choose to be good or everything’s just going to fall apart.”

A third participant indicated:

“What we’re trying to get at and the whole ‘hands-off’ sort of situation is that you’re trying to make it feel like…you’re trying to help people help themselves because that’s all you really can do in that situation.”

Reflections on reasons to change (“What are some reasons you might want to change your alcohol use? What are some reasons you don’t want to change?”) and facts that conveyed support (versus “warnings” above; “Medications are available to help with the physical symptoms you can feel when changing your drinking.”) were rated positively by 67% and 78% of participants.

On the Fence/Contemplation

Considering texts written for individuals who are predominately ambivalent, participants designated at least one good rating to 90% of these sample texts and 57% categorized as good more often than bad (10% neutral, 33% more bad than good). Similar to the results above, future-oriented questions (e.g., “There are lots of benefits of changing - health, $$, social - which are most important to you?”) and statements encouraging autonomy (e.g., “It’s never too late to decide to change your drinking.”) were viewed as most helpful (75% and 76%, respectively). With regard to thinking about the future and results of possible change, one participant commented:

“…for alcohol and weed, I think it would really only work for people like really on the fence, but like the amount of money that they would have if they didn’t smoke would be unreal. You know, so if you smoke every day, at least you’re spending 10 bucks every day and that’s a lot of money when it adds up…”

Asking for professional help (e.g., “You don’t have to do it alone. Counselors can help get you ready to change and support you in your efforts.”) was rated poorly by 67% of participants, as were statements concerning biological consequences (e.g., “Withdrawal symptoms include cravings, irritability, difficulty concentrating, sleep problems, and nausea.”; 71% “bad”). One participant elaborated on this message, in particular:

“I don’t think the one where it’s like, uh, oh yeah, um, what’s it called. ‘Not using drugs can lead to nausea, withdrawal and all these things.’ I’ll tell you one thing, if I was down low on money and I didn’t have any money for drugs and I got a text that’s like, you know when you run out of drugs, you get nauseous and sick, then I’m looking at that text like, damn I do gotta get some more drugs today. I’m running out soon. … If I like look at my phone and I know I’m hurtin’. ‘Oh these drugs, nausea,’ oh yeah, I know it does. I know it does. So let me take some more so I don’t get that way.”

Finally, messages about social support (e.g., “Having supportive friends and family can make the decision to change easier to handle.”) were often reported to not be helpful (80%), with participants indicating caution about negatively impacting those who do not have social supports.

Ready to Change/Preparation

Messages that spoke of advice and social support were rated best with 100% being categorized as good at least once and 73% seen as more helpful than not (13% neutral and 13% more bad than good). Similar to contemplation messages, statements encouraging the use of social support were viewed with ambivalence, as 60% of participants rated these messages positively. One participant indicated:

“These (messages) are all about loved ones. What if people don’t have any? … Some of them (the messages) was like in your face about support groups and don’t think about drinking, but think about people you can talk to, and it’s just like, ‘What if you don’t have someone to talk to?’ That’s the only bad thing about them.”

Another supported this assertion:

“Some people don’t have either (family or friends) so suggesting it would only make a person feel worse knowing there’s no one there.”

On the other hand, people expressed a desire for support:

“I just needed someone to be there for me, you know? I just needed someone to be like, ‘I understand that you have done some pretty bad things in your life, but I’m here to help.’ That probably would’ve helped me out more than anything.”

“If you have someone to cry to, it’s easier because then they can comfort you, they can make you feel better about yourself instead of turning to the other things.”

Planning for the future messages (e.g., “What activities in your life make you want to drink? Make a list and see if you can change your routine to avoid these situations.”) were seen as helpful (64%), as were those with outright advice (67%; e.g., “Do you usually drink while doing certain activities, like eating or watching TV? Try to skip drinking during one of these activities today.”). A marked change from the “stay the same” texts, retrospective messages (e.g., “What situations were difficult for you last time you tried to change? What can you do this time to make it easier on yourself?”) were viewed quite favorably (80%) for individuals looking to make a change.

Discussion

Creating intervention content for a specific population requires sensitivity to that group of people, which should include extensive background knowledge as well as an invitation for those individuals to take part in the process. Using a card sort methodology, youth exiting the foster care system were able to provide direct feedback on the content of text messages to be delivered in a preventive substance use intervention. Overall, feedback on the researcher-created messages was positive and in line with theories the intervention is purported to target (Miller & Rollnick, 2013; Prochaska & DiClemente, 1992). As anticipated, however, participants provided important knowledge about population-specific ways in which messages were not aligned with their experiences, needs, and language.

Specific to messages targeting individuals who are not interested in changing (i.e., Precontemplation) or who are ambivalent (i.e., Contemplation), participants were unanimously in favor of content that encouraged independence, choices, and freedom. Such language is consistent with the MI principle of Autonomy Support, which honors a person’s right to and capacity for self-direction. Somewhat by design, the lives of youth in foster care are governed by the social service agencies charged with their care. At a time when young people are beginning to experiment with independence, it is understandable that such oversight could be met with psychological reactance (Karno & Longabaugh, 2005a, 2005b) and, thus, become an obstacle to engagement in behavior change, especially when professional help is required or suggested. Alternatively, statements encouraging independent decisions leave room for youth to consider such a change.

Messages that referenced previous attempts at change were not rated well, particularly when directed at individuals not interested in behavior change. Strong evidence exists for the relationship between maltreatment and a low perception of self (Appleyard, Yang, & Runyan, 2010; Bolger, Patterson, & Kupersmidt, 1998). Thus, while a reflection on the past may be a worthwhile exercise in the general population, previous unsuccessful attempts could be viewed as failures in the eyes of foster youth and, thus, shaming by researchers. Indeed, statements that invited a look forward, but without a “look back,” were rated most favorably. Messages that promoted the assistance of professional help were viewed poorly, especially when aimed at individuals on the fence about change. Participants had many comments about poor experiences with professionals and individuals in authority positions:

“Yeah, talking with your case worker about certain situations sometimes makes you feel like you’re gonna get into a lot of trouble or it can affect your living situation and that can make you use more or try to block out all the bad feelings that may come from that…”

A second participant continued:

“Sometimes case managers don’t know how to be professional about it. One time, one of my case managers I labeled as PO on my phone, parole officer. I never picked up for her. Sometimes they don’t just know when to step off and just let us see what happens. They can only control so much to the point where they’re gonna push them away into something that they’re not supposed to do.”

Commenting on a previous decision making process, another focus group member stated:

“I didn’t want to go up to someone and say, ‘Hey, I have this problem. Feel free to judge me but can you help me out?’”

In addition to these negative experiences, self-reliance, learned throughout the course of being involved in the foster care system, has been purported to serve as both an asset and barrier to growth among populations experiencing maltreatment (Samuels & Pryce, 2008). Given that our intervention seeks to prevent the development of disorders (and, thus, contact with professional help), references to formal services may be best kept for individual circumstances (i.e., referrals will be made to those seeking assistance) and eliminated from iHeLP text messaging.

Social support, a construct believed to be among the strongest predictors of substance use outcomes (Kelly, Stout, Magill, & Tonigan, 2011; Longabaugh, Wirtz, Zweben, & Stout, 1998), was viewed with ambivalence. In particular, several comments were made about references to utilizing supports, with participants indicating that support is often not available and/or difficult to maintain. These individuals warned that reminders about support could be off-putting to other youth, causing feelings of anger and sadness. Indeed, previous research has suggested that the role of social support in mitigating poor outcomes is not always present among youth who have experienced maltreatment and neglect (Appleyard et al., 2010). Given its prominence in the general literature and the ambivalence expressed by the participants, the role of social support in iHeLP will be examined with great care.

Limitations of this study include arbitrary cutoffs for classification (3+ ratings), lack of representation of the full range of stages of change (i.e., Action and Maintenance), lack of data on participant stage of change, and further detail on why participants chose their rankings. As in any self-report-based study, participants (like most individuals) may not be cognizant of or accurate in predicting determinants of their behavior or engagement with a program. An alternative to our theory regarding social support is, plainly, that this small sample may have been inaccurate or underestimated the importance of social support for themselves, others, or a subset of potential participants. Group discussion about stages of change may have also impacted message ratings. It is possible that, once oriented to the stages, participants may have been more acutely aware of their own place, thereby focusing more of their attention on accurately rating messages written for their own stage and less care and/or ability to accurately rate messages for other stages. Strengths include obtaining perspective about messages that were felt to resonate with members of the target population in that they were perceived as beneficial, whereas offensive or counterproductive messages could be eliminated from the pilot texts in the rollout phase.

Possible future directions include the potential for a computerized card sort as a timesaving extension. Moreover, future intervention developers could build upon this initial work in more nuanced ways that retain all messages, as subpopulations may be attracted to different message content. In so doing, users could select their own messages, possibly sorting by rating or content area (e.g., social support).

In summary, interventions for marginalized and/or underserved populations can be greatly enhanced by collaboration with that group of people, particularly with regard to the creation of intervention content. Without the assistance of youth exiting the foster care system, aspects of iHeLP would have been greatly misinformed at best and offensive at worst.