Clinic Personnel, Facilitator, and Parent Perspectives of eHealth Familias Unidas in Primary Care

Abstract

Introduction

The purpose of this qualitative study was to understand the feasibility and acceptability of implementing eHealth Familias Unidas, an Internet-based, family-based, preventive intervention for Hispanic adolescents, in primary care.

Methods

Semistructured individual interviews with clinic personnel and facilitators (i.e., physicians, nurse practitioners, administrators, and mental health workers; n = 9) and one focus group with parents (n = 6) were audiorecorded, transcribed verbatim, and analyzed using a general inductive approach.

Results

Nine major themes emerged, including recommendations to minimize disruption to clinic flow, improve collaboration and training of clinic personnel and the research team, promote the clinic as a trusted setting for improving children’s behavioral health, and highlight the flexibility and convenience of the eHealth format.

Discussion

This study provides feasibility and acceptability findings, along with important considerations for researchers and primary care personnel interested in collaborating to implement an eHealth preventive intervention in pediatric primary care.

Hispanic adolescents experience significant health disparities related to behavioral health concerns, including drug use and sexual risk behaviors (Kann, Lowry, Eaton, & Wechsler, 2012; Miech, Johnston, O’Malley, Bachman, & Schulenberg, 2015). Relative to their non-Hispanic White and African American counterparts, Hispanic adolescents report a higher percentage of lifetime drug and alcohol use (Miech et al., 2015) and are less likely to report using a condom during their last sexual encounter (Kann et al., 2012). Although preventive interventions for reducing substance use, risky sexual behaviors, and other mental and behavioral health issues (e.g., depression, violence) have shown effectiveness in school and community settings (Hawkins, 1999; Teesson et al., 2014; Wolchik et al., 2002), few interventions have been implemented in the pediatric primary care setting (D’Amico, Miles, Stern, & Meredith, 2008; Patnode et al., 2013; Van Voorhees et al., 2007; White et al., 2010) or have targeted Hispanic youth. To combat existing health disparities, Familias Unidas, a Hispanic-specific, family-based intervention shown to be efficacious and effective in reducing and preventing conduct disorders, drug use, sexual risk behaviors, and internalizing symptoms in adolescents (Prado & Pantin, 2011), has been adapted for use in pediatric primary care settings (Prado, Pantin, & Estrada, 2015). Because of the protective effect of family functioning on multiple behavioral risk factors among youth (Bell, Forthun, & Sun, 2000), adapting and disseminating family-based or parenting preventive interventions from school and community settings for use in primary care settings is necessary to increase the reach and accessibility of prevention efforts.

Pediatric primary care settings are well positioned to provide Hispanic youth with needed access to behavioral health interventions. Given documented difficulties with recruiting, engaging, and retaining Hispanic families within family-based preventive interventions (Coatsworth, Duncan, Pantin, & Szapocznik, 2006), primary care presents a nonstigmatizing, trusted, and accessible entry point (Kolko & Perrin, 2014) for recruiting families. Moreover, in a study of primary care use, Hispanic youth were the racial and ethnic group that visited primary care clinics the second-most frequently, after White youth (Hofer, Abraham, & Moscovice, 2011). The Patient Protection and Affordable Care Act is also attempting to improve public health by increasing access to clinical preventive services in primary care (e.g., behavioral and substance use assessments), initiating several national and community prevention efforts (e.g., U.S. Preventive Services Task Force), and expanding health insurance coverage to more than 4 million uninsured youth (Koh & Sebelius, 2010). Even with a national focus on prevention and primary care, there are currently no effective behavioral problem preventive interventions for Hispanic populations in primary care settings. Because physicians and/or their staff often lack the time and resources to directly deliver preventive interventions, it has been recommended that technology-based interventions be delivered or initiated within these settings to decrease barriers for participants and providers alike (Prado et al., 2015). Flexible and culturally syntonic intervention options, such as the Internet adaptation of Familias Unidas (“eHealth Familias Unidas”), for increased engagement and retention are critical for decreasing health disparities among Hispanics.

Pediatric primary care settings are well positioned to provide Hispanic youth with needed access to behavioral health interventions.

eHealth FAMILIAS UNIDAS IN PRIMARY CARE

Familias Unidas, described in detail elsewhere (Prado & Pantin, 2011), has been traditionally delivered in person by trained mental health workers (“facilitators”) through eight face-to-face parent-only group sessions and four parent and adolescent–attended family sessions. The sessions are centralized around improving family functioning, including parent–adolescent communication and parental monitoring of peers, to reduce and prevent behavioral problems in Hispanic adolescents. eHealth Familias Unidas was recently developed to offer a cost-effective, convenient, and accessible way for families to receive the intervention (Perrino et al., 2016). In collaboration with three pediatric clinics in South Florida, eHealth Familias Unidas was piloted with 48 Hispanic adolescents (age range = 12–16 years) and their primary caregivers. Families were recruited during their pediatric primary care visits and randomized to one of two conditions: eHealth Familias Unidas or treatment as usual. Families randomized to the intervention condition viewed eight mock parent group sessions through a Web site and received four family sessions delivered by one of four trained facilitators (two mental health workers from the clinics and two research staff members) via a videoconferencing program. Mock group sessions consisted of a culturally appropriate telenovela (soap opera) that emphasized key constructs from Familias Unidas (i.e., positive parent–adolescent communication, drug use, risky sex), a group of parents discussing these topics with the facilitator, and interactive exercises.

Overall, embedding evidence-based preventive interventions within primary care offers a novel approach to prevention; however, very little data or recommendations are available regarding implementation best practices. The purpose of this descriptive qualitative study was to examine the feasibility and acceptability of implementing eHealth Familias Unidas in pediatric primary care settings from the perspectives of all those involved in the pilot study (i.e., clinic personnel, facilitators, and parents). The objectives were to (a) determine the acceptability of delivering Familias Unidas via the Internet, (b) identify potential barriers and facilitators to family recruitment and intervention implementation in the primary care clinics, (c) collect recommendations on how to overcome barriers and capitalize on strengths of implementation, and (d) elicit suggestions related to sustainability of the intervention in primary care.

Embedding evidence-based preventive interventions within primary care offers a novel approach to prevention.

Obtaining a better understanding of how well eHealth Familias Unidas was received from multiple perspectives may improve collaborations with clinic stakeholders (i.e., physicians, medical staff, mental health workers), further engage participating families, and increase the efficiency of eHealth Familias Unidas procedures within primary care for a future large-scale randomized controlled trial. Lessons learned from this study may also serve to inform the implementation of other eHealth interventions initiated and/or delivered within the context of primary care settings.

METHODS

Participants and Setting

Purposive sampling was used to recruit participants for this study from two pediatric primary care clinics involved in the eHealth Familias Unidas pilot study. Both clinics are affiliated with the University of Miami Miller School of Medicine. One clinic is a pediatric mobile clinic that delivers health care services free of charge to underserved areas in Miami–Dade County. The other is located on the university medical campus and serves a mix of patients with either private or public insurance. Although three clinics were involved in the eHealth Familias Unidas pilot study, participants from the third clinic were not included in this qualitative study because the third clinic’s cohort had not begun yet, and the research team was seeking recommendations (i.e., results from these interviews) to improve on the implementation of the pilot in the third clinic.

Participants involved in the eHealth Familias Unidas pilot study completed either individual interviews (i.e., clinic personnel and facilitators) or a focus group (i.e., parents). Participants who completed individual interviews (n = 9) included two physicians, one nurse practitioner, two clinic administrators, and four trained Familias Unidas facilitators (one psychologist from the clinic on the university medical campus, one mental health counselor from the mobile clinic, and two research staff members). Facilitators were compensated for their time in delivering the intervention. The first author, who also served as the project coordinator for this study, recruited interview participants via email, and 90% agreed to participate. One nurse was unable to participate because of her busy schedule.

Parents from the two clinics who participated in eHealth Familias Unidas were recruited to participate in the focus group during their baseline assessment. Their assigned facilitators followed up with them during the intervention to let them know when the focus group would take place. Parents were compensated for their time completing the assessments and focus group. Of the two clinics, 50% of the parents who participated in eHealth Familias Unidas participated in this focus group (n = 6; 100% Hispanic women). Two parents completed phone interviews, which were excluded from this article for methodologic purposes, and the other four parents did not provide reasons for not participating in the focus group.

Procedures

This study was approved by the University of Miami Institutional Review Board, and all participants completed informed consent before data collection.

Individual interviews

From December 2014 through January 2015, the first author conducted nine semistructured, face-to-face individual interviews with physicians, a nurse practitioner, clinic administrators, and facilitators. Interviews took place in one of the two aforementioned pediatric primary care clinics or at the university medical campus. The interviewer followed a semistructured interview guide developed by the Familias Unidas research team and took field notes during the interviews. The interview guides (one for clinic personnel [Supplement 1] and one for facilitators [Supplement 2]) were organized to reflect the study objectives including barriers and facilitators to implementation, acceptability of the intervention, and recommendations for future improvement. Example questions included, How was your experience working with Familias Unidas in a primary care setting? and What would facilitate implementation of the intervention? Interviews lasted approximately 30 minutes and were audiorecorded and transcribed verbatim by a professional transcription agency.

Parent focus group

In December 2014, the lead clinical supervisor of Familias Unidas, who had no prior relationship with participants, conducted a focus group with parents at the university medical campus. The leader of the focus group followed a semistructured interview guide used in prior Familias Unidas trials but adapted with questions specific to the primary care study. The interview guide (Supplement 3) was designed to capture parents’ experience with recruitment in the clinic and the eHealth intervention, as well as recommendations for future improvements. Example questions included How was your decision to participate influenced by the health care setting where you receive health care services? and Can you please describe your experiences in the intervention? The focus group lasted 73 minutes and was audiorecorded and transcribed verbatim by a research assistant.

Qualitative Data Analysis

A general inductive approach was used to condense the data into a summary format that met the evaluation objectives of the study (Thomas, 2006). A general inductive approach is a systematic procedure for analyzing qualitative data that allows findings to emerge from the raw data and is commonly used in health sciences evaluation studies (Thomas, 2006). The first author read all study transcripts in detail and created two initial codebooks, each with corresponding decision rules for the interviews and the focus group. Upper-level, or more general, categories for organizing data were derived from the study objectives, and lower-level, or more specific, subcategories were derived directly from participant responses. The second author reviewed the transcripts and codebooks before beginning the coding process to ensure that she understood each respective codebook. These two authors then independently coded study transcripts and met weekly to discuss codebook changes and codes, reach consensus on all discrepant ratings, and organize commonly endorsed themes. All excerpts were organized by code into a table format using Microsoft Excel. Because of the frequency with which similar codes were consistently endorsed across transcripts, it was determined that data saturation had been reached. Percentage agreement was calculated as an indicator of intercoder agreement. The two raters agreed on 76% and 85% of independently coded data for the interviews and focus group, respectively, and reached 100% consensus on all final codes. The developers of the Familias Unidas intervention and a researcher with expertise in qualitative methodology reviewed all study procedures and findings to ensure accuracy of representation of the data. The Consolidated Criteria for Reporting Qualitative Research checklist guided the reporting of study methodology and results (See Supplement 4; Tong, Sainsbury, & Craig, 2007).

RESULTS

Qualitative data analysis generated nine themes: six from the individual interviews and three from the focus group.

Individual Interviews With Clinic Personnel and Facilitators

For the interviews, a total of 65 codes were applied to 317 relevant participant comments. The most frequently endorsed codes were subsequently summarized into six broad themes to best reflect the study objectives. These themes, summarized as follows and with corresponding illustrative quotes in the Table, are framed as recommendations for future eHealth preventive interventions in primary care settings. They include (a) Establishing the Relevance of the Intervention to the Setting and Population, (b) Engaging Clinic Personnel, (c) Minimizing Disruption to Clinic Flow, (d) Improving Collaboration and Training of Clinic Personnel and Research Team, (e) Securing Administrative Support and Funding, and (f) Providing Intervention Facilitators and Participants With Technical and Clinical Support.

TABLE.

Themes and supporting quotes from interviews with clinic personnel and facilitators and parent focus group

Individual interviews
Themes	Supporting quotes
1. Establish the Relevance of the Intervention to the Setting and Population	1.1 “…I think that it’s excellent implementing Familias Unidas in the primary care setting because pediatricians, doctors, they are familiar with the problems that the kids might have…. They are very good reference…to refer families, to benefit from this information.”
	1.2 “I think any time…you try to open various communication with parents with children…it’s definitely a positive thing. So that from a parent as a parent, I think sometimes you don’t know how to talk to a kid. I don’t talk to the mas friend but to be honest with them, and, and let them know we’re not going to hurt them. You know their friends know nothing, most of them. So I always tell my kids, honestly I tell them I said, I’m going to tell you the truth. I think anytime you open up that door to communication it’s a positive experience.”
	1.3 “…the experience has been great. I think Familias Unidas is a great program. I have very good opinions of the family, about the benefit that they obtain, um, when they enroll in the program. So, to me, it’s magnificent, excellent…. It helps so much to the parents, to improving communication. Um, to talk about things as important as prevention of STDs, HIV, and drugs, and, uh, talk also about peer pressure. The importance of the parents to be involved in those aspects of the children’s life, and to know how to communicate with them. So, and the program has been so well prepared, so well organized, that to me, it’s excellent.”
	1.4 “The population’s the best…because they are…coming from countries where there is not the education…. They don’t know about the importance of taking care of themselves. Of preventing, um, drugs, alcohol, um being in sexual relationships without any prevention, without any protection…. So, if there is a population who needs this type of program, it’s this one.”
2. Engage Clinic Personnel	2.1 “…You have to have complete buy-in by the providers. Period. They’ve got to be, um, passionate you know, totally engaged. If they’re not, they’re going, they’re going to say no, we don’t want to do that. So you have to prove the value that you’re offering to the patient, um, and the benefits and once you sign them in, you know, once get them engaged, that I think is the biggest hurdle.”
	2.2 “…I think, as I said before, to me, it’s, you know, I find it excellent. And all the people I have spoken to about it, have given me high remarks, or high, um, comments. The comments were great about how good it is, and how well done it is.”
	2.3 “Enjoyed working with all the staff, you know, all the people, and, uh, I believe in the program and what it stands for.”
3. Minimize Disruption to Clinic Flow	3.1 “The initial engagement with the families in the clinic…was a little difficult. Because at the beginning we didn’t know…the routine of the clinic…I think that once time passes and we were able to know the people at the clinic, we were able to get more help from them…, know the flow,…and understand that we’re not interrupting the flow of the clinic and when to ask, when not ask.”
	3.2 “I don’t think its negative to the actual study but I would say that there was definitely, um, a few instances where we…we and or the staff felt okay there’s a time, you know, something was kind a road blocked, um, or things were taking a lot longer than what we had anticipated…. I think that the particular thing was the number of people that was there at a time was a little bit overwhelming for our facility.”
	3.3 “Yeah, I think that for sure that the physicians felt that it worked well and…it seemed, you know, from start to finish, meaning recruitment, it was definitely one of the quickest. I think that was a testament to the fact that you guys were organized, um, so I think from a physician stand point we didn’t really feel that impeded.”
4. Improve Collaboration and Training of Clinic Personnel and Research Team	4.1 “My only recommendation is…for the staff…that will have direct contact with the recruitment to maybe be trained, and to understand what are we doing, what we’re going to need, what is expected. And I think that that will make everything smoother, easier, and like I said before, we can be on the same page for the families.”
	4.2 “I think an orientation, not too long, just so they understand what’s happening, but then having brief check ins…where they can express concerns, and then also maybe having an anonymous system of competence, competency, or concerns. You know, where they, um, can just say, this has been a problem and if there’s something we can do about it…. Brief check ins how things are going, what are some things we can do, sometimes they are the best source of ideas, how to make it flow better.”
	4.3 “…My only frustration, if, if there was any frustration, would be that…I would like to be involved or at least kept up to date, as to, as to the progress. Even if I receive a progress note once a month or something, saying okay this is what’s happening with your patient. You know, they’ve completed X, Y, and Z, or something for the provider, so that…we know hey, this is a patient that is receiving intervention….We need to know, we need to be informed and in the loop of how things are progressing.”
	4.4 “.It can benefit, yes, to have an orientation and how to really understand the setting. Understand how to approach the families in that setting. Because it’s so fast, the setting is very fast. You cannot be shy…. But I think that for a facilitator in the future that comes to the primary care, I think that, yes, they might need training on the population, how to approach people in the setting. How to engage people in that fast paced setting and also how to work with the staff, that they’re busy and they’re doing their things.”
	4.5 “…In the future, I think that if the doctor really is explaining the program and really recommending the program and saying, ‘Hey, do it. I think that because this and this is happening to you or that you are going to benefit.’ Then, I would say that it would be good for me to go after because now she opened a door for me to go through.”
5. Secure Administrative Support and Funding	5.1 “You know, again, it’s the logistics, it’s you know, having someone who is a coordinator, who’s gonna be kind of overseeing it from our end, ‘cause we can’t add any more responsibilities to our plate, other than that one minute elevator speech, but …the actual… coordinating and so forth, uh takes resources.”
	5.2 “…Just kind of getting a sense of what’s entailed and how long it’s going to take you know how the room is going to be used for and not just necessarily communicating with us because obviously the nurses need to know what’s going on as well so, I think yes I would go back to that point person maybe and just having that one person in the clinic who’s kind of aware what is going on who’s being seen and that kind of thing.”
	5.3 “The funding would be for effort. The time, you know if it’s gonna take time—if I have to be involved with any type of planning or oversight then I have to have, you know I have to have our support for that, depending on how long much we think it’s going to take. If it’s going to take two hours a week, then that has to be built into funding, or if it’s going to take one of our schedulers, say, ten percent of their time.”
6. Provide Intervention Participants and Facilitators with Technical and Clinical Support	6.1 “The minor things…like being able to seeing both the parent and the child and ya know that would usually work that out, the parent would adjust the camera and step back if they needed to, we would be able to see them fine. Sometimes there was unanticipated technical difficulties that would happen and you would have a session scheduled and then it wouldn’t happen because you couldn’t get it done, get it fixed in time and that was just gonna happen sometimes with this format.”
	6.2 “…I had to work a little bit with the family, in the sense of, trying to see them if they were doing it on a tablet. I couldn’t really see them, and I had to be, ‘Okay, put it here, put it there.’ I couldn’t really appreciate seeing what the kid was doing or what mum was saying. I couldn’t hear what they were saying, but I couldn’t really get the gestures and the nonverbal. So, maybe for me to point back to each other. So, that was a little of a challenge, is doing it through the Internet.”
	6.3. “…Clinical supervision…you know, Maria’s great, and Lisa, and they have the experience…and you feel that support. Not only from Maria, and from everybody, from you guys, from Alexa. Everybody was very supporting. If you feel overwhelmed, you will know that there is, you know, a group that will support you. It will stand behind you, and help you, and it will give you a hand.”
	6.4 “…I enjoyed the program, I enjoyed doing, meeting with the families, working with them.”
Focus Group
7. Promote the Clinic as a Trusted Setting for Improving Children’s Behavioral Health	7.1 “…It’s a place where the majority of us, our children have been going there since [they were] young. So then it’s a place where we already have a trust with the doctors, a trust with the nurses. We’ve been there so many years that our kids, and we feel like we can trust them, that we’re uhm…that they talked to us there, and they approached us in that place, it was a place where…we feel good, safe, you know?…But what I’m telling you, it was a place where I feel like I can trust, I feel safe, in peace. That I know I’m going to go there and trust my doctor, the nurses, because I’ve been there for 12 years. So I know what they’re going to, what they’re going to come and tell me, or present to me, it’s something that will be good for my family. Good for my son; not something opposite to that, you know?”
	7.2 “…Because my kids, yeah it was something serious, because my kids have been going to that doctor for 12 years. So then I already have a relationship, trust in that place and I felt, well this is going to be something good for us.”
	7.3 “In my personal case, there wasn’t any influence…. They explained the topic to me, what it was, what we were going to learn, what we were going to learn to help our children and that. Very interesting. I think that the word influence doesn’t fit into what we’re looking at right now. Because influencing, influencing is saying, ‘Do it.’ No, no one influenced us. I think that word doesn’t fit within this context in this moment.”
8. Communicate the Perceived Benefits of the Intervention	8.1 “I had ear contact because I’m listening to him but I’m not looking at him, but it’s like I’m not taking it seriously and as important, so then I say ‘I need to have visual contact, I’m forgetting that.’… Now I look at him, I pay attention, we’re looking at each other. He knows I’m with him, and he’s with me; we’re both there.”
	8.2 “And she had great communication with us, with my daughter. The same with me, with Gretel too. She encouraged her to participate because she didn’t want to talk.”
	8.3 “It motivated me to know a little more about the world of drugs and sex at schools in this moment.”
	8.4 “Well for me, maybe the influence was that right now I have three boys who are a doles-cents. And for me, it’s not the first time I’m a mom and that I’m doing this, so it was very, for me it was very interesting to know how I can do things differently. Not with the son I’ min the program with, because with the son I’m in the program with things are different. But with my second one, uhm he’s a little…I would’ve liked it to be the second one the one to be in the program. But, but for that reason I knew that this was going to help me, not just with one but with the three of them. Especially with my second one, that I’ve been having a little bit of resistance with him.”
9. Highlight the Flexibility and Convenience of the eHealth Format	9.1 “Look, at least for the video you had a couple of weeks to watch it and you could watch it anytime. You could watch the video anytime. After work, at night, at midnight; whatever moment you could watch it, like, that wasn’t something…. The biggest problem were the visits. So then there I think the solution is the facilitator’s flexibility. Like, for that to always be open, to have time to adjust the times; because they also have their own life, personal problems, and so the fact that they can adjust to your needs, honestly that’s…”.
	9.2 “…Flexibility…. To be able to do it like the home visit…. I can imagine that was a success…because…for me, well in two or three occasions I had issues with the schedule, so then I would call Gladys and tell her: “Gladys look I have this problem,” I would send her a text, and then she would reply, “Ok no problem.”
	9.3 “I had many problems in the beginning also, until the third week I could not watch it because where I live…in the center…where they have computers, because I don’t have internet in my house, it was prepaid. Then it was limited, and I wanted to see the soap operas there. But they changed and blocked many things, then for the first three weeks I was trying to watch the videos and I could not. So for me but I resolved it all because I think the program that you have is very good, and I got a tablet.”
	9.4 “I think… when they’re recruiting the first thing that can be asked is ‘Do you have access to the internet? Do you have a computer at home? Do you have a tablet at home?’ Because they’re avoiding all the problems that she already went through, that she expressed for not being able to do it. Because also there can be access but not time.”
	9.5 “So for me, I believe, in my opinion, I think that if you guys talk and think I think it’d be good not only for the kids to be part of the part with the therapist which is difficult, but also watching the soap opera so that they can also see how it plays out.”

Theme 1: Establish the Relevance of the Intervention to the Setting and Population

Clinic personnel and facilitators described the pediatric primary care clinic as an appropriate setting for eHealth Familias Unidas (subtheme 1.1). They believed that the intervention was an important behavioral preventive service to offer in the clinic, noting specifically the relevance of its focus on family functioning (e.g., positive parent–adolescent communication and parental monitoring of adolescent peers) and the prevention of substance use and risky sexual behavior for their patient population (subthemes 1.2 and 1.3). They believed Hispanic families in their clinics would be especially receptive to and would benefit from enrolling in the intervention (subtheme 1.4).

Theme 2: Engage Clinic Personnel

Participants described buy-in from the physicians and others in the clinic as essential for the successful integration and implementation of eHealth Familias Unidas in primary care settings (subtheme 2.1). In fact, participants indicated that buy-in to the intervention was associated with positive responses from clinic personnel regarding multiple facets of the program, including satisfaction with the research team, the research procedures, and the intervention (subthemes 2.2 and 2.3).

Theme 3: Minimize Disruption to Clinic Flow

Regarding intervention implementation, the main barriers to emerge from the interviews were those related to logistics and disruption to the clinic’s flow (subtheme 3.1), including limited space in the clinic to conduct recruitment activities, and lengthy time of recruitment interfering with the family’s primary care appointment. For example, if there were two or three research team members in the clinic room, participants reported the space was too crowded, noting it overwhelmed the clinic staff working there (subtheme 3.2). When providers perceived limited to no disruption to clinic flow, clinic personnel endorsed more positive attitudes toward the implementation of the intervention (subtheme 3.3).

Theme 4: Improve Collaboration and Training of Clinic Personnel and the Research Team

Participants provided several suggestions for increasing the efficiency of intervention implementation. They recommended hosting a clinic orientation in advance of the intervention to strengthen relationships between the research team and clinic personnel and to proactively discuss issues of timing and space to minimally disrupt the clinic flow. Participants believed this orientation would allow physicians, nurses, medical administrators, trained facilitators, and research team members to formally meet and discuss key topics including the scope and rationale of the intervention, recruitment process, and expectations of everyone on the team (subtheme 4.1). Participants also discussed the importance of planning brief follow-up meetings to keep everyone informed on the progress of the intervention (subtheme 4.2). One participant recommended sending general progress reports via e-mail or including specific patient-related progress (subtheme 4.3).

As far as specific training for facilitators and physicians, facilitators recommended more intensive clinical training on how to successfully approach and engage families within a fast-paced clinical setting (subtheme 4.4). Participants described their desire for the physician to be more involved in the recruitment process and suggested that the physicians be trained on how best to introduce the program to the patient’s family before interactions with the research team (subtheme 4.5).

Theme 5: Secure Administrative Support and Funding

Participants suggested that having an assigned clinical administrator as the point person for the intervention would be an integral addition to the implementation of eHealth Familias Unidas in primary care, especially for coordinating logistics (subtheme 5.1, 5.2). For example, the administrator could set up the assigned space for the research team to recruit, or provide materials to physicians to help them in speaking to the patients about the program. In line with administrative support, participants also discussed the need for additional funding to cover the effort/salary of clinic personnel, including physicians, and coordination of the implementation of the intervention (subtheme 5.3). Participants agreed that funding eHealth Familias Unidas through insurance reimbursement would be ideal.

Theme 6: Provide Intervention Participants and Facilitators With Technical and Clinical Support

Because intervention delivery was done outside the clinic via technology, specific questions were asked to the facilitators about their experience with the eHealth format. Aside from reporting minor Wi-Fi connectivity issues (subtheme 6.1), facilitators reported challenges during virtual family sessions such as seeing both the parent and child on the screen, their gestures, and hearing their conversation (subtheme 6.2). Fortunately, facilitators also reported that they usually had immediate support from the research staff if any technical issues did occur and had clinical supervisions in which the Familias Unidas lead clinical supervisor would help them engage families and further explain how to deliver appropriate program content (subtheme 6.3). Overall, facilitators appreciated the experience of delivering the intervention, further supporting the importance of technical and clinical support (subtheme 6.4).

Focus Group With Parents

For the focus group, 16 codes were applied to 78 relevant participant comments. The most frequently endorsed codes were subsequently summarized into three broad themes. These themes are also framed as recommendations for future trials and include (a) promoting the clinic as a trusted setting for improving children’s behavioral health, (b) communicating the benefits of the intervention, and (c) highlighting the flexibility and convenience of the eHealth format. Findings are summarized as follows, with corresponding illustrative quotes in the Table.

Theme 7: Promote the Clinic as a Trusted Setting for Improving Children’s Behavioral Health

Parents reported having a long-standing relationship with their health care providers, which made them feel safe and comfortable when recruited to participate in the pilot program (subthemes 7.1 and 7.2). The family’s trust in the clinic facilitated recruitment and enrollment into the study. When asked if their decision to participate in the program was influenced by the primary care setting, parents reported that “influence” was not the correct word; parents emphasized that they participated in the pilot because of their desire to capitalize on an educational opportunity offered by their clinic to improve both their children’s health outcomes and their family relationships (subtheme 7.3).

Theme 8: Communicate the Perceived Benefits of the Intervention

Parents emphasized the importance of learning from eHealth Familias Unidas about effective parent–adolescent communication skills (subtheme 8.1). Parents reported their facilitators were engaging (subtheme 8.2), and helped them understand their adolescents’ knowledge of drug use and risky sex (subtheme 8.3). Parents also reported that they were able to apply the lessons learned from eHealth Familias Unidas to their daily lives and that these lessons were useful for their relationships with their other children who did not participate in the intervention (subtheme 8.4).

Theme 9: Highlight the Flexibility and Convenience of eHealth Format

Parents reported that they enjoyed the convenience of participating in virtual family sessions and watching group sessions at personally convenient times (e.g., after work) and locations (e.g., at home; subtheme 9.1). Parents were especially thankful for the facilitators’ flexibility in rescheduling family sessions (subtheme 9.2). Overall, parents did not report major technical difficulties with the eHealth format, but one parent reported difficulty with accessing the Internet, but her problem was solved once she was given an iPad tablet by the research team (subtheme 9.3). Parents recommended lending tablets to parents involved in the study as a way of overcoming this barrier. Additionally, parents suggested that better screening about Internet connectivity and explanation of the access needed during recruitment could help overcome this barrier (subtheme 9.4) Parents also recommended that, to augment adolescents’ experiences with the eHealth intervention, adolescents should also watch the online group sessions, instead of participating only in the family sessions (subtheme 9.5).

DISCUSSION

The purpose of this descriptive qualitative study was to better understand the feasibility and acceptability of implementing eHealth Familias Unidas in pediatric primary care settings from the perspectives of all those involved in the pilot study. The objectives were to (a) determine the acceptability of delivering Familias Unidas via the Internet, (b) identify potential barriers and facilitators to family recruitment and intervention implementation in the primary care clinics, (c) collect recommendations on how to overcome barriers and capitalize on strengths of implementation, and (d) elicit suggestions related to sustainability of the intervention in primary care. Overall, eHealth Familias Unidas was perceived as feasible and acceptable across participants, as evidenced by the clinic personnel engagement, parents’ trust in the clinic, and facilitators’ and parents’ positive experience with flexibility and accessibility in delivering and receiving eHealth Familias Unidas, respectively. In accordance with study objectives, nine emergent themes were framed as recommendations for clinic personnel and researchers interested in collaborating on eHealth preventive interventions, including ways they may overcome potential recruitment- and implementation-related barriers and, ultimately, sustain these interventions long term. Recommendations include (a) establishing the relevance of the intervention, (b) engaging clinic personnel, (c) minimizing disruption to clinic flow, (d) improving collaboration and training of clinic personnel and research team, (e) securing administrative support and funding, (f) providing intervention facilitators and participants with technical and clinical support, (g) promoting the clinic as a trusted setting, (h) communicating intervention benefits, and (i) highlighting the flexibility of the eHealth format.

The first two objectives of the study were to determine the acceptability/feasibility of eHealth Familias Unidas and determine barriers and facilitators to intervention recruitment and implementation. Findings indicated that pediatric primary care clinics are appropriate and trusted settings to recruit Hispanic families for eHealth Familias Unidas. Comments from clinic personnel, facilitators, and parents suggested that they strongly believed in the importance of drug use and risky sexual behavior prevention for Hispanic youth, highlighting their openness to delivering and/ or receiving preventive services in the clinics. Findings also showed the importance of engaging clinic personnel early in the implementation process so that the intervention is perceived as enhancing rather disrupting the daily clinic activities or flow. Previous studies have shown the importance of overcoming factors that disrupt clinic flow, such as lengthy time needed to recruit or limited space available. For example, an Internet-based intervention for prevention of adolescent depression showed that a busy practice schedule, limited practice time, and negative provider attitudes negatively affected recruitment of adolescents (Eisen et al., 2013). In the present study, although limited time and space were reported, parents’ perceptions of the clinic as a trusted setting combined with clinic personnel’s positive attitudes may have helped the research team reach the target number of families. Specifically, our high enrollment rate of 93.5% is a testament to the clinic as a nonstigmatizing environment to not only address physical health, but also mental, behavioral, and familial health. This impressive enrollment rate is in contrast to a previous Familias Unidas trial conducted in the public school system that had an enrollment rate of 50%. Because it has historically been difficult to recruit families into randomized controlled trials (Baker, Arnold, & Meagher, 2011), the enrollment rate from this study provides evidence for the acceptability of family-focused, evidence-based interventions in primary care (Leslie et al., 2016).

To fulfill the third study objective related to collecting recommendations for overcoming barriers and capitalizing on strengths of implementation, concrete suggestions for increasing the efficiency of study procedures through increased collaboration were provided by study participants. Results highlighted the need to improve collaboration and training between clinic personnel and the research team to facilitate a more fluid implementation process. Specifically, clinic personnel and facilitators suggested incorporating an initial clinic orientation and ongoing communication with the research team through e-mail and brief follow-up meetings. The initial clinic orientation would include information on the overall structure and aims of eHealth Familias Unidas, the clinic’s flow, and recruitment procedures. A previous study of participant recruitment in primary care has noted that extensive training of recruiters (e.g., clinic personnel or research team) through didactic presentations and role playing strategies for rapport-building is essential to the implementation process (Felsen, Shaw, Ferrante, Lacroix, & Crabtree, 2010). Another study of a parenting program in primary care showed that, after training, providers’ report of higher self-efficacy was positively associated with the implementation of the evidence-based intervention (Turner, Nicholson, & Sanders, 2011). Researchers have noted that although training is necessary, however, it is insufficient (Felsen et al., 2010). Others have articulated the need for multiple levels of the clinic hierarchy (i.e., from administrators to physicians to higher level organizational management) to work together to create a collaborative clinic culture (Bitar, Springer, Gee, Graff, & Schydlower, 2009). A survey of a nationwide sample of mental health service providers showed that more practiced organizational cultures, paired with more engaged and less stressful work climates, were associated with positive provider attitudes toward adoption of evidence-based practices (Aarons et al., 2012). Through the recommendations of a clinic orientation, brief follow-up meetings, and electronic progress reports, the aforementioned collaborative culture will help overcome barriers of implementation, create an engaged and organized team, and increase the efficiency of future research studies involving a behavioral intervention in primary care.

As discussed, several recommendations must be enacted before the delivery of the intervention. For the intervention itself, it is imperative that the actual delivery of the intervention complements the clinic’s ongoing duties and engages the families. Thus, the convenience of the eHealth format, which streamlines intervention delivery to occur outside the clinic, is a selling point, not only for the clinic, but also for the parents who prefer high flexibility and accessibility of the intervention. The research team also must provide the intervention facilitators with technical and clinical support to deliver the intervention, so that families, in turn, can fully engage and participate in the intervention. Parents completed 89% of the intervention sessions, emphasizing the high engagement they had with their facilitators. On average, free-standing Internet-based health and behavior change interventions experience 30% to 50% of participants visiting Internet sites, and most are used for less than 10 minutes (Christensen & Griffiths, 2002; Santor, Poulin, LeBlanc, & Kusumakar, 2007; Van Voorhees et al., 2009). The aforementioned eHealth Familias Unidas trial in the school system experienced a 74% completion rate (Perrino et al., 2016). Our 89% completion rate speaks to the promise that eHealth Familias Unidas in primary care has for not only enrolling families but also retaining families in the intervention.

The final objective of this study was to elicit suggestions regarding the sustainability of preventive (behavioral) interventions to promote adolescent behavioral health within primary care settings. Participants noted that the sustainability of the intervention may be most successful with additional administrative support and funding. Indeed, research in primary care settings shows that the most effective participant recruitment strategies included involvement of a designated clinic employee to help the research team, having simple patient inclusion criteria, and patient incentives (Eisen et al., 2013; Ruud et al., 2013). Although the present study used patient incentives and simple inclusion criteria, the recommendation to secure and clearly designate administrative support should be incorporated in future trials. Further, research on behavioral health interventions in children’s health care indicates that reimbursement and fiscal sustainability were main concerns for health care providers and researchers, and these need to be addressed before behavioral health interventions undergo wide-scale dissemination (Kolko & Perrin, 2014). Unfortunately, even with the implementation of the Patient Protection and Affordable Care Act and the health system shift toward accessible preventive services, not all preventive services are supported by the U.S. Preventive Services Task Force and are not reimbursable by health insurance. For funding sustainability, we must continue to secure the evidence on the prevention of adolescent substance use and other behavioral problems, such as testing eHealth Familias Unidas, so that the U.S. Preventive Services Task Force can recommend this as a preventive service that can be reimbursed via health insurance in primary care.

Strengths and Limitations

This study has several notable strengths and limitations. A main strength of this study is that we followed a structured and rigorous approach to data analysis, including the use of two independent raters. Numerous recommended validation strategies were also used to enhance the accuracy of reported findings (Creswell, 2012). These included both triangulation (i.e., using multiple sources for data collection) and peer debriefing (i.e., obtaining feedback from both intervention developers and a researcher with expertise in qualitative research methods). A limitation was that we had a small number of participants for the interviews and focus group because of the amount of people available from which to sample. In addition, because the person who conducted the individual interviews and led the analysis was also the project coordinator for the study, her presence may have increased participants’ social desirability in responding or introduced bias into the study. However, given the readiness of participants to provide critical feedback and suggestions, we hypothesize that the existing working relationship between the interviewer and the participants actually enhanced participant honesty.

CONCLUSION

Given that the implementation of preventive interventions for adolescents in pediatric primary care settings is a relatively novel approach, these findings offer encouraging evidence for primary care settings as a natural entry point for recruiting and retaining Hispanic families into evidence-based, behavioral problem preventive interventions. The strategies and recommendations put forth from the participants are an indication that not only is it possible to integrate and implement an eHealth intervention in a primary care setting, but it is also acceptable and needed. Some results from the interviews were immediately integrated into the pilot study for the third clinic (e.g., an initial clinic orientation before patient recruitment), showing how easily logistical barriers can be overcome with clearer communication and greater integration of the primary care team. Recommendations gathered from the pilot and qualitative study will also inform the implementation of a larger clinical trial in a primary care setting.

In summary, this project provides valuable recommendations for the Familias Unidas team, and practical considerations (e.g., is the target population and setting appropriate for the intervention?) and concrete steps (e.g., electronic progress reports) for researchers interested in collaborating with primary care personnel to implement eHealth interventions in pediatric primary care settings. Future research must consider the integration and intersection of quantitative and qualitative data collection and analysis to maximize the efficiency of intervention implementation and dissemination in primary care, particularly as it affects patient outcomes.