Proof-of-Concept Trial of a Tablet-based Program in Pediatrics to Motivate Parental Use of an Online Behavioral Parent Training Program

Abstract

Internet-based parent training is a promising intervention approach for child disruptive behavior. However, engagement in these interventions is limited. The Parenting Young Children Check-up (PYCC) was designed to improve engagement in internet-based parent training programs via three components: 1) an initial check-up, 2) text messages, and 3) a website. This proof-of-concept trial used feedback from parents and pediatric clinic staff to evaluate feasibility as well as the extent to which the initial check-up was associated with behavioral intentions to use the PYCC website. Pediatric staff and parents rated the PYCC highly, and parents reported interest in using the PYCC website.

Disruptive behavior problems (DBPs) in early childhood are prevalent and undertreated (Lavigne et al., 2009). While some challenging behavior in early childhood is typical, children with DBPs exhibit behaviors such as aggression, hyperactivity, non-compliance, defiance, and emotion dysregulation (e.g., tantrums, irritability) more than what is expected for their age. While ODD and ADHD each affect over 10% of preschool-aged children, only 3% of young children with any emotional or behavioral disorders receive treatment (Lavigne et al., 2009). DBPs in early childhood place children at risk for a series of negative outcomes like conduct disorder, school failure, criminality, and substance use in adolescence (Patterson et al., 1992; Moffitt, 1990; Shaw et al., 2000). Beginning around the toddler years, parents and disruptive children can fall into a pattern of coercive behavior in which children’s challenging behavior leads to heightened levels of parental harshness and control, which inadvertently intensifies children’s disruptive behavior (Shaw et al., 2000; Chang & Shaw, 2016). Intervening early at the level of parenting, including building a strong parent-child relationship and teaching positive discipline practices, is critical in promoting best outcomes for young children with DBPs.

Behavioral parent training is an evidence-based approach to promoting positive parenting strategies, building a strong parent-child relationship, and reducing children’s DBPs. Multiple behavioral parent training programs have overlapping content, such as teaching parents to use positive reinforcement and a time out discipline procedure, and numerous investigations show these programs are efficacious and effective (e.g., Day et al., 2021; Phillips & Mychailyszyn, 2023; Webster-Stratton et al., 2013; Zisser-Nathenson et al., 2008). While this approach to intervention for DBPs clearly benefits children and families, there are multiple barriers to parents finding, engaging in, and completing these face-to-face interventions (McGoron & Ondersma, 2015). For example, waitlists and provider shortages are common, and practical barriers, such as lack of transportation, make completing face-to-face BPT a challenge.

Promising innovative approaches to expand the reach of BPT are emerging, including delivering interventions through the internet and in settings parents already use, like pediatric primary care (Comer et al., 2017; Day & Sanders, 2018). In effectiveness trials, research shows that these approaches to delivering behavioral parent training have a similar impact as traditional, face-to-face delivery (Florean et al., 2020). For instance, Sanders, Baker, and Turner (2012) found strong support for the online version of the Positive Parenting Program (Triple P) in a randomized clinical trial with parents of children ages two to nine years old with early onset conduct problems. Parents randomized to the online intervention, rather than a waitlist control, had better outcomes in terms of reported child behavior and parenting practices. While accessible delivery is becoming commonplace, the need to proactively engage parents in behavioral parent training is still unmet. Parents often do not recognize the need for help with child DBPs nor do they actively seek help (Johnston & Burke, 2020; McGoron & Ondersma, 2015). Parents of children with DBPs may need help recognizing the need for assistance, support being directed to a behavioral parent training program, and efforts to enhance motivation to engage in the intervention.

The Parenting Young Children Check-up

The Parenting Young Children Check-up (PYCC; see overview in Figure 1) is a technology-based program being created and refined to meet the need for a high-reach approach to behavioral parent training. It is unique from other online behavioral parent training programs as it includes a built-in component to proactively engage parents and is designed specifically for use within a setting they already use: pediatric primary care. Program development is following the ORBIT (Obesity Related Behavioral Intervention Trials) model, a human-centered translational intervention development approach that was originally developed for obesity-focused programs but has been used more broadly for behavioral interventions (Czajkowski et al., 2015). The ORBIT model has four phases spanning from intervention design to effectiveness trials. Early phases of intervention creation encourage input from key stakeholders, refining the intervention, and returning to early phases of program development as needed. Once an intervention package is complete, investigators are able to move on to preliminarily testing the intervention in a controlled setting with a small number of participants.

Figure 1.

Overview of the PYCC Shown to Clinic Staff and Parents

The PYCC is designed to begin with screening for child DBP as part of a pediatric primary care visit. Parents that report elevated DBPs go through a brief, tablet-based program, called the initial check-up. Incorporating input from parents and pediatricians as part of the ORBIT-model phase 1 development (McGoron et al., under review), the initial check-up was designed to give parents feedback about their child’s behavior, orient them to the important role parents play in promoting children’s best behavior, introduce them to the PYCC, and motivate them to use the PYCC parent training content. Then, parents receive text messages that include links to the parent training content through a program website. While parents were involved in the early phase development of PYCC initial check-up, it is unclear if the initial check-up meets a key intended goal of motivating parents who were reached within a pediatric primary care setting to use the PYCC parent training website.

The Present Investigation

The present investigation, a proof-of-concept trial (ORBIT model phase 2a), presents one step in PYCC development and refinement. A proof-of-concept trial is a preliminary investigation to determine if further investigation of an intervention is warranted (Czajkowski et al., 2015). Proof-of-concept testing is done before pilot testing to see if intervention elements are capable of producing a desired outcome. Small sample sizes are typical in such designs and statistically significant results are not the end goal. This early phase testing is also consistent with literature on technology development, which indicates that five users is generally sufficient for early testing (Turner, Lewis, & Nielsen, 2006).

This investigation was guided by the Technology Acceptance Model (TAM), a theoretical framework that was first outlined by Davis (1986) and empirically validated by Davis et al., (1989) to understand resistance to, or use of, computer systems. The TAM was influenced by the theory of reasoned action and the theory of planned behavior, which placed a person’s attitude and subjective norms as important determinants of consciously intended behavior. The model has subsequently been evaluated, applied, and adapted for nearly forty years (see Gupta et al., 2022; Holden & Karsh, 2010; Marangunić & Granić, 2015, for reviews). TAM posits that ease of use and perceived usefulness influence intentions to use a technology-based application, and intentions are an important predictor of actual technology use. In initial work to validate the TAM, Davis et al. (1989) found that after an hour orientation to computer use, behavioral intention predicted actual computer use 14 weeks later. Ease of use and perceived usefulness impacted actual use through behavioral intentions to use the program. More recently, the TAM has been applied to many technology-based applications, including demonstrating its utility in health care settings (Holden & Karsh, 2010) and in the development of mHealth educational content for parents (Mobley et al., 2023).

The focus of the current investigation, as guided by the TAM, was to establish proof-of-concept and feasibility of the PYCC initial check-up, which is completed after screening for DBPs, during a pediatric primary care visit. We anticipated that going through the initial check-up would be followed by behavioral intentions to use the PYCC parent training website and high levels of reported ease of use and perceived usefulness. We also assessed if parents rated the PYCC favorably and allowed for parents to provide open-ended feedback on how to improve the intervention. Additionally, given their critical role in PYCC implementation, we elicited feedback about the intervention and recruitment plan from pediatric clinic staff prior to meeting with parents. This investigation was done while the PYCC website was being finalized and aimed to guide a future pilot randomized clinical trial (ORBIT model phase 2b) on the impact of the initial check-up on actual program use.

Method

Participants

This project took place at a pediatric primary care clinic in a large, Midwestern city. All procedures were concurrently approved by the IRBs at the healthcare system and the university overseeing the project. Participants were staff (physicians, nurses, medical assistants, and front office staff) at the pediatric primary care clinic and parents of children ages two to five years old who were bringing their child in for a pediatric visit.

Procedures

The project began when the project principal investigator (1^st author) and a research assistant met with clinic staff to introduce the PYCC, review data collection procedures, and elicit feedback about the procedures, the screener selected, and the initial check-up (i.e., the parts of the PYCC program that are done during a pediatric visit). The overall goal of this meeting was to promote partnership and buy-in from pediatric staff. Following this meeting, research assistants began working shifts at the pediatric clinic to be available for possible parent participants.

Staff Participants.

Staff members that chose to participate met with the research team over the lunch hour; the research team provided lunch. The meeting began with introductions and the research team described the PYCC intervention; Figure 1 was printed out and used as a visual aid to walk staff through each part of the program. Next, staff members viewed the screening that would be used for the project (i.e., the Eyberg Child Behavior Inventory [ECBI]; Eyberg & Pincus, 1999) and then were shown the initial check-up. Staff had the opportunity to ask questions, voice concerns, and make suggestions. Staff completed a very brief, anonymous survey at the end of the meeting and received $30 on a debit-like card.

Parent Participants.

Parents learned about the PYCC project, including completing a screener, from a pediatric clinic staff member who gave out a flier about the study. This was primarily done by the front office staff, but medical assistants, nurses, and physicians were also able to direct parents to the research opportunity. Parents in this clinic were accustomed to completing developmental screeners as the clinic already used the Ages and Stages Questionnaire as part of their clinic procedures at the time of the project. Parents were able to participate at any point during the pediatric clinic visit (e.g., before or after their child’s appointment). Parents that expressed interest in participating met with a research assistant to complete the ECBI as a screener for this project. Parents that reported elevated DBPs (i.e., a t-score of 55 or above) were asked to participate in the proof-of-concept trial and those that agreed completed a consent form. Parents were initially given a children’s book as a thank you for doing the screener, but at the suggestion of pediatric staff and after receiving IRB approval, this was later changed to five dollars.

Once enrolled, parents completed the initial check-up on a tablet while still at the pediatric visit. This took approximately ten minutes. The initial check-up is a web-based app through the Computerized Intervention Authoring System (CIAS) that includes an animated narrator (i.e., a green cartoon bird), which talks throughout most of the initial check-up, and motivational content to promote use of the PYCC parenting website. The check-up starts with the narrator telling parents about “strong-willed children,” (a term used by parents in the initial development of the PYCC; McGoron et al., under review), emphasizing the importance of parents in promoting positive outcomes for strong-willed children, and introducing and showing a video of a teacher talking about the special needs of strong-willed children. Parents are then asked if they would like feedback on the screener they completed. If they select “yes,” then they are shown a chart indicating that they reported more challenging behavior than average, suggesting their child may be “strong-willed,” (the chart is skipped for parents who say “no”). Through the program, parents are then told about and shown examples of the other parts of the PYCC (i.e., text messages and a list of all of the modules included on the PYCC parent training website). Next, parents are asked about their interest in using the parent training website; if they select “no”, then they see further motivational content that includes having them think about benefits and challenges to using the PYCC parent training website.

At the end of the initial check-up, while still on the tablet, parents were reminded about the components of the PYCC program and asked survey questions. While the primary objectives of this investigation were to determine feasibility of the initial check-up in pediatrics and assess parents’ behavioral intentions to use the PYCC website, parents were also asked about their perceptions of the screening questionnaire, text messages, and parent training website. Next, the research assistant asked parents brief qualitative questions about the PYCC (using Figure 1 as a reminder about elements of the program as needed), which were audio recorded. Parents were compensated $25.

Measures

Office staff feedback survey.

At the end of the staff meeting, staff were asked to complete a very brief survey to provide feedback on the screening and initial check-up (the two components of the PYCC completed in the primary care office). Specific questions are in Table 2. Responses were rated on a 5-point Likert scale with responses ranging from ‘definitely disagree’ (0) to ‘definitely agree’ (4). Means, standard deviations, and % of participants that agreed with each item are reported. Additionally, staff were asked if the screener should be done on paper or on a tablet.

Table 2.

Clinic Staff Ratings (N = 15)

	M(SD)	Min.-Max.	% That Agree*
Screening
This screening appears easy for parents to complete while waiting at a primary care visit.	3.1(.6)	2–4	86.7%
Completion of the screening will be useful for many parents.	3.1(.5)	2–4	93.3%
Results of the screening will be useful for providers.	3.1(.7)	2–4	80%
The screener could fit into our office flow.	2.8(.7)	2–4	66.7%
Brief Tablet-based Program
The tablet-based intervention appears easy for parents to go through while waiting at a primary care visit.	2.9(.7)	1–4	86.7%
Completion of the tablet-based intervention will be useful for many parents.	3.2(.7)	2–4	86.7
Parents’ completion of the tablet-based intervention will be useful for providers.	2.9(.7)	2–4	73.3%
This tablet-based intervention could fit into our office flow.	2.9(.7)	2–4	73.3%

^*Agreement considered a score of 3 or 4, corresponding to responses of “Agree” or “Definitely Agree” on the feedback survey

Note: The “tablet-based” intervention is referring to the initial check-up

Behavioral intentions to use PYCC parent training website.

Parents’ behavioral intentions to use the parent training website were assessed in three ways. First, embedded within the initial check-up, parents were asked “Are you interested in using the website to watch videos that teach parenting strategies?” with options of “Yes,” and “No.” Second, one survey item elicited parents’ interest in using the PYCC parenting website (see Table 3 for exact question). Third, as part of the structured interview, parents were asked “Would you use the Parenting Young Children Check-up website?” and then encouraged to elaborate (i.e., “why would [wouldn’t] you use it?”). Responses were transcribed, extracted, and organized into a table.

Table 3.

Parent Survey Responses About Each Part of the PYCC (N = 6)

	M(SD)	Min.-Max.	% That Agree*
Perceived Ease of Use
The screening appears easy to complete while waiting at a primary care visit.	3(1.5)	0–4	83.3%
The tablet-based program appears easy to go through while waiting at a primary care visit.	3.7(.5)	3–4	100%
The text messages will be an easy way for me to get information and access the website.	3.5(.05)	3–4	100%
The parent information website appears easy to use.	3.3(.5)	3–4	100%
Perceived Usefulness
Completion of the screening will be useful for me as a parent.	3.7(.5)	3–4	100%
Completion of the tablet-based program will be useful to me as a parent.	3.7(.5)	3–4	100%
It appears the text messages will have useful information.	3.7(.5)	3–4	100%
The information of the parent information website appears it would be useful to me.	3.3(.5)	3–4	100%
Likability
I like the idea of doing this screener at primary care visits.	3.6(.5)	3.4	100%
I like the idea of doing the tablet-based program at primary care visits.	3.6(.5)	3–4	100%
I like the idea of getting information through text messages.	3.6(.5)	3–4	100%
I like the idea of using the parent information website.	3.5(.5)	3–4	100%
Behavioral Intentions to use the PYCC
I would use the parent information website.	3.2(.8)	2–4	83.3%

^*Agreement considered a score of 3 or 4, corresponding to responses of “Agree” or “Definitely Agree” on the feedback survey

The “tablet-based” intervention is referring to the initial check-up

Parent feedback survey on ease of use, perceived usefulness, and likeability.

Items on the parent feedback survey were rated on a 5-point Likert scale with responses ranging from ‘definitely disagree’ (0) to ‘definitely agree’ (4). Specific items can be found in Table 3. Means, Standard Deviations, and % of participants that agreed with each item are reported.

Parent qualitative feedback and suggestions for program improvement.

After parents completed the initial check-up, they were asked structured interview questions. Research assistants read the items verbatim and, as needed, followed them up with prompts like “tell me more about that.” Specific interview questions were 1) “What are your thoughts about the program you just went through?” (i.e., the Initial Check-up), 2) “What did you like about the program?” 3) “What did you not like about the program?” 4) “Tell me your ideas about how we can make the program better?” 5) “What would you think about getting texts from the program? Interviews were audio recorded and transcribed verbatim by project research assistants. Specific quotes were then extracted and organized into a table reflecting the codes “Likes” (i.e., what did the participant like about the program), “Dislikes,” (i.e., what did the participant dislike about the program), “Suggested Changes,” and “Thoughts on receiving text messages.”

Observations of clinic flow and recruitment barriers.

Research assistants logged notes during each of their shifts recruiting parents at the pediatric clinic. Research assistants were instructed to make notes about any observations that influenced recruitment, comments from staff about the project, and any challenges they encountered. Notes were reviewed by the project PI, and a list was generated of observations made about clinic flow and recruitment barriers.

Results

Pediatric Staff Participants

Fifteen pediatric staff members learned about the PYCC and completed a feedback survey (all staff members present at the clinic on the day of this data collection participated). All pediatric staff participants were women, 7 (46.7%) reported being Black/African American, 3 (20%) reported being Asian, 2 (13.3%) reported being White, and 3 (20%) did not report their race.

Feedback survey.

The majority of pediatric staff “agreed” or “strongly agreed” with survey items, suggesting an overall positive impression of the proposed screener and the PYCC initial check-up. For all survey items, however, at least two staff participants responded “unsure,” suggesting at least some skepticism about if the program would be feasible. The survey item with the lowest average rating was, “The screener could fit into our office flow.” Additionally, one participant selected “disagree,” in response to the item “The tablet-based intervention appears easy for parents to go through while waiting at a primary care visit.” Fourteen participants answered the question about how parents would complete the screener: 9 (64.2%) selected tablet, 3 (21.4%) selected paper, and 2 (14.3%) selected both tablet and paper.

Parent Participants

Data collection with parents was done over a four-month period during which research assistants were present at the clinic part time. All parents that reported interest in participating completed the screener. Of 30 parents who completed the screener (see demographic information in Table 1), eight parents were eligible to participate in the proof-of-concept trial based on inclusion criteria (i.e., they reported that their child had elevated DBPs). A ninth parent who reported elevated DBPs was ineligible because they reported that their child had autism spectrum disorder, which was a criterion for exclusion. Six of the eight eligible parents agreed to participate in the proof-of-concept trial. The two parents who declined indicated that time constraints led to their decision (i.e., they did not participate until the end of their pediatric visit and had another appointment to get to). Demographic information for all parents screened (N = 30) and parents in the proof-of-concept trial (N = 6) is presented in Table 1. All parents were mothers who identified as Black/African American (3 participants also reported being of multiple races); data collection took place in a major city in the United States in which census data indicates that approximately 80% of residents are Black/African American.

Table 1.

Parent participant Demographics

	Entire Screening Sample (N = 30)		Proof-of-Concept Trial Sample (N = 6)
	n (%)	M(SD)	n (%)	M(SD)
Parent Race and Ethnicity *
Black/African American	27(90)	----	6(100)	----
White	4(13.3)	----	1(16.7)	----
American Indian or Alaskan Native	1(3.3)	----	1(16.7)	----
Asian	1(3.3)	----	1(16.7)	----
Parent Sex
Female	27(90)	----	6(100)	----
Male	3(10)		0(0)	----
Parent Age		31(6.4)		27.8(4.4)
Child Sex
Male	15(50)	----	2(33.3)	----
Female	15(50)	----	4(66.7)	----
Child Age
2 years old	9(30)	----	1(16.7)	----
3 years old	9(30)	----	4(66.7)	----
4 years old	9(30)	----	0(0)	----
5 years old	3(10)	----	1(16.7)	----
Does your child/family receive Medicaid?
Yes	22(73.3)	----	5(83.3)	----
No	8(26.7)	----	1(16.7)	----

^*Results exceed 100% as some parents selected more than one race; only selected races and ethnicities are included

Intentions to use the parent training website.

As part of the initial check-up, all 6 parents who participated in the proof-of-concept trial (100%) responded yes to the question “Are you interested in using the website to watch videos that teach parenting strategies?” On the feedback survey, 5 parents responded “Agree,” or “Strongly agree” and 1 parent responded “Unsure” for the question about intentions to use the parent training website (see Table 3). In the structured interview questions, all parents (100%) responded that they would use the parent training website (see Table 4).

Table 4.

Qualitative Interview Feedback from Proof-of-Concept Trail Participants (N = 6)

Likes:

“The teacher was good, and the description fit my kid. Like she looks like someone I trust, and the info she gave was good to know. Also, I like that I could go back, like you know when we were interrupted by” “How they focus on the child, but they also focus on the parent as well” “I like that it’s easily accessible to my phone and I don’t have to go make an appointment anywhere, I could just do it at home or on the go. But I like that.” “I liked what it- the information- that it told me that it will make available to me” “I like that they make videos to help you with parenting stuff. Especially if you have your issues. I did hear that in there. Like, what’d they consider them, strong willed? Yes, that just means another word for that. But yes, that’s him, strong-willed.” “That it kind of gives you an idea of what to do with strong-willed children because I have two of them. One is nine so”

Dislikes:

“The bird was a bit annoying and also the background looks a bit boring. Oh, it be nice to have the words on the screen, there could be people who may have hearing problems, like I have hearing problems but not that bad, so I could still understand” “Nope, not that I know of, no” “Honestly, it’s nothing I didn’t really like. Yeah, there is nothing I didn’t necessarily not like” “I don’t think there was anything I didn’t like. I mean, I would have to go through the program obviously to see how useful it would be.” “It wasn’t anything that I didn’t like about it.” “It wasn’t anything that I didn’t like about it.”

Changes:

“Nothing other than the captioning” “I don’t know, maybe use a different voice for the Parrot, other than that nothing else.” “I mean, I guess it’s at a primary care visit so there’s only so much information you can give in such short time, but maybe provide a pamphlet.” “None right now” “I think that it’s only online, so like, doing actual classes where we could meet. “ “Nothing”

What would you think about getting text messages from the program?

“It be cool, as long as they don’t spam me, lord knows I need some parenting skills with this one. Like not more than 2 or 3 times a week” “I’ll like that” “I think it’s good. I’m already signed up to a couple text services so it would helpful. I might you know just get a text that will pop up on my phone and remind me to keep working on it with her” “I wouldn’t mind” “That’s smart, because that’s the way that a lot of people nowadays are communicating. “ “It’s convenient, very convenient. I text everybody so-”

Would you the Parenting Young Children Check-up website?

“I’ll check it because it seems to have some stuff I haven’t tried yet like consequences, I feel like I am too soft on them or I am over the top, no middle ground for me. Who knows it might be helpful” “Yes, I would use it and I would because like said I’m looking for better ways to, you know, handle my kids instead of just trying to be aggressive with them or you know yell, or you know spank them or anything like that. I’m trying to take different measures, so yeah” “Yes, I would because I really need help controlling my kid” “I would. As long as it has useful information, yeah, of course” “Yes. I would because there’s a lot of helpful stuff on there and a lot of resources that could help me.” “Yes. I would because it’s just- it’s easy to use and you get like useful information.”

Feedback survey.

Parents who participated in the proof-of-concept trial (N = 6) rated the parts of the PYCC program highly in terms of perceived ease of use, perceived usefulness, and likability. The only exception to this was that one parent selected “strongly disagree,” for the item “The screening appears easy to complete while waiting at a primary care visit.” Importantly, all ratings of the elements of the PYCC (i.e., the brief intervention text messages, and parent training website) received a response of “agree,” or “strongly agree,” suggesting strong, positive perceptions of each part of the PYCC. Results are summarized in Table 3.

Feedback from structured interviews.

Extracted quotes from the structured interviews are found in Table 4. All parents reported things that they liked about the program. Only one parent reported things they did not like about the program—these were about the animated narrator (i.e., “the bird”) and it was suggested to change the voice of the animated narrator and add captioning. Additionally, one parent suggested adding a pamphlet about the program. All parents were open to receiving text messages from the program with positive comments including that texts were “convenient,” and “smart.”

Observations.

Research assistants made several notes that were important in understanding clinic flow and barriers to project recruitment. First, it was noted that there were both slow days and very busy days at the clinic. Slow days were marked by there being a low number of children (sometimes none) in the target age range scheduled for appointments or high rates of no shows/cancelled appointments. Busy days (or times) included many appointments in a short period of time, which made it challenging for staff to have time to inform parents about the study. Second, there were times when parents were interested in participating but did not have enough time due to learning about the opportunity later in the visit. Third, research assistants noted the need to remind clinic staff to tell parents about the study (this challenge was reduced as the project went on and staff became used to the research assistants’ presence) and challenges documenting how many parents received the study flier. Fourth, clinic staff made suggestions about improving recruitment such as suggesting research assistants get a copy of the schedule each day and that we expand the age range, suggesting older children may benefit from the program. Fifth, some parents reported frustration with the screening items, including expressing that there were too many questions and that the questions were repetitive or did not apply to their child (e.g., a question about school when their child was not in school). Finally, research assistants noted some technological problems with the program, which led to fixing program errors.

Discussion

The PYCC is a multi-component, technology-based behavioral parent training program for child DBPs that parents begin to engage in during a pediatric visit. The current study was a proof-of-concept trial (Czajkowski et al., 2015) to examine delivery of the initial check-up, after screening for DBPs, with parents of young children being seen as part of a pediatric visit. Given that the PYCC initial check-up is a novel approach to engaging parents in BPT, it was important to ensure that this approach was feasible and would promote parents’ interest in using the PYCC parent training website before moving into a time-consuming and costly randomized clinical trial.

In a pediatric primary care practice, parents (N = 6) that reported elevated child DBPs on a screener completed the initial check-up portion of the PYCC, including learning about receiving text messages and the PYCC parent training website, and provided feedback. Either orally or via a survey, all parents reported interest in using the PYCC parent training website. Because behavioral intention to use a technology-based program is an important predictor or of actual program use (Davis et al., 1989), these results provide important preliminary evidence to support subsequent research on the PYCC. In addition, parents reported positive perceptions of all PYCC components (initial check-up, text messages, and parent training website) in terms of perceived ease of use and perceived usefulness, which the Technology Acceptance Model (Davis, 1989; Marangunić & Granić, 2015) suggests are important elements for motivating intentions to use new technology. Finally, parents made suggestions to improve the PYCC, such as adding closed captioning and a program brochure. Observations about clinic flow and barriers to parent recruitment were noted by research assistants, which allowed us to learn about implementation of a technology-based program in a busy pediatric practice.

This preliminary investigation represents an important step in early program development. Given the many barriers to seeking, finding, and engaging in behavioral parent training for child DBPs (Johnston & Burke, 2020; McGoron & Ondersma, 2015), the overall goal of the PYCC is to be a high-reach intervention, allowing parents to access behavioral parent training in a convenient, accessible way. Assessing parents’ motivation to use the program and interest in engaging in behavioral parent training content is the first step in accomplishing this goal. The initial check-up was created to proactively introduce the PYCC in a setting parents already use (i.e., pediatrics) with content aimed at motivating further program use. Findings that parent participants who were recruited from a pediatric primary care setting, and not already seeking treatment for child DBPs, expressed motivation/intent to use the program suggests that the initial check-up can lead to the desired impact. This suggests the PYCC warrants further investigation in a pilot randomized controlled trial. These findings are in line with the goals of a proof-of-concept trial, which are to “determine, in an efficient way, whether or not the treatment merits more rigorous and costly testing using a randomized design” (Czajkowski et al., 2015, pg.7).

Beyond demonstrating PYCC proof-of-concept, this early-stage trial also involved gathering input from clinic staff about the program and its implementation potential. Overall, pediatric staff viewed the program favorably. Some concerns were raised about the screener fitting into the flow of the pediatric office. Interestingly, one parent also reported that the screener would not be easy to do while waiting at an office visit, and research assistants observed that some parents had concerns about the screening questions. Pediatric office staff and some parents also expressed concerns about the feasibility of parents completing the initial check-up component of the PYCC during the flow of standard pediatric visits. These findings were consistent with observations from research assistants that, even though there were slow days and times (i.e., days when no or only a few patients were in target age range), there were also exceptionally busy times when front office staff did not have adequate time to distribute the study flier. Additionally, how long parents and children spent in the waiting room varied greatly—there were times parents were waiting for only approximately ten minutes. We saw that not all parents had the time to do the screener or initial check-up during the visit but that completing both components prior to the office visit maximized completion rates (i.e., the two parents who elected not to participate after their visit indicated that they did not have time to stay and complete the study). Project staff were not able to record how many parents of children ages 2–5 years old were at the clinic during recruitment times, which is a clear limitation of the study as it is also likely that many parents were not reached before they were roomed, which meant that the screener was not completed.

Overall, an important lesson learned in this project is that brevity in the screening and initial check-up is key. It may be that successful implementation of this program requires an even briefer screener and initial check-up to fit into the sometimes-short wait times for parents being seen in pediatric primary care. Indeed, Foy (2010) outlined brevity as an important element of successful implementation of screeners in pediatric primary care settings. The ECBI (Eyberg & Pincus, 1999), the screener used in this investigation, includes 36-items, each of which is rated on a 7-point Likert scale (which yields an “intensity score”) and as a yes/no response that indicates whether the parent perceives the child behavior to be a problem (which yields a “problem score”). Thus, even though the ECBI is shorter than other widely used measures of child behavior problems (e.g., the Child Behavior Checklist; Achenbach, 1999), the length of the ECBI may have influenced ratings of the feasibility of completing the screener during a standard pediatric visit. Interestingly, Foy (2010) also noted that a successful screener would be open source, which is not true of the ECBI, and available both on paper and electronically. In this investigation, most pediatric staff thought the screener should be on a tablet, but some felt it could be on paper or that either option was fine. While there is not established guidance for the delivery of a program like the PYCC initial check-up in pediatric primary care, it is likely that brevity and flexible delivery are critical for successful implementation of the initial check-up.

There are limitations to this early stage, formative investigation. Results are not generalizable to other populations. Parent participants were homogenous in terms of race and gender—we note, however, that this work was done with a population of parents that are historically underserved (i.e., people of color in an economically depressed area where many barriers to behavioral health care exist) as promoting health equity is an important goal of this work. Only one pediatric practice was involved in this investigation. PYCC program materials may need to be customized to the needs and preferences of different populations of parents in different regions. While parents reported motivation to use the PYCC program, it is not clear if this motivation existed before they completed the initial check-up. Given that implementation of the PYCC in pediatric primary care may require making program elements briefer, gaining an understanding of what elements of the initial check-up are essential is important for further investigation. It may be that some parents are highly motivated to go through BPT and need only a very brief introduction to the program.

The promising results of this proof-of-concept trial do, however, suggest that further investigation of the PYCC is warranted. In terms of progressing along the steps to intervention development outlined in the ORBIT-model, a next step is a pilot randomized clinical trial (RCT; Czajkowski et al., 2015) of the PYCC. Building upon the current investigation, the pilot RCT will randomize parents to go through the initial check-up or learn about the PYCC through a flier and measure parents’ actual use of the parent training modules on the PYCC website in addition to measuring changes in parent-reported child DBPs and parenting.

Implications for Practice

The PYCC is an innovative approach to delivering behavioral health care within primary care practice. While this investigation represents only one step in developing and evaluating the PYCC, continued success of the program could provide a resource for pediatricians to share with parents of children with DBPs. Additionally, the program could inform best practices when offering patient-facing (and parent-facing) technology within healthcare settings. The PYCC is designed to deliver research-based information to patients/parents without creating a time burden for clinic staff. Similar programs could be developed for a range of health conditions. Ultimately, development, evaluation, and dissemination of the PYCC could lead to increased use of technology-based programs in primary care.