Barriers to recruitment of children with cerebral palsy in a trial of home-based training

Abstract

Many trials fail to include the targeted number of participants, causing scientific and ethical problems. The COAD trial of home-based training programs (HBTPs) for children with unilateral cerebral palsy (CP) encountered recruitment problems, even though the parent-delivered home-based approach complies with recent health-care developments in the Netherlands. The current project aimed to identify the barriers to recruitment in the COAD trial. This summative, multidimensional evaluation comprised informal conversational interviews in which stakeholders who had been involved reflected on the factors that impeded successful recruitment of participants into the COAD trial. Barriers to implementation and recruitment were clustered according to the constructs of the Consolidated Framework for Implementation Research (CFIR). Member checking validated the findings. A total of 41 stakeholders contributed to the evaluation. Barriers to the implementation of the HBTPs were identified within every domain of the CFIR (intervention characteristics, outer setting, inner setting, characteristics of individuals, and process). Parent-delivered home-based training was perceived as highly complex and in conflict with the pressures on and the needs of parents. Many parents preferred the alternative center-based group interventions. The involvement of a resonance group was highly valued, and opportunities for further enhancements emerged. Additionally, the importance of research consortia was emphasized. The appropriateness of the RCT as the study design was criticized. The findings of this study are summarized in a tool which provides a dozen directions for the successful recruitment of participants in pediatric rehabilitation research.

Abbreviations

CFIR

Consolidated Framework for Implementation Research

COAD

CO-creation At hanD: the road to independence

CP

Cerebral palsy

HBTP

Home-based training program

MREC

Medical Research Ethics Committee

NSRM

Netherlands Society of Rehabilitation Medicine

PenCRU

Peninsula Cerebra Research Unit for Childhood Disability Research

RCT

Randomized controlled trial

SCED

Single-case experimental design

1. Introduction

Successful recruitment of the requisite number of research participants is known to be challenging. A Dutch study showed that 38% of completed trials failed to include the targeted number of participants [1]. Similar and even lower rates have been reported elsewhere [2,3]. The scientific validity of underpowered studies is questionable, while participants are unnecessarily exposed to burdens and risks [1]. Additionally, it is a waste of already limited resources.

Numerous factors that influence recruitment have been reported for medical trials [4]. In rehabilitation research, however, motives for taking part are likely to be different. Firstly, interventions focus on activities, participation, and quality of life rather than body functions and structures, or extending life. Secondly, rehabilitation interventions often require more active and time-consuming involvement from participants and their families than do medical interventions [5]. Pediatric rehabilitation is even more distinctive, because children are vulnerable and parents advocate for their child's needs.

The COAD trial (CO-creation At hanD: the road to independence) is the most recent example of a study in Dutch pediatric rehabilitation that has encountered recruitment problems. The trial (NTR5743) aimed to include 78 children with unilateral cerebral palsy (CP) and their parents during a two-year period. After enrollment of only 12 participants in one year, the trial was terminated early. The experimental interventions were two home-based training programs (HBTPs), using either implicit or explicit motor learning principles. The HBTPs comprised a task-specific, bimanual approach. Parents provided intensive training in their home setting for 12 consecutive weeks. An interdisciplinary team consisting of an occupational or physical therapist and a remedial educationalist coached the parents. The protocols of the HBTPs have been reported elsewhere [6]. A three-armed, multicenter randomized controlled trial (RCT) of a pragmatic nature was planned to evaluate the effectiveness of the HBTPs (Fig. 1). The research team collaborated with a ‘resonance group’ consisting of parents of children with CP, a youngster with CP, the director of the BOSK (Dutch association for people with a physical disability and their parents), therapists, physiatrists, and a remedial educationalist. This group was consulted repeatedly during the planning and execution of the trial. The BOSK was also a co-applicant on the grant proposal.

Fig. 1.

Design of the COAD trial.

The parent-delivered home-based approach of the COAD trial complies with recent developments within health care towards self-reliance of families, professionals becoming coaches, and the use of e-health technologies [7]. A basic needs assessment among parents, including focus group discussions and surveys, confirmed the potential of home-based training. Despite these preconditions, enrollment in the COAD trial was disappointing. This justifies exploring whether and how the concept of home-based training contributed to the troublesome recruitment process. Many previous Dutch trials (i.e. BoBiVa, BOLIEN, LEARN 2 MOVE, POPEYE, and SPACE BOP), investigating other types of interventions, have also struggled to meet recruitment goals. Thus, the COAD trial also provided an excellent opportunity to advance the understanding of additional factors that impede successful recruitment in pediatric rehabilitation research. Hence, the aim of this project was to identify the barriers associated with the recruitment during the COAD trial, using a multidimensional perspective.

2. Methods

A summative evaluation of the COAD trial was made after its closing. To enable a multidimensional evaluation, six relevant stakeholder groups that had been involved in the COAD trial were invited to participate: collaborating rehabilitation centers, grant provider (i.e. manager of the subsidy program) and funding bodies, the approving Medical Research Ethics Committee (MREC), non-collaborating rehabilitation centers, parents of children with CP, and the research team (Fig. 2).

Fig. 2.

Contributors to the evaluation of the COAD trial.

All stakeholders willing to contribute reflected on the trial from their perspectives during informal conversational interviews with the principal investigator (LB). The conversations were either in person or over the phone and mostly individually. The interviews revolved around the question ‘What factors impeded successful recruitment of participants into the COAD trial’? The recruitment of participants and the implementation of the HBTPs and of the trial occurred within the collaborating rehabilitation centers. Consequently, recruitment issues were interrelated with and indistinguishable from implementation issues. Therefore, LB extracted barriers to recruitment as well as to implementation from the interview notes and clustered them according to the Consolidated Framework for Implementation Research (CFIR) [8]. The CFIR provides a framework of constructs that have been associated with effective implementation and is considered particularly useful for complex interventions. In this project, it was applied to evaluate the recruitment of participants and implementation of the HBTPs into clinical practice in the context of research. The definitions of the constructs as utilized are described in Supplemental File 1. Member checking of the preliminary report by the stakeholders was used to validate the findings.

3. Results

A total of 41 stakeholders, covering all six stakeholder groups, contributed to the evaluation (Fig. 2). The rehabilitation centers were represented by occupational and physical therapists, physiatrists, local ethical committee members, and a researcher. Thirty-two of the stakeholders completed the member checking. Many stakeholders stressed the importance of home-based training and still expected this to be the therapy of the future. Barriers to the implementation of the HBTPs in the COAD trial were identified within every domain, but not all constructs, of the CFIR (Supplemental File 1). The results are presented following the five CFIR domains: intervention characteristics, outer setting, inner setting, characteristics of individuals, and process. The stakeholders’ subjective commentaries are represented as faithfully as possible, allowing contradictory opinions and speculations. In the discussion section, the considerations will be put into perspective and recommendations will be provided.

3.1. Intervention characteristics

3.1.1. Intervention source

The involvement of the resonance group was valued by many stakeholders, especially the participation of the parents, the youngster with CP, and the patient association BOSK. One parent stated that the research team had sufficiently listened to the opinions of the resonance group. Nevertheless, several other stakeholders felt that some suggestions, for example reducing the training intensity, were overruled by the research team, based on scientific arguments. According to the involvement matrix, an instrument to define the level of involvement of patients and parents in research [9], the resonance group participated in the role of co-thinker. It was suggested that this involvement matrix might have been used to discuss the roles parents wanted to fulfill at each stage of the project, enabling their involvement as advisors or partners. Also, parents, as well as children and youngsters with CP, could have been more explicitly involved in the earliest stages of the project to jointly establish the topic, research question, research design, and intervention content and structure. However, the parents in the resonance group were well informed, highly engaged, and hence perhaps not fully representative of the average parent of a young child with CP. The research team might have attempted to involve a larger and more heterogeneous group of parents, using low threshold strategies such as social media, social events, or consultation with fellow parents by dedicated parents. Likewise, the resources and preferences of the collaborating rehabilitation centers might have been taken into account more at the beginning of the project. Such an impact analysis might have resulted in more convenient procedures. A different balance between conflicting scientific, family, and practical perspectives would probably have increased feasibility.

3.1.2. Relative advantage

While the trial was running, many alternative clinical treatment options were available to families. Intensive upper-extremity interventions using a playgroup setting around one theme, e.g. pirates [10], were very common. During the inclusion period, most parents had to decide whether their child would participate in either the COAD trial, including the unconventional concept of parent-delivered home-based training and remote coaching, or in a therapist-delivered group intervention of proven effectiveness at the center. Parents declared that these group interventions can be logistically challenging, having to travel to the center several times a week, arranging childcare for siblings, and requiring the child to be absent from school. Still, many parents seemed to favor these group interventions because they expected the burden for their family to be lower and felt it easier to relinquish control. Furthermore, clinicians observed that parents preferred the social contact with fellow parents and the motivational effect of the group process. Additionally, one parent expressed appreciation for the child's not having to compete with typically-developing peers. Children presumably liked the playful setting of the group interventions better than a task-specific approach at home. HBTPs, on the other hand, offer parents the opportunity to adjust both training moments and content to their family routine and interests. If families succeed in performing relevant activities in a meaningful setting and at a convenient time, practice may not feel like therapy. However, it appeared that parents' positive experiences of attending group interventions outweighed the potential benefits of the new HBTPs. For clinicians, a difficult home situation and the benefits of a group-based setting were important reasons to prefer group interventions over HBTPs for some potential participants. Since a sufficient number of participants is also needed for the center-based group interventions to be profitable, these and the COAD trial turned out to be competitors.

Finally, parents were frequently invited to participate in studies, which made them selective. Numerous outcome measures in this trial resulted in a relatively high research-related burden, also disadvantageous. The centers reported a similar ‘research fatigue’.

3.1.3. Adaptability

The centers perceived a lack of flexibility in the trial protocol with regard to two of the intervention components: the home visits, and coaching by pre-specified therapists. Both conflicted with the practical organization of other treatments provided by the rehabilitation teams. This led to the non-participation of some centers. Similarly, although the content of the HBTPs was individualized, the process was standardized, which provided limited possibilities to tailor it fully to the family needs.

3.1.4. Trialability

An RCT was used to test the newly developed HBTPs. In retrospect, many stakeholders suggested that this was premature, since developing and testing of interventions should be an iterative process. A priori focus group discussions with parents showed a positive attitude towards comparable interventions. A similar needs assessment in the specific target population of the HBTPs might have added useful information regarding parents’ perceptions of the concept and approach of home-based training and its clinical meaningfulness. Afterwards, feasibility and pilot studies by early adopters might have been beneficial in optimizing the interventions. Concerning effectiveness, other experimental designs (e.g. (stepped-wedge) cluster randomization) might have been less demanding for centers. A consecutive execution of several smaller projects can result in greater success in terms of sample size, coordination within dynamic clinical settings, costs, and early detection of difficulties. Hence, in retrospect, it seems that an RCT was not the most suitable approach for such complex interventions at this early stage.

Conversely, several stakeholders referred to the view of many funders, journals, referees, and others of the RCT as the ‘gold standard’ in intervention research, valuing other designs, especially qualitative studies, less. Similarly, the grant call for this project explicitly demanded the use of an RCT. The grant provider and funders mentioned in the interviews that their organizations recently initiated transitions to match appropriate designs to research questions [11]. Hence, a trend is beginning towards supporting small-scale, attainable projects. Positive results can then encourage funders to financially support more complex follow-up studies.

3.1.5. Complexity

The interventions were innovative in two ways, resulting in great complexity. The first was the home-based setting. The transition from conventional center-based to home-based interventions posed great demands on therapists and parents. Changing their habits and committing themselves to an unfamiliar treatment provoked resistance. Within centers, it was difficult and time-consuming for managers to facilitate deployment and for planners to arrange unusual appointments (e.g. home visits) alongside regular clinical care. Even though the research team provided an instructional course, extensive manual, and time-task matrix, therapists struggled to comprehend the expected duties and to complete them within the anticipated time. For families, the intensive training, including making video-recordings for the remote coaching, required an exceptional effort. Twelve weeks of repetitive, daily practice of the specific tasks was regarded as lengthy. Because remote coaching was unfamiliar to parents and practitioners, the research team provided a digital tool to facilitate secured data sharing. However, practical limitations of this system increased complexity.

Secondly, the incorporation of implicit and explicit motor learning approaches was innovative. Operationalization of these theoretical concepts led to complex intervention protocols. Parents, as well as therapists, indicated that the programs required background knowledge and a relatively high cognitive level in parents. The programs were also outside the ‘comfort zone’ of therapists, which made the programs even more challenging and time-consuming to grasp and apply. Furthermore, the research focused on both an implicit and an explicit program simultaneously and no crossover of therapists was allowed (to restrict contamination). This demanded the dedication of two therapists per intervention for each center and resulted in additional inconvenience in management and planning.

Regarding the research design, the comparison of three treatments was identified as a complicating factor. Especially with young children, parents advocate for the best care possible. Hence, potential randomization to a control group of usual care was mentioned as a reason for non-participation. Parents might also have been uncomfortable being randomized to an implicit or explicit teaching style. It was suggested that a wait-list control group and preference-based randomization might have resolved these delicate issues [12].

The complexity of both the interventions and the study limited comprehensibility of their benefits, which increased the threshold for centers and parents to participate.

3.1.6. Design quality and packaging

Due to parental involvement during its development, some parents as well as other stakeholders greatly esteemed the participant information, whereas others criticized the presentation of the study as being too research-oriented. To complement the parents' needs, it was suggested that the information provided to parents could have given more emphasis to the relevance of the HBTPs for families, revolving around the question ‘What's in it for me?’

3.1.7. Cost

As previously described, the research team developed the interventions according to theoretical and scientific standards. Consequently, the content was prioritized over the cost. Dependent of other care provided to individual children, the financial return sometimes made it unprofitable to provide the HBTPs. The HBTPs being merely cost-covering was a drawback for centers to participate. It emerged that centers and the research team had conflicting opinions about the trade-off between financial sustainability and desired functional outcomes.

Due to changes in the health care regulations (see the construct ‘External policies and incentives’), insurance companies would not pay for travel time and expenses for the home visits and this was not accounted for in the research budget, resulting in a major cost to centers and a reason to resist the intervention. An option might have been for the research team to consult with health insurance companies to find a solution to compensate for this.

3.2. Outer setting

3.2.1. Patient needs and resources

Numerous concerns regarding the needs and resources of the families arose from parents themselves, as well as from clinicians’ interactions with parents. Most frequently-mentioned was the pressure perceived by families. Parents of children with CP generally have a busy life, combining the care of their child(ren) with work, sports, and social events. In addition, some parents encounter more complex circumstances, such as moving house, divorce, or a sibling also requiring special care. On top of this, CP is a complex disorder, involving a range of problems with, for instance, mobility, communication, or behavior, aside from the arm-hand impairments. Hence, rehabilitation is multifaceted, which results in many time-consuming activities and arm-hand treatments sometimes not being prioritized. Children attend either special education, involving time-consuming door-to-door transportation, or mainstream education, which can be demanding, cause burden, and leave little time for practice. Parents balance the time and energy investment of their family with the expected benefits. Both parents and clinicians emphasized that non-participating parents did not use these points as excuses, implying that they were unable, rather than unwilling, to perform a HBTP. If they could not give it their best, they would decide against participation.

Additionally, clinicians reflected that, in present Dutch society, health professionals tend to patronize families and that many parents accept this, with restriction of their autonomy as a consequence. A reason why families accept this dependency on health professionals may be parents' lack of self-confidence. From clinicians’ impressions, many parents have more confidence in therapists than in their own competence and therefore feel unable to be highly engaged in a HBTP, making it difficult to achieve self-management. Several stakeholder groups suggested that, as the rehabilitation setting was not yet ready for parent-delivered interventions, the COAD trial was probably ahead of its time.

Some parents preferred to limit themselves to their parenting role, instead of being a co-therapist. They considered the home to be a safe environment for the child to unwind, wanting to be there for their child and to provide stimulation, but without forcing the child to practice activities. Parents who are employed in the health sector themselves either valued HBTPs more or, conversely, wanted to separate their occupation from their role at home. In certain cases, parents might have been discouraged because their social network was incomprehending of the effort parents needed to put into the treatment. Last, the intention of parents to avoid emphasizing the disability at home, in order to normalize the child's condition, was mentioned as a reason to decline home-based training.

It was also suggested that the parents' views regarding HBTPs may differ with time. Children in the project's target population make the transition from a (therapeutic) playgroup to kindergarten, and from kindergarten to school. During these transition periods, most parents do not want to start a new treatment on top of the child's getting acquainted with the new situation. In the youngest children with CP, HBTPs may overcome developmental disregard and fit in well with usual care (i.e. screening and providing advice). On the other hand, HBTPs may be less feasible for these young children from the perspective of parental coping. Clinicians indicated that, following the diagnosis, some parents appear to be looking for guidance and prefer the therapist to be involved continuously. Later on, parents may have come to terms with the child's disability and the prognosis may be clearer, decreasing the emotional pressure on parents. They may also have become more able to express their children's treatment needs.

Parents declared a need for guidance in stimulating and guiding their child in a playful way at home. In that context, the HBTPs may have created an impression more of therapy than play. In addition, the interventions may have lacked a ‘wow factor’ to motivate children and parents. Alternating therapy periods at the center and at home, following a looser protocol, might have been preferred. However, in one parent's opinion, parents should simply make time for home training.

3.2.2. External policies and incentives

First, although a multi-institutional research team carried out the COAD trial, stakeholders identified a lack of extensive collaboration between institutions before and during this and other projects, resulting in fragmentation of research across the country. The rehabilitation sector should agree on a research agenda reflecting the mutual interests of stakeholders, such as research teams, patient associations, and grant providers. Next, research should be performed by large consortia, alternating the coordinating role among the institutions while other centers facilitate, either in partnership or by providing potential participants. This strategy would be expected to be less sensitive to competition over acquiring funding or participants, and might positively affect mutual commitment. Collaboration is considered insurmountable for studies within pediatric rehabilitation in the Netherlands to succeed. To achieve this, transparency and willingness to share are regarded as highly important. Readiness to make steps in this direction has been perceived. The Dutch rehabilitation association (Revalidatie Nederland) and the Netherlands Society of Rehabilitation Medicine (NSRM/VRA) are currently initiating these transitions.

Second, despite providing a statement of intent, which is needed for the MREC to assess the attainability of recruitment targets, centers withdrew before the start of the study, as a result of reviews by their local research committees. The research team agreed that this was undesirable and that more binding agreements were needed. Proposed solutions were legal penalty clauses and a track-record of misconduct. Moreover, concerns were expressed about the procedure of local review. Incongruity was observed between experts on the content who were aware of what was going on in the workplace (e.g. therapists), those providing the statement of intent (i.e. the head of department), the local review committees, and the boards of directors who needed to approve the execution of the study. Reciprocal communication about interests, consequences, and deployability should be more effective to get and keep collective agreement about study involvement.

Third, as of 2015, due to decentralization of social policy, the Dutch financial incentives of the health care system have been amended [13]. This development has put pressure on the rehabilitation setting and restricted the possibilities of financial investments in innovation and research by centers (see the available resources described in the construct ‘Readiness for implementation’).

3.3. Inner setting

3.3.1. Structural characteristics

Since the HBTPs impacted the primary processes to a large extent, the infrastructure within centers had to be efficient for the implementation to succeed.

3.3.2. Implementation climate

Compatibility: the interventions under study did not fit with the existing workflow of the usual center-based therapies and, as previously mentioned, their adaptability was limited. Because the organizational systems varied, the perceived impracticality differed from one center to another. Some centers were, due to privacy concerns, reluctant to use video-recordings.

Relative priority: because of the regular workload and the urgency to achieve production levels, recruitment for the trial was only a side issue. For managers and clinicians to commit to the project, the expected gains had to outweigh the time and energy spent on implementation, for instance regarding the instructional course for the practitioners involved in the HBTPs.

As a result of the complex and time-consuming development of the intervention protocols, the research team was compelled to postpone recruitment, which affected the implementation climate.

3.3.3. Readiness for implementation

Leadership engagement: in addition to the financial considerations, two criteria were decisive for centers to participate, namely the ability to deliver the programs according to the protocols, and inclusion of the requisite number of subjects for both the center itself and the study as a whole being feasible.

Available resources: given the health care system changes, the available resources in the centers did not allow for additional financial investment. Moreover, the limited availability of therapists and remedial educationalists was a restraint on centers’ ability to include sufficient participants.

3.4. Characteristics of individuals

3.4.1. Individual stage of change

Overall, the clinicians declared that they could not have done better in recruiting families, taking the aforementioned matters into account. The enthusiasm and focus of attention on recruitment of the professionals within the centers varied. Some clinicians stayed focused, while the alertness of others decreased during the study. Several circumstances reduced the clinicians’ motivation: 1) a relatively long period between training and start of the interventions; 2) interruption of recruitment during holidays; 3) randomization of the first participant to the usual care control group; 4) the actual training being home-based and therefore not visible to the department; 5) problems with the digital data sharing tool; and 6) perceived insufficiency in active leadership by the research team to facilitate the therapists.

3.4.2. Other personal attributes

Certain clinicians were selective in their recruitment, for example, based on knowledge of the families’ home situations. They put more effort into the recruitment of parents that they expected to participate successfully and omitted to approach others.

3.5. Process

3.5.1. Planning

A new research team from multiple institutions carried out the COAD trial. Despite its dedication, the group dynamics in combination with the challenging operationalization of the interventions resulted in a too-gradual evolution of the project. Consequently, when the research team was ready to start implementation, the centers’ eagerness had declined. Furthermore, this delayed recruitment, hindering proceeding by centers and shortening the inclusion period.

The three-armed design required a relatively large sample size. The estimation of the number of potential participants was exaggerated for several reasons. First, a survey was conducted prior to the study in order to assess willingness to participate in HBTPs in a group of parents whose children were engaged in an intensive treatment program. This was likely a non-representative sample. Also, the characteristics and time-investments of the therapy and research-related activities were not known at that point. Second, the specified inclusion and exclusion criteria reduced the number of eligible participants. Third, expectations of recruitment rates were too optimistic. Therefore, centers were initially invited to collaborate based on their geographical location as a practical argument. Taking into account ‘Lasagna's law’ on patient recruitment might have reduced the overestimation [14]. The MREC members emphasized the need for them to be more critical in assessing whether the estimation of expected participants in future study proposals is adequate.

The grant provider believed that the research team, like most scientists, lacked strategic thinking skills. A priori barrier analysis, for instance including developments in the health care sector and competitive interventions, might have prevented some of the problems.

3.5.2. Engaging

External change agents: announcing the COAD trial and involving the centers from the beginning, as described earlier, might have had a positive influence on the centers’ enthusiasm and commitment. Clear communication of the benefits for the center and primarily focusing on the content experts were suggested as key factors in generating goodwill. The research team reflected that it was not proactive enough in the enrollment and follow-up of centers, and did not sufficiently take the dynamics of each individual organization into account. Moreover, the importance of PR was underestimated and options to better promote the study were missed. Likewise, the project lacked a clear figurehead. Consequently, the pediatric rehabilitation sector was not nationally attentive to the COAD trial.

3.5.3. Executing

In most centers, it was challenging to identify potential participants. Local or national registers could have eased recruitment for clinicians and researchers.

3.5.4. Reflecting and evaluating

Some stakeholders associated with the centers declared that the research team did not sufficiently inform them about the state of affairs, in particular when the project did not go well, while others appreciated the thoughtful communication by the research team. The research team felt that it had taken sufficient initiative and had been very transparent about the recruitment struggles. The grant provider, in contrast, stated that the researchers should have alerted them in a timelier manner. Their self-reflection revealed that they could have been more critical of the progress reports. Additionally, miscommunication between the grant provider and the funding bodies responsible for the grant was said to be partly accountable for the early termination of the trial.

4. Discussion

This project identified many interacting factors across all domains of the CFIR (i.e. intervention characteristics, outer setting, inner setting, characteristics of individuals, and process) that impeded implementation of the HBTPs and the COAD trial. Within the complex combination of barriers, three levels can be distinguished: barriers that directly affected the recruitment of participants; barriers related to implementation that indirectly influenced recruitment; and barriers related to the study design that indirectly influenced the recruitment conditions.

4.1. Recruitment

Most factors that directly impeded recruitment were related to the home-based training. The parent-delivered HBTPs' being frequently perceived to be incompatible with the pressures and the needs of parents is likely to be associated with the elevated levels of parental stress in this population [15]. An imbalance between personal patient burden and benefit, which has previously been reported as a barrier [16], is therefore particularly relevant in families of children with CP. An unanticipated finding was that, despite the unique potential of HBTPs to integrate training moments and activities into the family routine, many parents preferred the alternative center-based group interventions. There are similarities between the present results and a recent review which shows that parents’ feelings of capability is a determinant of parent-delivered interventions. Some parents do not see their parenting role as including delivery of therapy and tend to feel overwhelmed [17]. Consistent with the literature, the high complexity of the HBTPs also appeared to be an underlying factor [4]. This was to some extent an inevitable consequence of the operationalization of the innovative home-based training and the fundamental theoretical principles of motor learning that were used. These substantial barriers unique to the home-based training conflicted with the intended shift towards reinforcement of self-reliance of families.

Aarts et al.‘s RCT is an example of a study in the Netherlands that successfully enrolled the targeted number of children with CP and reported not a single drop-out in the experimental group. The authors noticed the treatment's being attractive to children as a major strength. Based on the results of the current evaluation, procedural simplicity may have been a facilitating factor from the therapists' viewpoint [18].

Considering the methodology of the study, the randomization aspect was assumed to be a disincentive for parents to participate. This is in agreement with the BoBiVa, as well as the SPACE BOP study, in which the intended recruitment procedures had to be adapted because many parents had a strong preference for or aversion to the experimental treatment (i.e. botulinum toxin A) and therefore did not want their child with CP to participate in a randomized trial [19,20].

4.2. Implementation

The two essential barriers of implementation identified were both associated with collaboration. While identifying facilitators was not the aim of the project, the engagement of the resonance group clearly emerged as being highly valued. This is consistent with the literature reporting that an internally-developed intervention, with transparency and participation in the decision-making process, is an effective engagement strategy for stakeholders to feel ownership of the innovation, and this contributes to implementation success [21]. In this context, the current study pointed out specific opportunities for further enhancements in pediatric rehabilitation: empowering parents to be involved as an advisor or partner, involving stakeholders from the very beginning of the project, and including a representative sample of targeted users. Our results support the general recommendation to engage all stakeholders as equal partners in the process [4]. It is important to realize that, besides being an opportunity, participation in research teams can be perceived as an additional pressure by parents. Therefore the level of parent and patient participation should correspond with the family wishes. A leading example of successful partnership between researchers, families, and health care professionals is the Peninsula Cerebra Research Unit for Childhood Disability Research (PenCRU) [22].

This evaluation emphasized the importance of research consortia in strengthening recruitment capacity. A research consortium has previously been defined as “a group of individuals and their organizations working together to address specific research needs and attract funding. Consortia design and implement research over one or several countries and sites” [23]. El Ansari et al. determined benefits and challenges of research consortia and provides recommendations for effective collaboration [23,24].

4.3. Study design

The appropriateness of an RCT as the study design and the estimated ability to recruit the necessary number of study participants were criticized. The former criticism is in agreement with, amongst others, the new Medical Research Council guidance, which recommends an iterative development and evaluation process for complex interventions [25]. Although Dutch grant providers and funders have recently started to encourage smaller-scale research projects, opportunities to gradually develop and investigate interventions are still limited, given the highly competitive process of acquiring research funding and associated demands. At the same time, the understanding and appreciation of alternative designs that require smaller sample sizes need to be advanced.

Given the heterogeneity of the population of children with CP, single-case experimental design (SCED) can be a powerful design to evaluate intervention effectiveness.



In interpreting this evaluation, it must be recognized that the results may point to contradictory conclusions. For instance, it was argued that there should be more intensive involvement of parents as stakeholders in the research project, while on the other hand the burden on parents was mentioned as a limiting factor to their participation in the trial. Similarly, other recommendations would have had drawbacks, such as a significant increase in time and costs necessary to develop and execute the study. In some instances, the research team decided from the outset that theoretical and scientific requirements of the HBTPs and the trial outweighed other interests. An example concerns the home visits, which were adhered to because the content of the HBTPs and their standardization within the trial were prioritized over the centers' concerns about the associated costs. Overall, it is a challenge to make a trade-off between the many sometimes incompatible interests, including family needs, centers' perspectives in terms of feasibility and costs, scientific rigor, formal policies, rules and regulations, and time and costs to prepare and perform the study. Effective collaboration approaches are assumed to constructively advocate each stakeholder's viewpoint.

In addition to the many aspects related to the COAD trial that could have been influenced, the amendment of the health care policy was a circumstance beyond the control of the research team and other stakeholders involved. This, as well as competitive group-based interventions, applies specifically in the Netherlands. The other results are expected to be generalizable to other Western societies and health care systems.

The number and variety of stakeholders involved was the major strength of this evaluation. However, due to legal and ethical constraints about approaching non-participants, the proportion of parents was relatively low. Another limitation was that, given the aim of the project, the focus was on barriers to participation only. Facilitators of recruitment were not systematically explored and their role in the process may therefore have been underestimated. Last, this evaluation produced a narrative of all opinions, sometimes opposed, rather than consensus-based outcomes.

Barriers directly related to the home-based training imply that those concerned were not yet receptive to the major transition to parent-delivered interventions. Efforts are needed to ensure that pediatric rehabilitation care is compatible with societal developments as well as family needs. A hybrid combination of home-based and center-based features may yield maximum benefits. The findings of this evaluation may improve the planning and conduct of studies and other implementation projects in pediatric rehabilitation. Fig. 3 provides a dozen directions for successful recruitment of participants in pediatric rehabilitation research. Further work is required to establish facilitators of both participant recruitment and implementation of interventions, and to create evidence-based strategies to facilitate these processes. Contrary to expectations, many of the barriers addressed are shared with other health care fields and populations. It is therefore essential that lessons learned from previous projects are also recognized.

Fig. 3.

A dozen directions for successful recruitment of participants in pediatric rehabilitation research.

5. Conclusion

This evaluation has shown that barriers to recruitment of participants during the COAD trial were mostly related to the home-based training approach. The findings can be used to develop a targeted transition towards parent-delivered interventions in pediatric rehabilitation. Furthermore, it has been confirmed that comprehensive collaboration between all stakeholders involved is fundamental in order to balance the many different interests. Last, the results favor the facilitation of an iterative process, including small-scale studies, of intervention development and evaluation.

Competing interests

The authors have no competing interests to declare.

Funding

This work was supported by Revalidatiefonds, Johanna Kinderfonds, and Stichting Rotterdams Kinderrevalidatie Fonds Adriaanstichting in the 3rd Program Rehabilitation Research of ZonMw (the Netherlands Organization for Health Research and Development), [grant number 630000001]; Revalidatiefonds [grant number R2016006]; and Stichting Vooruit [grant number 18–05/YvH/NS]. Representatives of the grant provider and funding bodies contributed as stakeholders to the data collection and member checking. They had no other role in study design, in the analysis and interpretation of data, in the writing of the report, and in the decision to submit the article for publication.

Appendix A. Supplementary data

The following is the Supplementary data to this article: