Skip to main content
PMC home page
BMC Research Notes logoLink to BMC Research Notes
. 2014 Nov 20;7:824. doi: 10.1186/1756-0500-7-824

What can we learn from a failed trial: insight into non-participation in a chat-based intervention trial for adolescents with psychosocial problems

Rik Crutzen 1,✉, Hans Bosma 2, Jano Havas 2,3, Frans Feron 2,3
PMCID: PMC4247599  PMID: 25409911

Abstract

Background

Psychosocial problems are highly prevalent among Dutch adolescents. We have conducted a trial to test whether offering chat-based consultations could be of added value within the context of Dutch Youth Health Care. This trial was ended prematurely because of recruitment issues. The aim of this study is to learn from this failed trial and to provide more insight into non-participation. Sources of data were non-participation forms, oral clarification, patient records, telephone interviews with adolescents that declined to participate, and a questionnaire-based process evaluation among nurses.

Results

Non-participation appears to be a multi-factorial problem. Of those 290 adolescents eligible to participate, the majority (n = 165, 57%) declined to do so. Two-third of those (n = 109) also refused usual care, which might be indicative of not wanting help and/or experiencing problems and the validity of the assessment instrument. The other one-third (n = 56) did enrol in usual care and indicated other reasons for non-participation, such as a preference for face-to-face consultations, the extensive information that was provided, and not liking the idea of being randomized.

Conclusions

This study shows that even if a trial fails, we can learn about the challenges of recruiting adolescents in intervention trials.

Trial registration

NL37668.068.11/METC11-3-077.

Keywords: Non-participation, Chat-based intervention, Ehealth, Adolescents, Psychosocial problems, Youth health care

Background

Psychosocial problems are highly prevalent among Dutch adolescents: prevalence rates range from 19 to 28%, depending on socio-demographic characteristics (e.g., more likely among those with a low educational level [1]) and definition of psychosocial problems (e.g., behavioural problems, emotional problems) [2, 3]. These problems do not only impair current daily functioning [4, 5] and result in school drop-out [6], but are also known to track into adulthood [7] and lead to increased future health care costs [8]. However, the majority of adolescents do not receive professional help for their problems [9, 10], because they are often reluctant to seek professional help (e.g., due to shame) [11–13]. Therefore, future intervention initiatives should focus on removing barriers to seek help for psychosocial problems.

The use of the Internet as the primary delivery mode for interventions has expanded substantially [14], mainly due to a growth in access to the Internet, especially in high-income areas, such as the United States (78.6%) and Europe (63.2%) [15]. Interventions targeting psychosocial problems are delivered online nowadays, because it is known that adolescents feel empowered to talk about sensitive topics in such an online setting [16, 17]. Moreover, such interventions fit in with adolescents’ needs and use of the Internet [18, 19]. A qualitative study among Dutch adolescents confirmed the idea that they do feel the need to get information and help regarding their psychosocial problems via the Internet [20].

Intervening via chat could be a valuable opportunity, because approximately 90% of adolescents use the Internet to chat [21]. Chat refers to individual synchronous online chat counselling and therapy [22]. The use of chat in the context of an intervention can have a positive impact on outcome measures. Chat sessions led by mental health professionals, for example, resulted in a reduction of adolescents’ depressive complaints [23]. Another example, focusing on eating disorder prevention, showed that even a 1-hour moderated chat session about body image and eating significantly reduced eating pathology and improved self-esteem [24]. Moreover, shyness or anxiety does not pose an obstacle to use chat-based interventions and is even advantageous [25]. As concluded in a narrative review, chat-based interventions aimed at adolescents have a large potential public health impact [26].

Therefore, we have decided to investigate whether chat-based interventions could be of added value in the context of Dutch Youth Health Care (YHC). According to the Dutch Public Health Act, YHC Centres of the Regional Public Health Services (RHPS) have a statutory task to prevent and identify psychosocial problems among adolescents. As part of the YHC system, all children living in the Netherlands receive free preventive health examinations (PHE) regularly from birth through adolescence. The first PHE in adolescence takes place in the second year of secondary school (at the age of 13–14). If adolescents have psychosocial problems, then they are offered three additional face-to-face consultations with a YHC worker (in a time span of 3 months). We have conducted a trial to test whether offering chat-based consultations instead could be of added value, because adolescents might feel less shy and anxious to talk about their psychosocial problems.

The main reason to use YCH as a setting to conduct this trial was that this setting ensures access to adolescents, because assessing psychosocial problems and offering additional consultations are part of the regular procedure in PHE. In other words, the intervention is embedded in an existing setting, which is deemed beneficial [27]. Unfortunately, after almost 8 months of recruitment, only 12 adolescents participated (5 in the chat condition, 7 in the face-to-face condition) and the trial was terminated. This was not the first trial among youngsters that was ended prematurely because of recruitment issues [28]. Therefore, the aim of this article is to be able to learn from this failed trial, for future intervention initiatives, and to provide more insight into non-participation.

Methods

To be able to learn from this failed trial, we (1) describe the trial procedure and (2) explain how we gained insight into non-participation from a point of view of both adolescents and YHC workers. The trial was a single-centre trial conducted at the RHPS Zuid Limburg (covering 18 municipalities). Adolescents were randomized to either the chat condition or the face-to-face condition. The Strengths and Difficulties Questionnaire (SDQ; as described later on) was used as primary outcome measure. Sample size calculations, assuming an alpha of .05 and a power of .90, resulted in 360 adolescents that had to be included at baseline.

Ethics statement

The medical ethics committee of the University Hospital Maastricht and Maastricht University (METC azM/UM) granted ethical approval (Protocol ID: NL37668.068.11/METC 11-3-077).

Trial procedure

First, the chat application was developed. The chat application was added to a separate website (i.e., not the website of the RHPS). Adolescents had to log on to this website, to make sure that it was only accessible to those who had agreed on a consultation with a YHC worker. Such agreements could be made directly, per phone, and per e-mail. The chat application was previously used by other organisations offering online support, which made sure that it was over its teething troubles. Subsequently, the YHC workers (i.e., nurses) were trained. A total of 19 nurses (i.e., all nurses at the RHPS Zuid Limburg) participated in this training. The training consisted of two days; the first day concerning online support in general, the second day focusing on online support by means of a chat-based intervention. It was an in-company training provided by an external partner with previous experience in offering online support. (Note: Additional meetings for nurses were planned to exchange their experiences regarding use of the chat application, but these were cancelled because the trial ended prematurely).

After these preparations, the actual recruitment started. This was kicked-off by a meeting for all nurses to explain the recruitment procedure and to provide them with all the materials needed (e.g., information letters, informed consent forms). Additional meetings were scheduled to discuss the disappointing number of recruited adolescents and the reasons for non-participation, and possible solutions. Based on these meetings, an incentive was raffled among participants (i.e., an iPad), the information letter for adolescents was shortened (from 3 pages to 1 page, with a link to a website containing the other information), and a YHC physician visited class rooms to explain the trial procedure before the PHE.

To assess psychosocial problems, adolescents had to fill in the self-administered Dutch version of the SDQ before the PHE, which is the common procedure. Furthermore, adolescents’ gender and educational level was registered (categorized as low – intermediate secondary education – and high – higher secondary education/preparatory university education). The SDQ has been validated among Dutch adolescents [29, 30] and its use is in line with national YHC guidelines [31]. The SDQ includes 25 symptom items and measures both negative (difficulties) and positive (strengths) behavioural and emotional attributes of the adolescent. A total score of 12 or higher on the SDQ is a proxy for an adolescent having psychosocial problems [32] and this cut-off is used as the primary inclusion criterion for the trial. Secondary exclusion criteria, which were only assessed if an adolescent met the first inclusion criterion, were adolescents already receiving treatment for their psychosocial problems (e.g., by a psychologist) and adolescents attending special education.

If adolescents were eligible to participate, they were invited to participate in the trial during the PHE. In line with ethical guidelines, they were offered one week time for reflection regarding this decision. Moreover, they were given an information letter and informed consent form, as well as a non-participation form if they decided not to participate. Adolescents were also given an information letter and informed consent form for their parents. Ethical guidelines require that both the adolescents as well as their parents sign the informed consent form. After one week, adolescents were contacted again to confirm whether they agreed to participate and to remind them about the non-participation form. Subsequently, adolescents were randomized and a consultation was scheduled based on the condition they were assigned to (i.e., face-to-face or via chat).

Assessing non-participation

We had initially planned to conduct a questionnaire-based process evaluation among those adolescents using the chat-based intervention. We cannot draw strong conclusions based on these data due to the limited number of participants in the chat condition (n = 5). Therefore, we have decided instead to focus on the non-participation among adolescents. Four sources of data were used to gain more insight into possible reasons behind this non-participation from the point of view of adolescents.

Non-participation form

Both adolescents and their parents could fill out a non-participation form with predefined categories. Categories were, for example, the trial being too burdensome, not seeing the added value of the trial, and no preference for online support.

Oral clarification

Adolescents’ remarks about the trial that were made to the nurse during the PHE and phone calls from parents afterwards were collected as field notes. This was non-standardized information.

Patient records

Adolescents’ patient records were examined to ascertain whether they enrolled in usual care (i.e., whether appointments were made for face-to-face consultations) despite declining to participate in the trial.

Telephone interviews

A brief telephone interview was conducted with a random sample of adolescents that declined to participate, but did enrol in usual care. The main open-ended question asked to those adolescents was what their reasons were to decline participation in the trial.

The first source of data (i.e., the non-participation form) was planned a priori, all the other sources of data were used after the trial was terminated. Besides the point of view of adolescents, we have also decided a posteriori to pay extra attention to non-participation in the planned questionnaire-based process evaluation among nurses (n = 19). We have added open-ended questions regarding how to improve information regarding the trial (e.g., to nurses, to parents, to adolescents) and how to improve the recruitment procedure.

Results

Recruitment in the trial

Recruitment lasted from April 1st, 2012 until November 24th, 2012. Figure 1 shows the flowchart of recruitment in the trial. A total of 3,943 adolescents were invited to the PHE, of whom 574 did not show up (15%). Of the remaining 3,369 adolescents that attend the PHE, 360 adolescents scored 12 or higher on the SDQ (11%) and 290 of them were eligible to participate. Of those eligible to participate, 74 adolescents were not invited to participate (26%) and 12 adolescents were wrongly excluded (4%). There was no information available regarding 10 adolescents (3%). A majority of 165 adolescents (57%), however, declined participation. Adolescents that declined to participate did not differ in terms of gender (χ2 = 0.10, p = .75) or total score on the SDQ (t(288) = 0.89, p = .38), but declining to participate was more likely among those with a high educational level (χ2 = 5.96, p = .02). The reasons behind this non-participation were identified by means of data triangulation from several sources (as described in the Methods section).

Figure 1.

Figure 1

Open in a new tab

Flowchart of recruitment in trial.

Non-participation

Of those 165 adolescents declining to participate, only 15 completed a non-participation form as well as 8 of their parents. Although these results cannot be generalized, a few of them (i.e., 4 out of 15) indicated that they did not prefer online support; 2 out of 8 parents mentioned the same reason. The patient records revealed that the decision not to participate was made mainly by adolescents themselves (n = 152; 92%) and only to a limited extent by their parents (n =13, 8%). Fifty-six adolescents (34%) that declined participation in the trial, enrolled in usual care. Oral clarifications revealed limited additional information regarding this group. Nevertheless, 109 adolescents (66%) did not only decline to participate in the trial, but did also refuse the offer for additional consultations (i.e., usual care). Oral clarifications indicated reasons such as “does not experience problems”; “will call if help is needed”, and “does not see added value of further intervention by YHC”. A random sample of 17 adolescents that declined participation in the trial, but did enrol in usual care, were invited for a telephone interview. A total of 13 adolescents agreed to take part in this interview (response rate: 76%). Some (n = 3) mentioned being randomised as a reason for non-participation, while others (n = 2) had no objection to this. Some (n = 5) preferred face-to-face consultations, while others (n = 3) preferred chat consultations.

All 19 nurses indicated in the questionnaire that it was clear to them that the trial concerned a chat-based intervention for adolescents that scored 12 or higher on the SDQ. Thirteen nurses indicated that information given to the parents could be improved; e.g., “the letter to the parents was quite scary, as if we assumed that there were psychological problems”. A suggestion, for example, was to provide “a clear and compact flyer describing the inclusion criteria and steps within the project”. With regard to information given to the adolescents, it was suggested to shorten the information letter and to provide more information in the class room (both suggestions were implemented during the trial). Some remarks were made about using the SDQ-score as an inclusion criterion. Regarding high SDQ-scores, nurses reported e.g., “this can usually be explained by characteristics of reaching puberty”; “this does not always mean that there are problems”. Nurses mentioned the extensive information and signing of informed consent forms as the main reason for non-participation. Another reason, according to the nurses, was a preference for face-to-face consultations or “online support, but preferably really anonymous”.

Discussion

Non-participation in a chat-based intervention trial for adolescents with psychosocial problems appears to be a multi-factorial problem, which does not only depend on trial procedures (e.g., information letters, randomization). On the one hand, staff adherence might have been a problem. Despite a kick-off meeting to explain the recruitment procedure and to provide all the materials needed to nurses, adolescents were not invited to participate, were wrongly excluded, or there was no information available about their status in the trial. It is important, therefore, to constantly monitor staff adherence to the trial procedure. Moreover, recruitment via research staff might come with an additional cost when compared with practitioners (e.g., nurses), but this might improve recruitment [33]. On the other hand, nurses and adolescents reported several other issues. The focus here is to explore non-participation among the majority of adolescents that was eligible to participate, but declined to do so.

Two-third of those that declined to participate in the trial also refused usual care. This might be because they do not want help for their psychosocial problems by YHC or other mental health professionals or they do not experience problems [34]. A systematic review revealed barriers that adolescents perceive to seek help. The main barriers are stigma and embarrassment, problems recognising symptoms (i.e., poor mental health literacy), and a preference for self-reliance [35]. Therefore, future intervention strategies should address adolescents’ desire for self-reliance (e.g., by providing evidence-based self-help material) and should aim to reduce stigma associated with ‘mental illness and mental health help-seeking’. Incentives do not always encourage participation, as this also depends on the information provided about the intervention itself [36], especially for those who experience problems. With regard to recognising symptoms, it is important to increase adolescents’ mental health literacy. These suggestions by Gulliver and colleagues [35] are all preventive in nature and are relevant for all adolescents.

The high percentage refusing usual care as well might also raise questions about the validity of the self-administered version of the SDQ for adolescents, as they might score 12 or higher but still not experience problems. Although the SDQ was not initially designed for this purpose, it has become an integral component of assessment packages for service and research purposes (e.g., as an outcome measure). Nevertheless, the use of the SDQ alongside a number of other tools is the preferred method of its use [37]. It is recommended, for example, to complete such assessments with evaluations by parents and teachers [38]. The high specificity of the SDQ and its strong associations with psychiatric disorders make it a good indicator of whether additional clinical assessment and intervention may be necessary. A previous study concluded that “the SDQ may have maximal value as part of a routine assessment in high-risk paediatric clinics based on perceived difficulties by parents, teachers, or children. While improving the efficiency of case-finding, the burden of medical cost and potential distress caused by false positives can also be significantly reduced” [39]. It is worth noting that declining to participate was more likely among those with a high education level. It is unclear why this is so, as non-participation is generally higher in the lower educated groups [40], where problems are generally more severe and needs for intervention accordingly higher [1]. It might be that those with a high educational level are more inclined to use private services and by-pass RPHS. Focusing on differences between groups in terms of non-participation is an interesting avenue for further research.

The other one-third of those that declined participation in the trial did enrol in usual care. Some of those preferred face-to-face consultations, which might be because counselling is still viewed as an interpersonal experience [41]. This was not expected, however, based on previous findings among this target group indicating that they do feel the need to get information and help regarding their psychosocial problems via the Internet [20]. Being anonymous is one of the reasons why people prefer online support [14], but as adolescents were already in a face-to-face setting with a nurse, this feeling of anonymity might have been less in the study at hand. We think it is safe to conclude, however, that chat-based consultations should merely be an addition that is appropriate for some adolescents and not a replacement of usual care (i.e., face-to-face consultations) for all adolescents.

Other reasons for non-participation in the trial were the information letter that was provided and adolescents that did not like being randomized. We tried to improve the information letter, but were limited in the sense that the medical ethics committee requires certain information to be given to adolescents and their parents. A previous study testing strategies such as providing additional information and direct mailing to the parents did not find any differences in terms of improved consent rates [42]. It might be worthwhile to explore a two-stage recruitment process with a youth-friendly invitation before the official information letter. Another study indicated that adolescents became confused by being presented with options (e.g., chat-based consultations) that were then removed by randomisation [43]. A Zelen randomised consent design where eligible participants are randomly allocated prior to being approached about the trial could be considered in future research [44]. So, consent can be sought conditionally (i.e., for the condition an eligible participant is assigned to), without the uncertainty of randomization.

A limitation of this study is that the decision to use certain sources of data was often made a posteriori. This is in the nature of this study, as its aim was determined after the trial was terminated. Moreover, non-participation in the trial is also likely to result in less willingness to complete non-participation forms and reluctance to participate in telephone interviews concerning non-participation. This resulted, in the current study, in relatively small samples for these sources of data. Although additional efforts (e.g., extra phone calls) might result in larger samples, one may also wonder whether this results in ethical conflicts if adolescents already reported that they did not want to participate in the trial. Nevertheless, it might be wise for future trials to use such additional sources of data, even if the trial is successful in terms of recruitment. Moreover, non-participation forms (completed by those refusing to participate) and patient records (completed by staff) can also be routinely checked during the recruitment phase to detect problems at an early stage. Future research might also include adolescents that do participate to assess ‘reasons for participation’ besides ‘reasons for non-participation’. This requires a “comprehensive data monitoring system with strong administrative support at both central and local levels” [45]. The issue of non-participation is also important outside the context of a trial. Many interventions that prove efficacious in trials are much less effective when disseminated outside the trial context [46]. It has been argued that much efficacy research is conducted in such a way that it “decontextualizes” an intervention effect by, for example, studying narrowly selected participants or using intervention strategies that are hard to implement by typical intervention agents [47]. This has been acknowledged by the field, as indicated by an increase in adoption and implementation research [48].

Conclusion

This study shows that even if a trial fails, we can learn about the challenges of recruiting adolescents in intervention trials. Non-participation is a multi-factorial problem and is amongst others related to a preference for face-to-face consultations, the extensive information that was provided, and not liking the idea of being randomized. Moreover, adolescents do not necessarily want help for their psychosocial problems or they might not experience problems.

Acknowledgments

This study was supported by a grant from ZonMw – The Netherlands Organisation for Health Research and Development (grant 15651.2004). The funding body had no role in design, in the collection, analysis, and interpretation of data; in the writing of the manuscript; nor in the decision to submit the manuscript for publication.

Abbreviations

YHC

Youth health care

RHPS

Regional public health service

PHE

Preventive health examinations

SDQ

Strengths and difficulties questionnaire.

Footnotes

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All authors have made substantial contributions to conception and design of the study, and interpretation of data. JH was responsible for data acquisition. RC has made the first draft of the manuscript. All authors have been involved revising the manuscript critically for important intellectual content and have given final approval of the version to be published.

Contributor Information

Rik Crutzen, Email: rik.crutzen@maastrichtuniversity.nl.

Hans Bosma, Email: hans.bosma@maastrichtuniversity.nl.

Jano Havas, Email: Jan.Havas@ggdzl.nl.

Frans Feron, Email: f.feron@maastrichtuniversity.nl.

References

  • 1.Havas J, Bosma H, Spreeuwenberg C, Feron FJ. Mental health problems of Dutch adolescents: the association with adolescents’ and their parents’ educational level. Eur J Public Health. 2010;20:258–264. doi: 10.1093/eurpub/ckp172. [DOI] [PubMed] [Google Scholar]
  • 2.Wittchen HU, Nelson CB, Lachner G. Prevalence of mental disorder and psychosocial impairments in adolescents and young adults. Psychol Med. 1998;28:109–126. doi: 10.1017/S0033291797005928. [DOI] [PubMed] [Google Scholar]
  • 3.Reijneveld SA, Brugman E, Verhulst FC, Verloove-Vanhorick SP. Area deprivation and child psychosocial problems: a national cross-sectional study among school-aged children. Soc Psychiatry Psychiatr Epidemiol. 2005;40:18–23. doi: 10.1007/s00127-005-0850-0. [DOI] [PubMed] [Google Scholar]
  • 4.Wille N, Bettge S, Wittchen HU, Ravens-Sieberer U. How impaired are children and adolescents by mental health problems? Results of the BELLA study. Eur Child Adolesc Psychiatry. 2008;17:42–51. doi: 10.1007/s00787-008-1005-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Gore FM, Bloem PJ, Patton GC, Ferguson J, Joseph V, Coffey C, Sawyer SM, Mathers CD. Global burden of disease in young people aged 10–24 years: a systematic analysis. Lancet. 2011;377:2093–2102. doi: 10.1016/S0140-6736(11)60512-6. [DOI] [PubMed] [Google Scholar]
  • 6.Kearney CA. School absenteeism and school refusal behavior in youth: a contemporary review. Clin Psychol Rev. 2008;28:451–471. doi: 10.1016/j.cpr.2007.07.012. [DOI] [PubMed] [Google Scholar]
  • 7.Hofstra MB, Van der Ende J, Verhulst FC. Adolescents’ self-reported problems as predictors of psychopathology in adulthood: 10-year follow-up study. Br J Psychiatry. 2001;179:203–209. doi: 10.1192/bjp.179.3.203. [DOI] [PubMed] [Google Scholar]
  • 8.Scott S, Knapp M, Henderson J, Maughan B. Financial cost of social exclusion: follow up study of antisocial children into adulthood. BMJ. 2001;323:191. doi: 10.1136/bmj.323.7306.191. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Sourander A, Helstelä L, Ristkari T, Ikäheimo K, Helenius H, Piha J. Child and adolescent mental health service use in Finland. Soc Psychiatry Psychiatr Epidemiol. 2001;36:294–298. doi: 10.1007/s001270170047. [DOI] [PubMed] [Google Scholar]
  • 10.Verhulst FC, Van der Ende J. Factors associated with child mental health service use in the community. Child Adolesc Psychiatry. 1997;36:901–909. doi: 10.1097/00004583-199707000-00011. [DOI] [PubMed] [Google Scholar]
  • 11.Zwaanswijk M, Van der Ende J, Verhaak PF, Bensing JM, Verhulst FC. Factors associated with adolescent mental health service need and utilization. J Am Acad Child Adolesc Psychiatry. 2003;42:692–700. doi: 10.1097/01.CHI.0000046862.56865.B7. [DOI] [PubMed] [Google Scholar]
  • 12.Vanheusden K, Van der Ende J, Mulder CL, Van Lenthe FJ, Verhulst FC, Mackenbach JP. The use of mental health services among young adults with emotional and behavioural problems: equal use for equal needs? Soc Psychiatry Psychiatr Epidemiol. 2008;43:808–815. doi: 10.1007/s00127-008-0367-4. [DOI] [PubMed] [Google Scholar]
  • 13.Rickwood DJ, Deane FP, Wilson CJ. When and how do young people seek professional help for mental health problems? Med J Aust. 2007;187:S35–S39. doi: 10.5694/j.1326-5377.2007.tb01334.x. [DOI] [PubMed] [Google Scholar]
  • 14.Kohl L, Crutzen R, De Vries NK. Online prevention aimed at lifestyle behaviours: a systematic review of reviews. J Med Internet Res. 2013;15:e146. doi: 10.2196/jmir.2665. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Countries with highest penetration rateshttp://www.internetworldstats.com/top25.htm
  • 16.Gould MS, Munfakh JL, Lubell K, Kleinman M, Parker S. Seeking help from the Internet during adolescence. J Am Acad Child Adolesc Psychiatry. 2002;41:1182–1189. doi: 10.1097/00004583-200210000-00007. [DOI] [PubMed] [Google Scholar]
  • 17.Suzuki LK, Calzo JP. The search for peer advice in cyberspace: an examination of online teen bulletin boards about health and sexuality. Appl Deve Psychol. 2004;25:685–698. doi: 10.1016/j.appdev.2004.09.002. [DOI] [Google Scholar]
  • 18.Crutzen R, De Nooijer J, Brouwer W, Oenema A, Brug J, De Vries NK. A conceptual framework for understanding and improving adolescents’ exposure to Internet-delivered interventions. Health Promot Int. 2009;24:277–284. doi: 10.1093/heapro/dap018. [DOI] [PubMed] [Google Scholar]
  • 19.Crutzen R, De Nooijer J, Brouwer W, Oenema A, Brug J, De Vries NK. Qualitative assessment of adolescents’ views about improving exposure to Internet-delivered interventions. Health Educ. 2008;108:105–116. doi: 10.1108/09654280810855568. [DOI] [Google Scholar]
  • 20.Havas J, De Nooijer J, Crutzen R, Feron F. Adolescents’ views about an Internet platform for adolescents with mental health problems. Health Educ. 2011;111:164–176. doi: 10.1108/09654281111123466. [DOI] [Google Scholar]
  • 21.Van Rooij AJ, Schoenmakers TM, Meerkerk G-J, Van de Mheen D. Wat Doen Jongeren op Internet en hoe Verslavend is dit? [What are Young People Doing on the Internet and how Addictive is This?] Rotterdam: IVO; 2009. [Google Scholar]
  • 22.Dowling M, Rickwood D. Online counseling and therapy for mental health problems: a systematic review of individual synchronous interventions using chat. J Technol Hum Serv. 2013;31:1–21. doi: 10.1080/15228835.2012.728508. [DOI] [Google Scholar]
  • 23.Gerrits RS, Van der Zanden RAP, Visscher RFM, Conijn BP. Master your mood online: a preventive chat group intervention for adolescents. Aus Adv Mental Health. 2007;6:e1–e11. doi: 10.5172/jamh.6.1.1. [DOI] [Google Scholar]
  • 24.Zabinski MF, Wilfley DE, Calfas KJ, Winzelberg AJ, Taylor CB. An interactive psychoeducational intervention for women at risk of developing an eating disorder. J Consult Clin Psychol. 2004;72:914–919. doi: 10.1037/0022-006X.72.5.914. [DOI] [PubMed] [Google Scholar]
  • 25.Scealy M, Phillips JG, Stevenson R. Shyness and anxiety as predictors of patterns of Internet usage. Cyber Psychol Behav. 2002;5:507–515. doi: 10.1089/109493102321018141. [DOI] [PubMed] [Google Scholar]
  • 26.Crutzen R, De Nooijer J. Intervening via chat: an opportunity for adolescents’ mental health promotion? Health Promot Int. 2011;26:238–243. doi: 10.1093/heapro/daq062. [DOI] [PubMed] [Google Scholar]
  • 27.Crutzen R, De Nooijer J, De Vries NK. How to reach a target group with Internet-delivered interventions? Eur Health Psychol. 2008;10:77–79. [Google Scholar]
  • 28.Riper H, Van Ballegooijen W, Kooistra L, De Wit J, Donker T. Preventie & eMental-health [Prevention & eMental-health] Amsterdam: VU Amsterdam; 2013. [Google Scholar]
  • 29.Muris P, Meesters C, Van den Berg F. The Strengths and Difficulties Questionnaire (SDQ) - further evidence for its reliability and validity in a community sample of Dutch children and adolescents. Eur Child Adolesc Psychiatry. 2003;12:1–8. doi: 10.1007/s00787-003-0298-2. [DOI] [PubMed] [Google Scholar]
  • 30.Van Widenfelt BM, Goedhart AW, Treffers PD, Goodman R. Dutch version of the Strengths and Difficulties Questionnaire (SDQ) Eur Child Adolesc Psychiatry. 2003;12:281–289. doi: 10.1007/s00787-003-0341-3. [DOI] [PubMed] [Google Scholar]
  • 31.Postma S. JGZ-richtlijn ‘Vroegsignalering van psychosociale problemen’. Bilthoven: RIVM; 2008. [Google Scholar]
  • 32.Vogels T. Vergelijking van de KIVPA en de SDQ als signaleringsinstrument voor problemen bij adolescenten in de Jeugdgezondheidszorg. Leiden, The Netherlands: TNO; 2011. [Google Scholar]
  • 33.Heller C, Balls-Berry JE, Nery JD, Erwin PJ, Littleton D, Kim M, Kuo WP. Strategies addressing barriers to clinical trial enrollment of underrepresented populations: a systematic review. Contemp Clin Trials. 2014;39:169–182. doi: 10.1016/j.cct.2014.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Patel V, Flisher AJ, Hetrick S, McGorry P. Mental health of young people: a global public-health challenge. Lancet. 2007;369:1302–1313. doi: 10.1016/S0140-6736(07)60368-7. [DOI] [PubMed] [Google Scholar]
  • 35.Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: a systematic review. BMC Psychiatry. 2010;10:113. doi: 10.1186/1471-244X-10-113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Crutzen R, De Nooijer J, Brouwer W, Oenema A, Brug J, De Vries NK. Effectiveness of online word of mouth on exposure to an Internet-delivered intervention. Psychol Health. 2009;24:651–661. doi: 10.1080/08870440802521094. [DOI] [PubMed] [Google Scholar]
  • 37.Vostanis P. Strengths and difficulties questionnaire: research and clinical applications. Curr Opin Psychiatry. 2006;19:367–372. doi: 10.1097/01.yco.0000228755.72366.05. [DOI] [PubMed] [Google Scholar]
  • 38.Achenbach TM, Rescorla LA, Ivanova MY. International epidemiology of child and adolescent psychopathology I: diagnoses, dimensions, and conceptual issues. J Am Acad Child Adolesc Psychiatry. 2012;51:1261–1272. doi: 10.1016/j.jaac.2012.09.010. [DOI] [PubMed] [Google Scholar]
  • 39.He J-P, Burstein M, Schmitz A, Merikangas KR. The strengths and difficulties questionnaire (SDQ): the factor structure and scale validation in U.S. Adolescents. J Abnorm Child Psychol. 2013;41:583–595. doi: 10.1007/s10802-012-9696-6. [DOI] [PubMed] [Google Scholar]
  • 40.Winding TN, Andersen JH, Labriola M, Nohr EA. Initial non-participation and loss to follow-up in a Danish youth cohort: implications for relative risk estimates. J Epidemiol Community Health. 2014;68:137–144. doi: 10.1136/jech-2013-202707. [DOI] [PubMed] [Google Scholar]
  • 41.Bradford S, Rickwood DJ. Adolescent’s preferred modes of delivery for mental health services. Child Adolesc Mental Health. 2014;19:39–45. doi: 10.1111/camh.12002. [DOI] [PubMed] [Google Scholar]
  • 42.Glenny A-M, Worthington HV, Milsom KM, Rooney E, Tickle M. Strategies for maximizing consent rates for child dental health surveys: a randomised controlled trial. BMC Med Res Methodol. 2013;13:108. doi: 10.1186/1471-2288-13-108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Allen J, Stapleton H, Tracy S, Kildea S. Is a randomised controlled trial of a maternity care intervention for pregnant adolescents possible? An Australian feasibility study. BMC Med Res Methodol. 2013;13:138. doi: 10.1186/1471-2288-13-138. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Homer CS. Using the Zelen design in randomized controlled trials: debates and controversies. J Adv Nurs. 2002;38:200–207. doi: 10.1046/j.1365-2648.2002.02164.x. [DOI] [PubMed] [Google Scholar]
  • 45.Hunninghake DB, Darby CA, Probstfield JL. Recruitment experience in clinical trials: literature summary and annotated bibliography. Control Clin Trials. 1987;8:6S–30S. doi: 10.1016/0197-2456(87)90004-3. [DOI] [PubMed] [Google Scholar]
  • 46.Glasgow RE, Eakin EG, Toobert DJ. How generelizable are the results of diabetes self-management research? The impact of participation and attrition. Diabetes Educ. 1996;22:573–585. doi: 10.1177/014572179602200605. [DOI] [PubMed] [Google Scholar]
  • 47.Glasgow RE, Lichtenstein E, Marcus AC. Why don’t we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93:1261–1267. doi: 10.2105/AJPH.93.8.1261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, Glasziou P, Ilott I, Kinmonth A-L, Leng G, Eccles MP, Armstrong D, Baker R, Cleary K, Davies H, Davies S, Glasziou P, Ilott I, Kinmonth AL, Leng G, Logan S, Marteau T, Michie S, Rogers H, Rycroft-Malone J, Sibbald B. An implementation research agenda. Implement Sci. 2009;4:18. doi: 10.1186/1748-5908-4-18. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from BMC Research Notes are provided here courtesy of BMC

RESOURCES