Abstract
Although evidence-based interventions exist, estimates suggest that about 60% percent of children and adolescents with mental health disorders do not receive the treatment they need. In this context, one expanding strategy for increasing access to mental health care for children and adolescents is the use therapeutic, or serious, games. REThink is one such therapeutic game, developed to offer a CBT-based prevention that was documented in a controlled trial to develop psychological resilience in children and adolescents, aged between 10 and 16, helping them learn healthy strategies for coping with dysfunctional negative emotions such as anxiety, anger and depression. This study aims to test the effectiveness of the REThink therapeutic online video game in promoting emotional health in children and adolescents in a pilot study.
Participants (N = 31), aged between 10 and 16 years, were recruited on a volunteer basis from a school. Emotional problems, irrational beliefs, negative automatic thoughts, rational beliefs, and problem solving abilities were assessed pre- and post-using the therapeutic game. We also measured participants' satisfaction with the game. Results obtained show improvements in terms of emotional problems of the youths, their irrational beliefs, negative automatic thoughts and high levels of intervention satisfaction. of this study are in support of the previous findings suggesting that the REThink online game can be a valuable tool for large-scale mental health efforts aimed at the prevention of emotional disorders in children and adolescents, in accordance with evidence-based prevention protocols.
Keywords: Therapeutic game, Effectiveness, Emotional symptoms, Emotion-regulation, Satisfaction
Highlights
-
•
Therapeutic games can increase access to mental health care for children and adolescents and its attractiveness
-
•
The REThink therapeutic game aims at the prevention of emotional disorders in children and adolescents, aged between 10-16
-
•
The REThink therapeutic online video game is effective in promoting mental health in children and adolescents
-
•
Future studies will further investigate its effectiveness in naturalistic studies.
1. Introduction
Currently, child mental health services are a priority on the world mental health care agenda (Kieling et al., 2011; Nock et al., 2013; Patel et al., 2013), as research indicates that untreated mental illness has dire societal costs, as well as individual ones, such as lower academic and professional achievements, and a worse quality of life (Kilian et al., 2011). More so, early onset of mental disorders has a considerable degree of lifetime persistence (Merikangas et al., 2010; Merikangas et al., 2009).
Globally, government spending on child mental health care is considered to be insufficient, especially in low-middle-income countries (Kassebaum et al., 2014). The lack of sufficient resources allocated for mental health tends to incur further individual and institutional barriers to care, such as individual costs associated with treatment, and institutional costs associated with adequately trained therapists or available facilities (Kassebaum et al., 2014; Mukolo et al., 2010). As such, although evidence-based interventions exist (Weisz et al., 2005), estimates suggest that about 60% percent of children and adolescents with mental health disorders do not receive the care they need (Bijl et al., 2003; Patulny et al., 2013).
In the context of this care gap, the necessity to improve access to mental health care for children and adolescents is stringent. One expanding strategy for increasing access to mental health care for children and adolescents is the use of therapeutic, or serious games (Ceranoglu, 2010). As computers, smartphones, and tablets are becoming ever more ubiquitous, the use of computerized games found its way into clinical care of youth. Serious games have been developed to facilitate psychotherapeutic interventions for children and adolescents, and there is already research indicating that they can be effective in addressing a variety of mental health symptoms (Horne-Moyer et al., 2014). However, although previous research pinpoints serious games as a promising solution for improving access to care and engaging youth, studies also highlight the need for more theory-based, effective, serious games, with clearly specified intervention protocols, that can be used for preventing mental disorders in children and adolescents.
In this context, the REThink game was designed to respond to these concerns, aiming to offer a theory-based prevention tool that can develop psychological resilience and reduce emotional problems in children and adolescents. REThink is anchored in the Rational Emotive Behavior Therapy (REBT; David et al., 2010), a psychotherapeutic approach that focuses on preventing and reducing psychopathology by helping people change their irrational beliefs (absolutistic shoulds/musts and rigid demands regarding their goals) and cultivate their rational beliefs, such as flexible preferences in relation to their goals. The game was documented in a recent clinical trial (David et al., 2019a) to help children and adolescents aged between 10 and 16 years to improve their emotional symptoms (e.g., depression and anxiety), emotion-regulation, and reduce irrational thinking.
This study aims to advance the knowledge we currently have on the effects of the therapeutic game under controlled conditions and pilot test the effectiveness of the REThink therapeutic online video game in promoting mental health in children and adolescents, as part of their naturalistic community school-based services.
2. Methods
2.1. Participants
Thirty-one children and adolescents, 18 girls and 13 boys (Mage = 12.25; SD = 1.43), participated in the present study. They were recruited on a voluntary basis from one middle school located in a small urban community, based on their appointment at the Counseling Center in the school. Informed consent to participate in the experiment was obtained from their parents and school authorities. The study was approved by the University's Institutional Review Board.
2.2. The REThink game description
REThink is a therapeutic videogame meant to be used as a standalone application to promote emotional resilience in children and adolescents. The game includes a main character, RETMAN, associated to Rational-Emotive Behavioral Therapy (David, 2010; Ellis, 1995) and five “rational” friends: Preferilizer (representing preferences beliefs), Ponderancer (representing non-awfulizing beliefs), Toleraser (representing high frustration tolerance beliefs), Acceptableizer (representing unconditional acceptance beliefs) and Optimizer (representing happiness). RETMAN has an enemy, called Irrationalizer, who promotes irrational thinking together with his servants: Necessitizer (representing demandingness beliefs), Awfulizer (representing awfulizing beliefs), Frustralizer (representing low frustration tolerance beliefs) and Discourager (representing global evaluation beliefs). Player's mission is to help people on the planet to escape from the power of Irrationalizer, by helping people to be more rational and happier. The game has seven levels (see Fig. 1) situated on different territories on Earth. At the end of each level the player wins the key to go into the next level. Each level has a trial part at the beginning, in which RETMAN explains what the player has to accomplish. Each level has various degrees of complexity, which increase as the player progresses in the game. The scenario of the game was developed based on the REBT model, such that it focuses on: a. identifying the emotional and behavioral reactions; b. identifying cognitive processes; c. identifying the relation between cognitive processes, emotions, and behavioral reactions; d. changing irrational cognitions into rational cognitions; e. building problem solving skills; f. building relaxation skills, and g. building happiness skills.
Fig. 1.
REThink game levels illustration.
2.3. Procedure
The REThink game was delivered sequentially, during three sessions of approximately 60 min each. In the first session, the participants played the first 3 levels of the REThink game. The second session included level 4 and 5, while the third session comprised the last level.
Participants played each level on an Apple iPad Air 2. After they arrived in the experiment room, children and adolescents completed the pre-intervention measures (i.e., rational and irrational beliefs level, negative automatic thoughts, negative emotional symptoms, and problem solving abilities). Further, they were asked to start playing the game. Finally, after completing all levels (over 3 consecutive weeks), assessments with children and adolescents were conducted. Before leaving the experiment room from every session, participants were debriefed and thanked for their participation.
2.4. Measures
Primary outcome considered was emotional problems of the children, and secondary outcomes were cognitive changes (irrational beliefs and negative automatic thoughts), problem solving abilities and user satisfaction with the game intervention.
Strengths and Difficulties Questionnaire - child version (SDQ; Goodman et al., 1998) is a 25-item scale that was used to assess emotional problems in children and adolescents. The SDQ comprises five subscales that measure difficulties in several psychological domains: emotional symptoms, conduct problems, hyperactivity-attention, peer problems, and prosocial behavior. Each item is scored on a three-point Likert scale, with 0 (not true), 1 (somewhat true) and 2 (certainly true) answers. The SDQ - child version has demonstrated acceptable psychometric properties (Goodman et al., 1998).
Attitudes and Beliefs Scale, Short form (ABSs; David, 2006) is a shortened, 8-item scale of the ABS 2 (DiGiuseppe et al., 1988) that was used to assess the rational (RBs) and irrational beliefs (IBs), consistent with REBT theory. The scale reflects four irrational beliefs (demandingness - DEM, global evaluation - GE, low frustration tolerance - LFT awfulizing - AWF) and four rational beliefs (preferences - PREF, no global evaluation – non GE, frustration tolerance - FT, badness - BAD). Each item is scored on a five-point Likert scale, from 0 (strongly disagree) to 4 (strongly agree). The ABSs has been demonstrated to be a reliable and valid measure of RBs and IBs (e.g., David, 2006).
The Child and Adolescent Scale of Irrationality (CASI; Bernard and Cronan, 1999) is a 28- item scale that was used to measure irrational cognitions in children and adolescents. Items reflect irrational cognitions in several domains: demandingness for fairness (DEM—F), low frustration tolerance for work (LFT-W), low frustration tolerance of rules (LFT-R), and global evaluation (GE) of self and others. Children and adolescents were asked to express their agreement/disagreement with the 28 statements on a 5-point Likert scale, from 1 (strong disagreement) to 5 (strong agreement). The CASI has adequate psychometric properties (see Trip and Popa, 2005).
Children's Automatic Thoughts Scale-Negative/Positive (CATS-N/P; Hogendoorn et al., 2010) is a 50-item questionnaire that assesses negative and positive automatic thoughts in children and adolescents. The questionnaire comprises five subscales: physical threat (e.g., “I'm going crazy”), social threat (e.g., “I'm going to look silly”), personal failure (e.g., “I am worthless”), hostility (e.g., “Other kids are stupid”), and positive thoughts (e.g. “I don't give up”). Children and adolescents rate to what extent they had a specific thought over the past week on a 5-point Likert scale, from 0 (not at all) to 4 (all the time). The CATS-N/P has demonstrated good psychometric properties (Hogendoorn et al., 2010).
Problem Solving Abilities - Visual Analogue Scale (VAS; Ohnhaus and Adler, 1975) is a self-report visual analogue scale that was used in the present study to assess problem solving abilities in children and adolescents. Participants were asked to rate how prepared they are to solve problems that they face in everyday life on a 10 cm visual analogue scale, with anchors from 0 (not at all) to 10 (very well). Numerous studies validated the use of the VAS with children and adolescents. The VAS has demonstrated good psychometric properties (von Baeyer, 2006).
Treatment Satisfaction Visual Analogue Scales (TS-VAS; Zecca et al., 2014) is a self-report visual analogue scale that was used to measure children and adolescents' satisfaction with the intervention. Participants were asked to rate on a 10 cm visual analogue scale, with the anchors 0 (not at all) and 10 (very much), how much they think they would like the REThink game (at pre-intervention), as well as how much they actually liked the REThink game (at post-intervention). Studies investigating participants' satisfaction with a treatment demonstrate that VAS is a suitable measure with good psychometric properties (Brokelman et al., 2012).
3. Results
Means and standard deviations of variables investigated within the study are summarized in Table 1.
Table 1.
Descriptive statistics for the main outcomes.
| Variables | N | Pre-intervention | Post-intervention | Paired t-test | Effect size |
|---|---|---|---|---|---|
| ABSs irrational beliefs | 17 | M = 5.82 (SD = 2.50) | M = 4.41 (SD = 2.31) | t(16) = 2.48, p = .02 | d = 0.58 |
| ABSs rational beliefs | 15 | M = 10.86 (SD = 2.61) | M = 11.46 (SD = 2.94) | t(14) = −1.31, p = .20 | d = 0.21 |
| CASI | 28 | M = 73.78 (SD = 9.82) | M = 70.75 (SD = 9.85) | t(27) = 2.25, p = .03 | d = 0.30 |
| CATS negative total | 29 | M = 32.06 (SD = 17.50) | M = 23.79 (SD = 22.03) | t(28) = 2.93, p = .08 | d = 0.41 |
| SDQ emotional problems | 24 | M = 3.50 (SD = 2.87) | M = 2.37 (SD = 2.49) | t(23) = 2.40, p = .02 | d = 0.42 |
| Problem solving abilities | 22 | M = 8.29 (SD = 2.79) | M = 8.47 (SD = 2.29) | t(21) = −0.30, p = .76 | d = 0.07 |
| Satisfaction with the intervention | 20 | M = 8.65 (SD = 2.97) | M = 8.70 (SD = 2.22) | t(19) = −0.08, p = .93 | d = 0.01 |
Note: ABSs = Attitudes Beliefs Scale Short-Form, CASI = The Child and Adolescent Scale of Irrationality, CATS = Children's Automatic Thoughts Scale, SDQ = The Strengths and Difficulties Questionnaire, M = mean, SD = standard deviation, N = number of participants, d = Cohen's d.
To examine changes in the dependent measures from pre to post-treatment, we performed Paired t-test, with 1) REThink intervention as independent variable and 2) treatment outcomes as dependent variables. The analysis yielded significant improvements in emotional problems level, measured with SDQ emotional subscale, t(23) = 2.40, p = .02, irrational beliefs level, measured with ABSs, t(16) = 2.48, p = .02, and CASI, t(27) = 2.25, p = .03, and marginally in negative automatic thoughts level, measured with CATS, t(28) = 2.93, p = .08. No significant improvements were identified tough for rational beliefs level, ABSs, t(14) = −1.31, p = .20, and problem solving abilities, t(21) = −0.30, p = .76. Satisfaction with the intervention remained unchanged t(19) = −0.08, p = .93.
4. Discussion
The present study aimed to investigate the effectiveness of the REThink game in improving emotional resilience in children in a community setting.
Results indicate that REThink was effective in reducing emotional problems, irrational beliefs and negative automatic thoughts, with moderate effect sizes (ranging from 0.30–0.58). Considering the dearth of REBT literature indicating that irrational beliefs are the psychological cornerstone of children and adolescent emotional and behavioral problems (see David, 2020), our results provide further support for the changes in outcomes and the previously documented mechanisms of the game (David et al., 2019b), namely irrational beliefs which were considered secondary outcomes in this study.
Small effect sizes have been obtained for increasing rational beliefs without significant changes, a result that falls in line with other findings from REBT intervention. The REBT goal when working with young people is not only the elimination of irrational beliefs (and associated unhealthy emotions and self- defeating behaviors), but also helping them to acquire, strengthen and practice rational beliefs (David and Cramer, 2009; David et al., 2010). Thus, it might be that participants need to use the game for a longer duration in order to obtain higher magnitude of changes. Also, since the effect sizes obtained for problem-solving abilities are modest as well, this opens the discussion for potentially adding more intense practice within the game. We found that children made accurate predictions of their satisfaction with the game initially, and the high level of the satisfaction was maintained after playing the game (non-significant pre-post changes). This result is important since a high user satisfaction shows the potential that the REThink game has to engage the youth (see also David et al., 2018).
The present study was not without limitations. This field study includes a small sample, which limits our ability to generalize the findings. However, this study builds on our previous randomized clinical trial results (David et al., 2019a, David et al., 2019b; David et al., 2020b; Predatu et al., 2021) as the first step of our efforts to comprehensively assess both the efficacy and effectiveness of the REThink therapeutic game. Indeed, while it is important to know that the REThink game is effective based on controlled studies, it is essential to also document if it can have similar effectiveness in naturalistic community settings, as part of the mental health support services offered. Furthermore, our results suggest that more time for practice might be needed for the youth to strengthen their emotional skills and obtain stronger outcomes. Future studies will need to investigate the effectiveness of the game in a larger community samples. Also, given that in a recent review (see David et al., 2020a) we found a low maintenance rate of gains, the lack of follow-up is an important limitation of this study, and thus our future studies need to investigate the long term maintenance of the improvements brought by the REThink therapeutic game.
Results of this study are in support of the previous findings suggesting that the REThink online game can be a valuable tool for large-scale mental health efforts to help children and adolescents reduce their emotional symptoms, negative automatic thoughts and irrational beliefs, in accordance with evidence-based prevention programs. REThink offers thus an innovative, theory-based approach for increasing children and adolescents' resilience, integrating an evidence-based prevention package into a therapeutic online game for promoting mental health.
Funding
This work was supported by a grant awarded to Oana A. David from the Romanian National Authority for Scientific Research, CNCS – UEFISCDI [grant number PN-III-P4-ID-PCE-2020-2170 and project number PN-III-P2-2.1-PED-2019-3837].
References
- Bernard M.E., Cronan F. The child and adolescent scale of irrationality: validation data and mental health correlates. J. Cogn. Psychother. 1999;13(2):121–132. [Google Scholar]
- Bijl R.V., de Graaf R., Hiripi E., Kessler R.C., Kohn R., Offord D.R.…Wittchen H.U. The prevalence of treated and untreated mental disorders in five countries. Health Aff. 2003;22(3):122–133. doi: 10.1377/hlthaff.22.3.122. [DOI] [PubMed] [Google Scholar]
- Brokelman R.B., Haverkamp D., van Loon C., Hol A., van Kampen A., Veth R. The validation of the visual analogue scale for patient satisfaction after total hip arthroplasty. Eur. Orthop. Traumatol. 2012;3(2):101–105. doi: 10.1007/s12570-012-0100-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ceranoglu T.A. Video games in psychotherapy. Rev. Gen. Psychol. 2010;14(2):141–146. doi: 10.1037/a0019439. [DOI] [Google Scholar]
- DiGiuseppe R., Leaf R., Exner T., Robin M.W. Paper presented at the World Congress of Behavior Therapy, Edinburgh, Scotland, September. 1988. The development of a measure of rational/irrational thinking. [Google Scholar]
- David D. Polirom; Iasi: 2006. Handbook of Cognitive and Behavioral Psychotherapies. [Tratat de Psihoterapii Cognitive ¸si Comportamentale] [Google Scholar]
- David D. RTS Cluj; Cluj-Napoca, CJ: 2010. Retmagia şi minunatele aventuri ale lui Retman. Poveşti raţionale pentru copii şi adolescenţi [Retmagic and the wonderful adventures of Retman. Stories for children and adolescents] [Google Scholar]
- David O.A. Rational emotive parent education. In: Bernard M., Terjesen M., editors. Rational-Emotive and Cognitive-Behavioral Approaches to Child and Adolescent Mental Health: Theory, Practice, Research, Applications. Springer; 2020. [Google Scholar]
- David D., Cramer D. Rational and irrational beliefs in human feelings and psychophysiology. In: David D., Lynn S.J., Ellis A., editors. Rational and Irrational Beliefs: Research, Theory, and Clinical Practice. Oxford University Press, Inc; New-York: 2009. [Google Scholar]
- David D., Lynn S.J., Ellis A. Oxford University Press; New York: 2010. Rational and Irrational Beliefs: Research, Theory, and Clinical Practice. [Google Scholar]
- David O.A., Predatu R., Cardos R. A pilot study of the REThink online video game applied for coaching emotional understanding in children and adolescents in the therapeutic video game environment: the feeling better resources game. J. Evid. Based Psychother. 2018;18(1):57–68. doi: 10.24193/jebp.2018.1.5. [DOI] [Google Scholar]
- David O.A., Cardoș R.A.I., Matu S.A. Is REThink therapeutic game effective in preventing emotional disorders in children and adolescents? Outcomes of a randomized clinical trial. Eur. Child Adolesc. Psychiatry. 2019:1–12. doi: 10.1007/s00787-018-1192-2. [DOI] [PubMed] [Google Scholar]
- David O.A., Cardoș R.A.I., Matu S.A. Changes in irrational beliefs are responsible for the efficacy of the REThink therapeutic game in preventing emotional disorders in children and adolescents: mechanisms of change analysis of a randomized clinical trial. Eur. Child Adolesc. Psychiatry. 2019;28(3):307–318. doi: 10.1007/s00787-018-1195-z. [DOI] [PubMed] [Google Scholar]
- David O.A., Costescu C., Cardos R., Mogoase C. How effective are serious games for promoting mental health and health behavioral change in children and adolescents? A systematic review and meta-analysis. Child Youth Care Forum. 2020 doi: 10.1007/s10566-020-09566-1. [DOI] [Google Scholar]
- David O.A., Predatu R., Maffei A. REThink online video game for children and adolescents: effects on state anxiety and frontal alpha asymmetry. Int. J. Cogn. Ther. 2020 doi: 10.1007/s41811-020-00077-4. [DOI] [Google Scholar]
- Ellis A. New approaches to psychotherapy techniques. J. Clin. Psychol. 1995;11(3):207–260. doi: 10.1002/1097-4679. [DOI] [PubMed] [Google Scholar]
- Goodman R., Meltzer H., Bailey V. The strengths and difficulties questionnaire: a pilot study on the validity of the self-report version. Eur. Child Adolesc. Psychiatry. 1998;7(3):125–130. doi: 10.1007/s007870050057. [DOI] [PubMed] [Google Scholar]
- Hogendoorn S.M., Wolters L.H., Vervoort L., Prins P.J., Boer F., Kooij E., De Haan E. Measuring negative and positive thoughts in children: an adaptation of the Children’s Automatic Thoughts Scale (CATS) Cogn. Ther. Res. 2010;34(5):467–478. doi: 10.1007/s10608-010-9306-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Horne-Moyer H.L., Moyer B.H., Messer D.C., Messer E.S. The use of electronic games in therapy: a review with clinical implications. Curr. Psychiatry Rep. 2014;16(12) doi: 10.1007/S11920-014-0520-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kassebaum N.J., Bertozzi-Villa A., Coggeshall M.S., Shackelford K.A., Steiner C., Heuton K.R.…Lozano R. Global, regional, and national levels and causes of maternal mortality during 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2014;384(9947):980–1004. doi: 10.1016/S0140-6736(14)60696-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kieling C., Baker-Henningham H., Belfer M., Conti G., Ertem I., Omigbodun O.…Rahman A. Child and adolescent mental health worldwide: evidence for action. Lancet. 2011;378(9801):1515–1525. doi: 10.1016/S0140-6736(11)60827-1. [DOI] [PubMed] [Google Scholar]
- Kilian R., Losert C., McDaid D., Park A., Knapp M. Cost-effectiveness analysis in child and adolescent mental health problems: an updated review of the literature. Psychiatr. Prax. 2011;38(S 01):S03–S04. doi: 10.1055/s-0031-1277752. EC. [DOI] [Google Scholar]
- Merikangas K.R., Nakamura E.F., Kessler R.C. Epidemiology of mental disorders in children and adolescents. Dialogues Clin. Neurosci. 2009;11(1):7–20. doi: 10.31887/DCNS.2009.11.1/krmerikangas. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Merikangas K.R., He J.P., Brody D., Fisher P.W., Bourdon K., Koretz D.S. Prevalence and treatment of mental disorders among US children in the 2001-2004 NHANES. Pediatrics. 2010;125(1):75–81. doi: 10.1542/peds.2008-2598. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mukolo A., Heflinger C.A., Wallston K.A. The stigma of childhood mental disorders: a conceptual framework. J. Am. Acad. Child Adolesc. Psychiatry. 2010;49(2):92–103. doi: 10.1016/j.jaac.2009.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nock M.K., Green J.G., Hwang I., McLaughlin K.A., Sampson N.A., Zaslavsky A.M., Kessler R.C. Prevalence, correlates, and treatment of lifetime suicidal behavior among adolescents results from the National Comorbidity Survey Replication Adolescent Supplement. Jama Psychiatry. 2013;70(3):300–310. doi: 10.1001/2013.jamapsychiatry.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ohnhaus E.E., Adler R. Methodological problems in the measurement of pain: a comparison between the verbal rating scale and the visual analogue scale. Pain. 1975;1(4):379–384. doi: 10.1016/0304-3959(75)90075-5. [DOI] [PubMed] [Google Scholar]
- Patel V., Kieling C., Maulik P.K., Divan G. Improving access to care for children with mental disorders: a global perspective. Arch. Dis. Child. 2013;98(5):323–327. doi: 10.1136/archdischild-2012-302079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Patulny R., Muir K., Powell A., Flaxman S., Oprea I. Are we reaching them yet? Service access patterns among attendees at the headspace youth mental health initiative. Child Adolesc. Mental Health. 2013;18(2):95–102. doi: 10.1111/j.1475-3588.2012.00662.x. [DOI] [PubMed] [Google Scholar]
- Predatu R., David O.A., David D.O., Maffei A. Emotion regulation abilities as a predictor of emotional response experienced by children and adolescents in the face of stress. Anxiety Stress Coping. 2021;15(1):1061–5806. doi: 10.1080/10615806.2020.1867717. [DOI] [PubMed] [Google Scholar]
- Trip S., Popa M. CASI–the child and adolescent scale of irrationality. Analele Universitătii din Oradea Fascicula Psihologie. 2005;8(2):39–51. [Google Scholar]
- Von Baeyer C.L. Children’s self-reports of pain intensity: scale selection, limitations and interpretation. Pain Res. Manag. 2006;11(3):157–162. doi: 10.1155/2006/197616. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weisz J.R., Sandler I.N., Durlak J.A., Anton B.S. Promoting and protecting youth mental health through evidence-based prevention and treatment. Am. Psychol. 2005;60(6):628–648. doi: 10.1037/0003-066X.60.6.628. [DOI] [PubMed] [Google Scholar]
- Zecca C., Riccitelli G.C., Calabrese P., Pravatà E., Candrian U., Guttmann C.R., Gobbi C. Treatment satisfaction, adherence and behavioral assessment in patients de– escalating from natalizumab to interferon beta. BMC Neurol. 2014;14(1):38–46. doi: 10.1186/1471-2377-14-38. [DOI] [PMC free article] [PubMed] [Google Scholar]

