Abstract
Objective:
To develop, implement, and verify the impact of a training program for health care providers working with children with autism spectrum disorder (ASD) in psychosocial care centers for children and adolescents (Centro de Atenção Psicossocial à Infância e à Adolescência – CAPSi) in São Paulo, Brazil.
Methods:
This quasi-experimental study was conducted with 14 professionals from four CAPSi units. The training program consisted of six phases: 1) pre-intervention observation; 2) meeting with staff to assess the main needs of the training program; 3) developing materials for training and evaluation; 4) meetings to discuss program implementation; 5) a final meeting for case discussion and evaluation; and 6) distance supervision. Three measures were used to evaluate the training program: i) the Knowledge, Attitudes, and Practices (KAP) questionnaire; ii) videos containing questions designed to assess program comprehension; and iii) a satisfaction survey.
Results:
Thirteen videos were produced to as visual aids for use during the training program, and a further 26 videos were developed to evaluate it. The program was well evaluated by the participants. The video responses and KAP questionnaire scores suggest that staff knowledge and attitudes improved after training.
Conclusion:
The positive findings of this study suggest that the tested training program is feasible for use with multidisciplinary teams working in the CAPSi environment.
Keywords: Child psychiatry; autism; community mental health; education, psychiatric; interdisciplinary relations
Introduction
Autism spectrum disorder (ASD) affects different areas of an individual’s global development, primarily socialization, behavior, sensory functions, and language.1 The term spectrum indicates that a significant number of features and dimensions can be involved in the disorder, with differences mainly related to the severity of each symptom.2,3 Reports of ASD prevalence vary somewhat in the literature, but current systematic reviews estimate an overall prevalence of approximately 60-72:10,000.4,5
Due to the complexity and variability of clinical settings, specialized training is needed for professionals working with children who have ASD. Ideally, such training would be provided in an expanded form to cover aspects related to non-health sectors as well, such as education, social welfare, and justice.
The Brazilian Unified Health System (Sistema Único de Saúde – SUS) guarantees free health care for the population. It includes a mental health division, in which psychosocial care centers (Centro de Atenção Psicossocial – CAPS) are the main facilities through which care for individuals with severe and persistent mental health problems is provided. Several different types of CAPS facilities exist in the system, including psychosocial care centers for children and adolescents (Centro de Atenção Psicossocial à Infância e à Adolescência – CAPSi), which are responsible for individuals up to 24 years of age. The minimum personnel requirements for a CAPSi team include physicians experienced in child development, psychologists, nurses, occupational therapists, speech therapists, social workers, and educational specialists. To enable a broad approach to the diagnosis and management of people with mental health problems, teams must provide care from an interdisciplinary perspective.6,7 Therefore, CAPSi units are the most suitable SUS service for managing children and adolescents with ASD, although these units are both numerically insufficient and unevenly distributed on a national scale.6,8
Seeking to understand the CAPSi workflow, a number of studies have assessed the profiles of cases managed at these units and the availability of staff. A recent nationwide analysis of 837,259 CAPSi health records revealed that 65.8% of the care provided in these units focuses on three nosological groups: behavioral disorders and disorders that usually appear during childhood or adolescence (F90-F98), accounting for 29.7%; developmental disorders, including ASD (F80-F89), which account for 23.6%; and intellectual disability, accounting for 12.5%.8 Among CAPSi units in the state of São Paulo, children and adolescents with ASD occupy more available treatment slots than subjects with any other disorder.9 This suggests that children with ASD are among the most frequent CAPSi users nationwide, which is consistent with the nature of ASD as a disorder of severe, persistent symptoms that significantly compromise the adaptive behavior of the affected child/adolescent.
On the other hand, studies conducted in Brazil and abroad have shown that the lack of specialized training in mental health, particularly regarding autism, represents a major challenge to providing care for this population, which highlights the need for continuing education among professionals in the field.5,10 To meet this demand, several training models have been tested and implemented, especially in developed countries11; in Brazil, a number of domestic trials have been conducted in SUS primary care centers, and although preliminary, their findings are promising.12,13 Therefore, training models for professionals in specialized public services in Brazil are still lacking. Furthermore, since the evidence indicates that many children and adolescents with ASD are using CAPSi units nationwide, improving staff knowledge of ASD could be a good strategy for maximizing the use of existing SUS resources.
The National Policy for Continuing Health Education (Política Nacional de Educação Permanente em Saúde – PNEPS), a SUS strategy for the training and development of workers in the health sector, was launched in 2004 and redefined in 2007 by federal order.14 This policy is based on models of workplace education in which learning and teaching activities are incorporated into the daily routine. The professionals participating in the training program are viewed as the most important people in the process, since it is they who are most aware of their daily routine. Their active stake in building knowledge improves the possibility of effective change due to their professional commitment to the program’s goals and their own development.15 Thus, one of the principles of PNEPS is that the involved professionals participate in all processes, including decisions about scheduling.
One form of professional training recommended in PNEPS is the use of audiovisual material, which helps objectively show the characteristics of children with ASD and demonstrates the best course of care in terms of symptom management. The utility of audiovisual resources for training parents has been well established as one of the most effective approaches for treating children’s behavioral problems.16 Videos combine images, sounds, and speech with limited written content, thereby condensing ideas and reaching viewers through their different senses, which ultimately maximizes learning.17
Similarly, a number of studies have shown that the video-modeling technique is an effective visual tool for teaching new skills and behaviors; it has been used to train parents, professionals, and individuals with ASD.This method allows the viewer to acquire information related to the management of different behaviors and, subsequently, to use these models in their daily activities.16,18 A 2015 study developed a video modeling package to teach applied behavior analysis (ABA) techniques to caregivers of children with ASD, and found promising results.19 A review of the literature found several advantages in using video to improve parent guidance and concluded that this technique reduced the duration of interventions, promoted greater acceptance, and reduced costs.20
Considering the lack of Brazilian research about improving care for children with ASD, particularly within the CAPSi setting, the objective of this study was to develop, implement, and assess the impact of a training program aimed at professionals who work with these children in CAPSi units, focusing primarily on behavior management, communication, and social interaction problems. A secondary objective was to verify the impact of this program among trained professionals.
Methods
Based on a quasi-experimental study design, 14 participants were nominated by the managers of four CAPSi units managed by Coordenadoria Regional de Saúde Norte, city of São Paulo: four nursing assistants, three nurses, two speech therapists, an occupational therapist, a pharmacy technician, a support worker, an educational psychology specialist, and a nutritionist.
The study was approved by the local research ethics committee and by the ethics committee of the São Paulo Municipal Department of Health (protocol no. 45304515.3.3001.0086). The development, implementation, and evaluation of the training model lasted 14 months. The training program consisted of six phases: 1) pre-intervention observation; 2) meeting with participants to assess the main needs for the training program; 3) development of training and evaluation materials; 4) further meetings to discuss implementation of the training program; 5) a final meeting for case discussion and evaluation; and 6) distance supervision.
Study phases
Phase 1 – Pre-intervention observation
Before the intervention began, each of the four CAPSi units participating in the study was visited so the investigators could observe the professionals’ work routines and present the proposal to each unit manager. This observation component lasted 1 hour in each unit.
Phase 2 – Meeting with participants to assess the main needs of the training program
The main aim of this meeting was to take stock of the greatest difficulties encountered by the professionals regarding behavior management and the communication and social interaction difficulties of ASD children. To establish which subjects should be addressed in the training program, a list of the most relevant topics reported in the literature was presented to the 14 participants, who were given the opportunity to add any further topics they felt warranted attention. This set of subjects was analyzed and discussed in the group until the following consensus set of priority issues was defined: i) diagnosis and identification criteria and early intervention; ii) stereotypies (including echolalia, ritualization, hypo/hypersensitivity, and sexuality); iii) inappropriate behavior and hetero/auto-aggression; iv) how to prepare the therapeutic environment for care and how to conduct group sessions; and v) functional play and guidance for parents. It was decided that five 2-hour meetings would be held to develop these five topics, as well as a final meeting to discuss cases in each participating CAPSi unit. The final meeting consisted of 2-hour sessions in either the morning or afternoon.
Phase 3 – Development of training and evaluation materials
Once the topics of the training program had been defined, lectures on these themes were prepared and the respective videos were selected for evaluation. Moreover, new videos were selected for use during the training sessions.These videos were used to illustrate the lectures and provide action models for video modeling templates. The videos were selected from the first author’s filmed private practice (all obtained with parental consent) or obtained from YouTube.
Phase 4 – Additional meetings to discuss implementation of the training program
This phase took place at a single CAPSi unit in the North side of São Paulo, with sessions offered in the morning and afternoon to allow participants to choose the most convenient time slot. For operational reasons, the CAPSi management asked that the number of sessions be reduced from five to three. Pursuant to this request, we discussed two topics per 4-hour session.
The first training meeting was divided into two parts. In the first part, the staff completed the questionnaire and the video assessments. The second part consisted of a theoretical lecture on one of the defined topics. Videos were used to illustrate problem situations and to provide interview models for certain situations (video modeling), and a PowerPoint presentation explored specific themes and strategies. The other training meetings were divided into three parts. In the first part, cases related to previously addressed topics were discussed; in the second and third parts of the meetings, two subjects that had been selected during the first meeting were discussed.
Phase 5 – Final meeting for case discussion and evaluation
The final meeting was held at each of the four CAPSi units. Although this meeting was initially conceived to provide a moment of reflection for the professionals to raise any final issues, share experiences, and enhance learning in smaller groups or even individually, this did not take place as planned due to scheduling difficulties. Instead, these final meetings involved only quick discussion and administration of the post-training instruments (Knowledge, Attitudes, and Practices [KAP] questionnaire, video assessment, and satisfaction questionnaire), due to heavy staff workloads. This meeting lasted 1 hour at each unit.
Phase 6 – Distance supervision
This feature was offered to the participants to clarify any questions, discuss difficulties, and exchange experiences, as well as for other purposes, such as discussion on how to maintain patient confidentiality. The first author (LCS) was to conduct this component via telephone or Skype.However, none of the participants contacted LCS for such supervision.
Instruments used to evaluate the training program
KAP survey
KAP questionnaires have been used since the 1950s to collect data on various health problems, and can measure participant knowledge levels on specific subjects, as well as how respondents would act when facing specific problems.21 The KAP is not a preset questionnaire; it must be prepared with items specific to the particular research project. Thus, in this study, the KAP questionnaire included the topics to be addressed during the training program.The questionnaire consisted of 24 questions structured on a 10-point Likert scale, ranging from strongly disagree to strongly agree. The KAP was self-administered during the second and final meetings to identify any post-training improvements in the participants’ knowledge, attitudes, and practice/skill levels.
Video-based knowledge assessment
More than 70 hours of video recordings from previous studies of ASD children were analyzed.22 After watching all the videos, two investigators (including the first author) selected the parts of the videos containing most relevant content required for the training program, such as examples of stereotypy, echolalia, auto/hetero-aggression, and changes in joint attention. From this material, 28 videos involving situations related to the issues to be discussed in the training, each approximately 2 minutes long, were selected and edited: 14 to be shown during and 14 to be shown after the training. Two experts (the first and third authors of this paper) then prepared multiple-choice questions related to the video content. The questions were the same in both assessments, but the content of the videos, although featuring the same themes, was different at each stage to assess post-training changes in practice while avoiding a learning effect. At the end of the process, the videos and their corresponding questions were reviewed by three ASD specialists.
The training program participants watched the videos and answered one multiple-choice question on each video at the first and last training session. In the post-training phase, the decision was made to exclude one of the videos, since all participants had agreed on the same response during the pre-training stage.
Satisfaction questionnaire
This questionnaire was specifically designed and developed by the authors to assess participant satisfaction and verify the extent to which they intended to use the training program in the future by assessing their opinions about the content and training format. This measure, applied only during the final session, was included because the literature suggests it is an important method of evaluating the service/program being tested.23 Items included: “In your opinion, was the overall quality of the course: excellent, good, average, poor, or very poor?”; and “Would you recommend this training to a colleague who works at CAPSi (yes or no)?”
Statistical analysis
Data analysis was performed in SPSS version 20, and a significance level of 5% was used for all statistical tests.Any p-values > 0.05 and < 0.10 were considered marginally significant. Comparisons of the pre- and post-training response distributions for each video were performed using the McNemar test. A comparison of the overall pre- and post-training means (total correct responses), as well as of the total scores and KAP subscales, were performed using Student’s t-test for paired samples.24 Normality of data distribution was verified with the Kolmogorov-Smirnov test.
Results
Once the participants had agreed upon a set of topics to be covered by the program, during the training implementation meeting, the video prospecting process began. To develop the training and evaluation material, over 70 hours of footage of ASD children and adolescents in different situations were screened. These videos were selected from two sources: i) YouTube, using the following search criteria and the corresponding keywords (in Portuguese): autism, ASD, echolalia, stereotypy, tantrum autism, self-harm autism, aggressive autism, autism diagnosis, early identification autism, signs of autism, functional play, and playing autism; ii) videos from the private practice of one of the researchers (LCS), all obtained and used with the permission of the children’s parents.
The selected videos were edited in MovieMaker software to shorten them to between 1 and 4 minutes, focusing on the content necessary for evaluation or training, by presenting different aspects of the disorder.
The videos used in the evaluation and their respective questions were checked by two ASD experts (MCT, CSP). Table 1 describes the content of each video used in the study.
Table 1. Description of videos used for evaluation in the pre- and post-training phases.
| Video theme | Task in the pre-training phase | Task in the post-training phase |
|---|---|---|
| Stereotypy (management) | Video content: A child exhibits stereotyped movements while watching a musical cartoon; the mother tries to promote social interaction with other children to curtail these movements. | Video content: During a therapy session, a child does not look at the presented stimulus, but looks at his hand while performing stereotypic movements with it. |
| Question about the video: How is the child encouraged to engage in social interaction and how can stereotyped movements be reduced? | Question about the video: How can the therapist correctly manage the patient to reduce stereotyped movements? | |
| Correct answer: Wait for the video to finish, turn off the computer, and then talk to the child. | Correct answer: Redirect the behavior by offering the child something to hold. | |
| Sensory hypo-/hypersensitivity | Video content: A child rubs his elbow uninterruptedly during a therapy session. | Video content: A child exhibits hand-to-mouth behavior during a therapy session, which ultimately provokes vomiting. |
| Question about the video: Knowing that this child would rub his arm until it bleeds and would not complain of pain, what would you do? | Question about the video: Knowing that this child would not remove his hand from his mouth until he vomited, what would you do? | |
| Correct answer: I would divert attention from this behavior by providing something to do with his hands. | Correct answer: I would divert attention from this behavior by providing something to do with his hands. | |
| Self-harm | Video content: A child in bed in the dark (probably at bedtime) self-harms by banging his head against the wall while crying. | Video content: A child appears sitting at the table on an adult’s lap while attempting to bang his head against the adult and the table. |
| Question about the video: What should be done to stop self-harm? | Question about the video: What should be done to stop self-harm? | |
| Correct answer: I would hold the child, protecting the head, and try to divert his attention. | Correct answer: I would hold the child, protecting the head, and try to divert his attention. | |
| Early signs of autism spectrum disorder | Video content: A child, approximately 1 ½ years old, is playing with blocks while her mother insistently calls to her. The child shows no reaction or attempt to communicate. | Video content: A child, approximately 1 year old, is playing with a toy that emits sound and light. Her mother insistently calls to her, but the child shows no reaction or attempt to communicate. |
| Question about the video: Is this an early sign of autism in this child? | Question about the video: Is this an early sign of autism in this child? | |
| Correct answer: The child does not share attention while playing. | Correct answer: The child does not share attention while playing. | |
| Stereotypy (trigger) | Video content: A child, seated at a table, exhibits stereotypic movements involving his hands and mouth. | Video content: A child, lying on the floor, seemingly idle, exhibits stereotypic behaviors involving his hands and mouth. |
| Question about the video: What possible factors are associated with this behavior? | Question about the video: What possible factors are associated with this behavior? | |
| Correct answer: Being idle, lack of targeted activity. | Correct answer: Being idle, lack of targeted activity. | |
| Auditory sensory hyperactivity | Video content: A child under evaluation covers his ears when the evaluator speaks louder. | Video content: A child in the subway with his parents covers his ears to dampen the sound from an approaching train. |
| Question about the video: You may notice that this child has auditory hypersensitivity. What can therapists do to lessen this sensitivity? | Question about the video: You may notice that this child has auditory hypersensitivity. What can therapists do to lessen this sensitivity? | |
| Correct answer: Apply techniques so that the child gradually becomes accustomed to noise, begins to anticipate it, and is less bothered by it. | Correct answer: Apply techniques so that the child gradually becomes accustomed to noise, begins to anticipate it, and is less bothered by it | |
| Echolalia | Video content: A child playing Pop-up Pirate with a therapist in his office exhibits echolalic behavior, repeating the therapist’s phrases several times. | Video content: A child is in a car with her mother and begins to verbalize random words. The mother initiates a dialogue based on the child’s speech. |
| Question about the video: If you were the therapist, what would you do about the child’s echolalia? | Question about the video: What correct attitude did the mother display regarding her child’s echolalia? | |
| Correct answer: I would provide context for the child’s speech, initiating a dialogue. | Correct answer: She provided context for the child’s speech, initiating a dialogue. | |
| Tantrum 1 | Video content: A child stands next to the door of a therapist’s office, crying and throwing a tantrum, desperate to leave the room. | Video content: A child is in a car with his mother and sister and begins a tantrum that involves screaming, crying, and kicking the front seat because he wants something that is not there. The mother tries to calm him down by talking to him. |
| Question about the video: The boy threw a tantrum because he did not want to stay in the therapy room. He cried for more than 20 minutes. He stopped crying for a few moments during the episode. What could the therapist have done at this point? | Question about the video: In what way did the mother incorrectly handle the tantrum? | |
| Correct answer: The therapist could have presented an interesting toy or proposed a different activity. | Correct answer: Talking to him while he’s screaming. | |
| Functional play | Video content: A boy, approximately 7 years old, is playing on the floor with toy cars, analyzing them and spinning their wheels. | Video content: A boy, approximately 7 years old, is playing with a pogo stick. His play consists of leaning on the handles and moving his body stereotypically. |
| Question about the video: How would you rate this child’s play? | Question about the video: How would you rate this child’s play? | |
| Correct answer: He does not use the toy in a functional way and seems interested only in parts of the toy. | Correct answer: He does not use the toy functionally, but in a stereotyped way. | |
| Early signs 2 | Video content: An infant, less than 1 year old, is lying on the floor, exhibiting stereotypic movements with his hands and feet. The mother calls to him several times, but he does not look at her. | Video content: An infant, around 1 year old, is sitting in a feeding chair, playing with a spoon and a small bottle. The mother calls to him several times and touches him, but he does not look at her. |
| Question about the video: Which two signs of autism does this child exhibit? | Question about the video: Which two signs of autism does this child exhibit? | |
| Correct answer: Lack of shared attention and stereotypy. | Correct answer: Lack of shared attention and unresponsiveness when called. | |
| Tantrum – self-harm | Video content: A boy, around 14 years old, apparently did something wrong and tries to apologize to his father. The father continually reaffirms that the boy has done something wrong. The boy begins to beat himself and the father is passive about this behavior. | Video content: A boy, around 6 years old, is at the mall with his parents and does not want to leave. He exhibits tantrum behavior, crying, throwing himself on the floor and hitting anyone who tries to speak with him. |
| Question about the video: The child verbalizes repentance for his inappropriate behavior, but as his father speaks, this transforms into increasing self-aggression. Which of the attitudes displayed by the father would you consider appropriate for managing the self-harm behaviors of children in similar situations? | Question about the video: This child does not want to leave the mall, so he throws a tantrum. What could have been done? | |
| Correct answer: Neither alternative is correct. | Correct answer: Visualization strategies could have been used; combining activities at the mall and on the way home could have made the entire experience pleasurable for the child. | |
| Aggressiveness | Video content: A child in a car on the way to school begins crying and shouting that he does not want to go. At one point, he attacks an adult. | Video content: A child in a therapy session throws a tantrum, crying and attempting to strike and bite the therapist. |
| Question about the video: This child does not want to go to school; he would rather go to the park and, thus, throws a tantrum. What could have been done? | Question about the video: What management is appropriate for this situation? | |
| Correct answer: On the way to school, parents should provide pleasurable activities for their children, distracting them until they arrive at school. | Correct answer: Hold the child without speaking to him. When he calms down, redirect him to another activity. | |
| Stereotypy (avoidance) | Video content: A child under evaluation exhibits stereotypic behavior, placing his hand in his mouth. The evaluator asks the child to remove his hand from his mouth. | Video content: A child in a therapist’s office exhibits stereotypic behavior by placing his hand in his mouth. The therapist says: “putting your hand in your mouth is nasty.” |
| Question about the video: What improper attitude does the evaluator display? | Question about the video: What improper attitude does the evaluator display? | |
| Correct answer: Asking him to remove his hand from his mouth. | Correct answer: Reinforcing stereotyped behavior. |
In addition to the videos regarding evaluation, another 13 videos were selected, edited, and used during the training sessions to illustrate ASD characteristics and management models for different situations (Table 2). It is worth mentioning that, in addition to these videos, PowerPoint presentations and topical expository lectures were given. Table 2 contains descriptions of the videos used for each theme.
Table 2. Description of videos used during the training program.
| Theme | Description |
|---|---|
| Theory of mind | A clip from the TV series The Big Bang Theory, in which the protagonist, Sheldon (who has ASD), cannot comprehend the sarcasm and irritation displayed by the other characters. |
| Visual contact | A video from the researcher’s private practice in which she calls a child by name several times and the child does not respond. |
| Stereotypy | A video from the researcher’s private practice in which she and the child are at a table engaged in an activity. When the researcher gives a command, the child engages in vocal stereotypy (screaming) and slapping herself on the head. |
| Hearing sensitivity | A video from the researcher’s private practice in which she and a child are at a table engaged in an activity. When the researcher “celebrates” an achievement, slightly raising her voice, the child covers his ears with both hands. |
| Early signs | A video showing a toddler, about 1 year old, playing with blocks while her mother tries to interact with her. At no point does the child look at her mother (lack of shared attention) or answer when called. |
| Early signs | A video showing an infant, less than 1 year old, lying on the floor engaged in stereotypy involving his hands and feet. |
| Early signs | A video showing a toddler, about 1½ years old, spinning around and looking out of the corners of her eyes due to the sound a dishwasher is making. |
| Functional play – symbolic | A video from the researcher’s private practice in which she and a child are sitting at a table playing make believe with toy fruit. |
| Functional play – games | A video from the researcher’s private practice in which she and a child are sitting at a table playing a memory game with adapted rules. |
| Functional play – games | A video from the researcher’s private practice where she and a child are sitting at a table playing the game Don’t Break the Ice. |
| Stereotypy management | A video from the researcher’s private practice in which she and a child are seated at a table. The child begins to engage in stereotypy with his hands and the researcher blocks the behavior by redirecting him to another activity. |
| Echolalia | A video from the researcher’s private practice where she and a child are seated at a table, engaged in a quiz. The child exhibits echolalia by repeating the therapist’s question. |
| Tantrum – management | A video of a child screaming and crying at the mall; her parents don’t know what to do. (Note: This video does not provide a correct model, but was used as a discussion starter.) |
In summary, the training was provided over four 4-hour meetings with two groups: one in the morning and one in the afternoon. The following topics selected by the CAPSi staff were discussed: i) diagnosis, identification criteria, and early intervention; ii) stereotypy (including echolalia, ritualization, hypo/hypersensitivity, and sexuality); iii) inappropriate behavior and hetero-/auto-aggression; iv) how to prepare the therapeutic environment for care and how to conduct group sessions; and v) functional play and guidance for parents.
A training model consisting of eight sections of 2 hours each had originally been planned with the participants and CAPSi coordinators. However, during the process, the CAPSi coordinators requested that the number of meetings be reduced to four 4-hour sessions, in an effort to increase productivity by concentrating the sessions. Thus, the final model consisted of 4-hour sessions in two groups, to avoid loss of content or reduction in total number of training hours.
Regarding the impact of the training program, there was a general post-training improvement in knowledge and management of ASD patients, with a mean increase in overall correct responses to the video questions (from 5.9±1.8 to 9.6±1.8; p < 0.001, Student’s t-test for paired samples).
The percentage of correct answers increased for 11 of the 13 training videos (Table 3). However, the changes were not statistically significant in four videos: two on tantrums (video 8, p = 0.003; video 11, p = 0.002), the video on aggression (video 12, p = 0.021), and one of two videos about stereotypy (video 13, p = 0.004). A mean improvement in scores for the second video on stereotypy (video 5) was observed, but the level of significance was marginal (p = 0.07). It should also be noted that the number of correct post-training responses was 100% for two other videos: number 2 on hyposensitivity and number 9 on functional play (which prevented completion of the McNemar test). Surprisingly, the number of correct answers regarding self-harm actually declined (video 3, p = 0.031), and no significant change was observed for the video on hearing sensitivity (video 6, p = 0.180).
Table 3. Distribution of correct answers before and after the training videos (n=14).
| Video | Before | After | p-value* |
|---|---|---|---|
| 1 – Stereotypy (management) | 6 (42.9) | 10 (71.4) | 0.289 |
| 2 – Hypo/hypersensitivity | 12 (85.7) | 14 (100.0) | † |
| 3 – Self-harm | 13 (92.9) | 7 (50.0) | 0.031 |
| 4 – Early signs 1 | 4 (28.6) | 8 (57.1) | 0.388 |
| 5 – Stereotypy (triggering) | 2 (14.3) | 8 (57.1) | 0.070 |
| 6 – Hearing sensitivity | 12 (85.7) | 7 (50.0) | 0.180 |
| 7 – Echolalia | 7 (50.0) | 11 (78.6) | 0.219 |
| 8 – Tantrum 1 | 2 (14.3) | 13 (92.9) | 0.003 |
| 9 – Functional play | 10 (71.4) | 14 (100.0) | † |
| 10 – Early signs 2 | 8 (57.1) | 9 (64.3) | 1.000 |
| 11 – Tantrum 2 | 3 (21.4) | 13 (92.9) | 0.002 |
| 12 – Aggressiveness | 1 (7.1) | 9 (64.3) | 0.021 |
| 13 – Stereotypy (avoidance) | 2 (14.3) | 11 (78.6) | 0.004 |
Data presented as n (%).
Descriptive level of the McNemar test.
Test statistic could not be obtained due to lack of post-training errors.
As in the video responses, the KAP survey data showed that, in general, the training program led to an improvement in the knowledge, attitudes, and practices of the participating professionals, with total KAP scores increasing more than 20 points from baseline (p = 0.02) (Table 4). Regarding the three KAP subscales, improvements were observed in knowledge (p = 0.003) and attitudes (p = 0.050), but no difference was confirmed in the practice category (p = 0.186) (Table 4).
Table 4. Comparison of mean Knowledge, Attitudes, and Practices (KAP) questionnaire scores before and after training (n=14).
| Mean | SD | Minimum | Maximum | Median | n | p-value* | |
|---|---|---|---|---|---|---|---|
| KAP – Total | 0.020 | ||||||
| After | 168.0 | 22.4 | 126.0 | 203.0 | 167.0 | 11 | |
| Before | 144.9 | 32.3 | 69.0 | 180.0 | 144.0 | 11 | |
| Difference | 23.1 | 27.7 | -23.0 | 59.0 | 20.0 | 11 | |
| KAP – Knowledge | 0.003 | ||||||
| After | 65.5 | 8.3 | 50.0 | 78.0 | 68.0 | 11 | |
| Before | 53.5 | 12.0 | 29.0 | 73.0 | 53.0 | 11 | |
| Difference | 12.1 | 10.1 | -4.0 | 26.0 | 15.0 | 11 | |
| KAP – Attitude | 0.050 | ||||||
| After | 47.8 | 9.1 | 36.0 | 66.0 | 44.0 | 13 | |
| Before | 42.1 | 8.6 | 19.0 | 50.0 | 44.0 | 13 | |
| Difference | 5.7 | 9.4 | -12.0 | 21.0 | 4.0 | 13 | |
| KAP – Practice | 0.186 | ||||||
| After | 56.7 | 8.7 | 40.0 | 72.0 | 55.0 | 13 | |
| Before | 51.9 | 13.7 | 21.0 | 67.0 | 59.0 | 13 | |
| Difference | 4.8 | 12.2 | -13.0 | 23.0 | 7.0 | 13 |
SD = standard deviation.
Descriptive level of Student’s t-test for paired samples.
The participants had a generally positive reaction to the training, as 100% of the assessments related to: i) course organization; ii) teaching; iii) topics; and iv) duration of training received a score of excellent (52%) or good (48%). In addition, all participants reported that they would not change any of the chosen topics and would recommend the training to their colleagues.
Discussion
The primary objective of this study – developing and implementing a feasible continuing education program for CAPSi professionals to manage children and adolescents with ASD – was met. Comprehensive sets of videos to assist the teaching-learning process during the training program (13 videos) and to evaluate participants after the program (26 videos) were developed and were well evaluated by the participants.
One challenge to implementing this study, which was simultaneously one of its strengths, was to offer training to a multidisciplinary team working in the CAPSi environment, given the relevance of applying the findings of the study to clinical practice.23 To improve feasibility, the implementation process underwent several adaptations to fit the CAPSi routine.
Furthermore, the interaction between participants from different professional categories should be considered an important factor in optimizing the sharing of different types of knowledge and experiences, thereby diversifying clinical views of the same condition and enhancing the skills of each professional, while allowing a productive, respectful working relationship to develop across the multidisciplinary team.25
In summary, operational issues in the SUS network led to several challenges in implementing the training program within the CAPSi routine. Such constraints should be considered when developing proposals of this nature, particularly placing excessive demands on the unit’s daily schedule. The number of sessions had to be reduced, and staff did not participate in distance supervision, which is a component that should be rethought. Distance learning, including case supervision, has been reported as an important learning resource for adults, optimizing the teaching and learning process, as well as providing more equal opportunity.26 Initially, the participants showed interest in this feature as a channel of communication with the research team but, in practice, it was ineffective due to lack of demand. Our main hypothesis for this outcome is that web technology is an uncommon medium for professional communication among CAPSi staff, perhaps due to a lack of time or internet access at work. Difficulties in e-mail communication have been observed in other studies on mental health training for primary care professionals in the city of São Paulo.27 Thus, a different strategy for added trainee support should be sought in future initiatives, such as face-to-face supervision at CAPSi units themselves.
In addition, the final session, which ideally should have been more individually tailored for each CAPSi unit, must also be restructured. The experience of this study suggests that, in future training programs, this final meeting should take place in the work environment at a pre-established time, scheduled like other training meetings, which would make it a priority. However, if any of the staff participating in the training program are on duty during a scheduled meeting and problems arise within the unit, they will have to respond.
Overall, the results of this study showed the positive impact of training, both through the video evaluations and the KAP questionnaire, but particularly in the former. Staff exhibited increased knowledge of virtually all 13 subjects, which aligns with findings reported in the national and international literature.12,13,27 Although we did not carry out a direct assessment of changes in the care staff provided to patients before and after training, studies indicate that professional training programs aimed at increasing knowledge levels about children with ASD also lead to improvements in care.23
We have several hypotheses regarding the decrease in mean post-training scores for the self-harm and hearing sensitivity videos. First, the subject of self-harm led to some of the most difficult and complex debate experienced during the program. This debate stemmed from the fact that, although the participants claimed to be familiar with management strategies for dealing with children who self-harm, these were not applied on a day-to-day basis in the CAPSi unit. A second hypothesis is that the selected video, which included self-harm behavior and its management, may have been unclear or did not sufficiently illustrate the necessary concepts to the staff. Regardless of the ultimate reasons for the negative result, this specific video should be replaced in any future ASD training program, and the content/form of the specific training on this issue should be restructured. However, the subject should not be eliminated altogether, since it represents one of the major challenges in caring for more serious cases of ASD.
Regarding the video on hearing sensitivity, our main hypothesis for the decreased mean scores (although statistically insignificant) was inappropriate video selection, as this was the only topic in which the setting of the pre-training video differed from that of the post-training video: the former depicted a situation in a consulting room, while the latter took place outdoors. Thus, in future ASD training programs, videos about hearing sensitivity should feature the same setting, either a consulting room or a noninstitutionalized setting, although a therapeutic environment (where some intervention is being undertaken) would seem preferable, as it would allow participants to visualize management more clearly and in greater detail. The results of the KAP questionnaire showed that the training mainly improved staff knowledge and attitudes toward the demands of their work. This corroborates the literature on the need to train mental health professionals to care for children more effectively. High-quality information is essential for professionals to increase their understanding of their field of work and apply scientific knowledge to everyday clinical practice.11,13
We presume that the specific lack of improvement in the KAP ‘practice’ subscale was due to the brief duration of the training program, as well as to the fact that the evaluation was carried out immediately at the end of training, meaning there was no time for more substantial changes in clinical practice to take root. Thus, another recommendation of this study is that future training models should include a follow-up measure that can identify whether changes in knowledge and attitudes resulting from continuing education programs also lead to effective changes in clinical practice in the care of children with ASD.
Regarding the staff satisfaction assessment, one likely reason for the positive results was that the program followed a continuing education model, with professionals actively participating in the planning and organization of program meetings. Allowing the staff to be involved in the entire process will ensure the success of future training programs, since the issues discussed will be of interest and importance to them. Another possible explanation for the positive training feedback is that continuing education for mental health professionals in the public health system is a recent phenomenon, particularly in the setting of CAPSi units. There is still a gap between academia and the service network, and programs of this nature are scarce.28
In conclusion, the training program for CAPSi professionals who care for children with ASD described herein proved feasible and yielded positive results, considering the challenges faced by programs designed to meet the specific needs of this type of SUS institution. Nevertheless, the present study had certain limitations, including the small convenience sample and the non-random selection of participants. Thus, we assume that the results obtained may be affected by individual motivation, willingness to change, and existing attitudes toward training. In addition, the post-training evaluation immediately followed the conclusion of the program, there was no attempt to control for other educational interventions, and there was no untrained control group without training, so the results may not be exclusively related to the training program. Finally, there was no direct assessment of changes in the care provided by the participants.
Based on the findings of this study, we suggest that future training programs should: i) implement similar training models with larger samples of mental health care professionals, including a control group; ii) improve the training and evaluation materials that lacked evidence of effectiveness in the present study; and iii) extend the follow-up period to at least 4 months after the end of training.
Disclosure
The authors report no conflicts of interest.
References
- 1.American Psychiatric Association . Diagnostic and Statistical Manual of Mental Disorders. Fifth Edition (DSM-5). Arlington: American Psychiatric Publishing; 2013. [Google Scholar]
- 2.Lampreia C. Perspectivas da pesquisa prospectiva com bebês irmãos de autistas. Psicol Cienc Prof. 2009;29:160–71. [Google Scholar]
- 3.Bejerot S. An autistic dimension: A proposed subtype of obsessive-compulsive disorder. Autism. 2007;11:101–10. doi: 10.1177/1362361307075699. [DOI] [PubMed] [Google Scholar]
- 4.Baxter AJ, Brugha TS, Erskine HE, Scheurer RW, Vos T, Scott JG. The epidemiology and global burden of autism spectrum disorders. Psychol Med. 2015;45:601–13. doi: 10.1017/S003329171400172X. [DOI] [PubMed] [Google Scholar]
- 5.Elsabbagh M, Divan G, Koh YJ, Kim YS, Kauchali S, Marcín C, et al. Global prevalence of autism and other pervasive developmental disorders. Autism Res. 2012;5:160–79. doi: 10.1002/aur.239. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Brasil, Ministério da Saúde, Secretaria de Atenção à Saúde, Departamento de Ações Programáticas Estratégicas, Coordenação Geral de Saúde Mental . Saúde mental no SUS: os centros de atenção psicossocial [Internet]. Brasília: Ministério da Saúde; 2004. [cited 2016 Dez 12]. http://www.ccs.saude.gov.br/saude_mental/pdf/SM_Sus.pdf. [Google Scholar]
- 7.Brasil, Ministério da Saúde, Secretaria de Atenção à Saúde, Departamento de Ações Programáticas Estratégicas, Coordenação Geral de Saúde Mental, Álcool e Outras Drogas Saúde mental em dados 12 [Internet]. 2015 [cited 2017 Jul 28]. www.mhinnovation.net/sites/default/files/downloads/innovation/reports/Report_12-edicao-do-Saude-Mental-em-Dados.pdf. [Google Scholar]
- 8.Garcia GYC, Santos DN, Machado DB. Centros de atenção psicossocial infantojuvenil no Brasil: distribuição geográfica e perfil dos usuários. Cad Saude Publica. 2015;31:2649–54. doi: 10.1590/0102-311X00053515. [DOI] [PubMed] [Google Scholar]
- 9.Paula CS, Lauridsen-Ribeiro E, Wissow L, Bordin IA, Evans-Lacko S. How to improve the mental health care of children and adolescents in Brazil: actions needed in the public sector. Rev Bras Psiquiatr. 2012;34:334–51. doi: 10.1016/j.rbp.2012.04.001. [DOI] [PubMed] [Google Scholar]
- 10.Brookman-Frazee LI, Drahota A, Stadnick N. Training community mental health therapists to deliver a package of evidence-based practice strategies for school-age children with autism spectrum disorders: a pilot study. J Autism Dev Disord. 2012;42:1651–61. doi: 10.1007/s10803-011-1406-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Clark A, Browne S, Boardman L, Hewitt L, Light S. Implementing UK autism policy & national institute for health and care excellence guidance- assessing the impact of Autism training for frontline staff in community learning disabilities teams. Br J Learn Disabil. 2016;44:103–10. [Google Scholar]
- 12.Lampert SS. Efetividade de um programa de capacitação em identificação precoce do transtorno do espectro autista na atenção básica em saúde [dissertation]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2015. [Google Scholar]
- 13.Bordini D, Lowenthal R, Gadelha A, Araujo GM, Filho, Mari Jde J, Paula CS. Impact of training in autism for primary care providers: a pilot study. Rev Bras Psiquiatr. 2015;37:63–6. doi: 10.1590/1516-4446-2014-1367. [DOI] [PubMed] [Google Scholar]
- 14.Brasil, Ministério da Saúde, Secretaria de Gestão do Trabalho e da Educação na Saúde, Departamento de Gestão da Educação em Saúde . Política Nacional de Educação Permanente em Saúde [Internet], Série B. Textos Básicos de Saúde Série Pactos pela Saúde 2006, v. 9. 2009. [cited 2017 Jul 28]. bvsms.saude.gov.br/bvs/publicacoes/politica_nacional_educacao_permanente_saude.pdf. [Google Scholar]
- 15.Ceccim RB. Educação permanente em saúde: descentralização e disseminação de capacidade pedagógica na saúde. Cien Saude Coletiva. 2005;10:975–86. [Google Scholar]
- 16.Bteshe M, Estellita-Lins CEF. Os diferentes usos do vídeo no cuidado à saúde materno-infantil. Rev Electron Comun Inf Inov Saude. 2011;5:53–64. [Google Scholar]
- 17.Gomes LF. Videos didáticos: uma proposta de critérios para análise. Rev Bras Estud Pedagog. 2001;89:477–92. [Google Scholar]
- 18.Corbett BA, Abdullah M. Video modeling: why does it work for children with autism? J Early Intensive Behav Interv. 2005;2:2–8. [Google Scholar]
- 19.Barboza AA. Efeitos de videomodelação instrucional sobre o desempenho de cuidadores na aplicação de programas de ensino a crianças diagnosticadas com autismo, [thesis]. Belém: Universidade Federal do Pará; 2015. [Google Scholar]
- 20.Moura CB, Silvares EFM. O uso de vídeo em intervenções clínicas com pais: revisão da literatura e hipóteses comportamentais sobre seus efeitos. Psicol Teor Prat. 2008;10:144–61. [Google Scholar]
- 21.Launiala A. How much can a KAP survey tell us about people’s knowledge, attitudes and practices? Some observations from medical anthropology research on malaria in pregnancy in Malawi. Anthropol Matters. 2009;11:1–13. [Google Scholar]
- 22.Zaqueu LC, Teixeira MCTV, Alckmin-Carvalho F, Paula CS. Associações entre sinais precoces de autismo, atenção compartilhada e atrasos no desenvolvimento infantil. Psic Teor e Pesq. 2015;31:293–302. [Google Scholar]
- 23.Reeves S, Zwarenstein M, Goldman J, Barr H, Freeth D, Hammick M, et al. Interprofessional education: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2008;1:CD002213. doi: 10.1002/14651858.CD002213.pub2. [DOI] [PubMed] [Google Scholar]
- 24.Winter JCF de. Using the Student’s t-test with extremely small sample sizes. Pract Assess Res Eval. 2013;18:1–12. [Google Scholar]
- 25.Rhoades RA, Scarpa A, Salley B. The importance of physician knowledge of autism spectrum disorder: results of a parent survey. BMC Pediatr. 2007;7:37. doi: 10.1186/1471-2431-7-37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Vianna JA. Educação à distância, inclusão e mobilidade social. Inter Sci Place. 2009;1:5–12. [Google Scholar]
- 27.Lowenthal R. Capacitação em saúde mental na infância e na adolescência para profissionais da atenção primária, [dissertation]. São Paulo: Universidade Presbiteriana Mackenzie; 2014. [Google Scholar]
- 28.Lyon AR, Stirman SW, Kerns SEU, Bruns EJ. Developing the mental health workforce: review and application of training approaches from multiple disciplines. Adm Policy Ment Health. 2011;38:23853. doi: 10.1007/s10488-010-0331-y. [DOI] [PMC free article] [PubMed] [Google Scholar]