Impact of educational and psychological interventions compared with standard care in college/university students with autism spectrum disorder: a systematic review protocol

Objective:

This review will explore the impact of educational and psychological interventions on educational, social, behavioral, and mental health outcomes in students with autism spectrum disorder in tertiary education.

Introduction:

This systematic review will inform a new guideline on support for students with autism spectrum disorder in the tertiary education environment. These students face multiple educational, behavioral, social, and health-related problems that require effective interventions.

Inclusion criteria:

Participants are students with autism spectrum disorder in a tertiary education study program. Educational and psychological interventions will be included, such as accommodations, meta-cognitive and self-regulation training, psychological counseling, social skills training, and peer-mentoring/academic coaching. The comparator will be standard care. The study outcomes will include academic drop-out rates and evaluations, learning skills, social skills and social engagement, behavior, mental health (including anxiety, stress, and depression), and employment after graduation. This review will consider quantitative studies only.

Methods:

A 3-step search strategy will be used to find both published and unpublished studies in MEDLINE, CINAHL, APA PsycINFO, SocINDEX, Web of Science, Clinical Trials, ProQuest Dissertations and Theses, Open Dissertations, ERIC, WHO ICRTP, and Google Scholar. There will be no date or language limitations. All stages of article screening, critical appraisal, and data extraction will be conducted by 2 independent reviewers with the resolution of any disagreements done via consensus or through a senior reviewer. If possible, the included studies will be pooled using meta-analysis. The degree of certainty of the evidence will be assessed according to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

Systematic review registration number:

PROSPERO CRD42022323554

Introduction

Autism spectrum disorder (ASD) is a neurodevelopmental disorder that affects behavior, communication, and social skills.1 The International Classification of Diseases 11th Revision2 defines ASD as persistent deficits in the ability to initiate and sustain reciprocal social interaction and social communication, and by a range of restricted, repetitive, and inflexible patterns of behavior and interests. Global ASD prevalence is approximately 1/100 children and the prevalence estimate has increased over time.3 Studies in Asia, Europe, and North America have identified an average prevalence between 1% and 2% of individuals with ASD.4 Approximately 33% of autism cases co-occur with intellectual disability.3

The former classifications of ASD5 (eg, autism, Asperger’s syndrome, childhood disintegrative disorder) are now classified1 into 3 levels of severity according to the support needed (requiring support, requiring substantial support, requiring very substantial support). There are several subtypes of ASD depending on the level of intellectual and language development (a disorder of intellectual development present or not present; impairment of functional language not present/mild/present).2

With ASD prevalence on the rise, the number of young adults with ASD entering tertiary education is also increasing,6,7 bringing with it a relatively low study success rate and health-related risks.8,9 The comparatively high university drop-out rate of these students may be caused, among other reasons, by a lack of support structures within universities.9 Universities have started to provide support similar to the support provided for students with other special educational needs7; however, many students need a unique type of support corresponding to the specifics of their diagnosis.10 Ideally, this support would aim to equip these students with the ability to protect their mental health, solve unique challenges and situations in the university setting,7 and improve academic outcomes. For example, in the US, the support offered by universities to students with ASD is provided via disability resource centers,11 which provide:

• organizational and service support, such as amendments or reduction of courses or added time during exams (an adequate provision of amendments is, by law, such that maintains the quality of education)

• personnel adjustments (eg, a note-keeper, sign language translator)

• material equipment (eg, computers with special hardware and software, recorders).

Similarly, in the Czech Republic and other European countries, most public universities have special departments providing support in the form of interventions and services that typically include:

• accommodations (eg, individual terms and extended time for tasks, technological support, assistive services, accommodation of the physical space and exam conditions) and training of meta-cognitive and self-regulation strategies to improve planning, monitoring, and evaluation of their own learning

• psychological counseling, including specific psychotherapy techniques and programs.

Additional services, social skills training, peer-mentoring, academic coaching, and other interventions are also applied depending on the individual needs of students. Social skills training programs with different theoretical backgrounds include a combination of different techniques and methods (eg, video-modeling or structured model planning). This includes peer-mentoring and academic coaching, which involves pairing the student with ASD with a more knowledgeable and experienced person, ideally at the same college/university.

Apart from the low academic outcomes caused by using ineffective learning strategies and problems with executive functions,12 the majority of challenges that university students with ASD encounter are in the social domain. They frequently manifest as a lack of social skills, problems in communication and social interaction, and low social engagement. These problems bring multiple challenges to social relationships and cause stress, anxiety, and other health-related problems.

Educational and psychological interventions may have a positive impact on various educational, social, behavioral, and mental health outcomes. These educational outcomes include drop-out rates, evaluations of students’ academic progress, and finding a job after graduation. Normally these outcomes are evaluated using scales, although only rare examples of formalized or quasi-formalized outcome measures exist in this area, such as study skills assessments (according to the authors’ knowledge, none of the existing tests are validated for university students with ASD). Social skills training programs are often targeted at problems with eye contact, facial expression, turn-taking in conversation, and initiating conversation, among others.12,13 Most studies use multiple baseline designs and observations of targeted behavior. Less often, standardized questionnaires are included, such as the Multidimensional Scale of Perceived Social Support,13 Life Effectiveness Questionnaire,14 or self-constructed scales.15 Behavioral and mental health outcomes of students with ASD include the incidence of risk behavior, and the level of anxiety, stress, and depression. Although the presence of a specific behavior is most often measured by scales, there are outcome measures with good psychometric qualities and high social relevance in the area of mental health outcomes (eg, the Beck Anxiety Inventory and the Beck Depression Inventory16; 21-Item Depression, Anxiety, Stress Scales; Hamilton Depression Scale17; Hamilton Anxiety Scale17).

The information on the effects of interventions related to educational, social, behavioral, and mental health outcomes in university students with ASD has not been previously evaluated in a systematic review. The lack of a comprehensive systematic review makes it challenging for university support centers and stakeholders to make evidence-based decisions about the appropriate support for this population. A preliminary search of Epistemonikos, the Cochrane Database of Systematic Reviews, JBI Evidence Synthesis, Web of Science, and PROSPERO found only 1 relevant literature review (on a broader topic),6 but no systematic review or systematic review protocol addressing this topic was published or registered. As such, a systematic review is necessary to evaluate the evidence base addressing this topic. This review will inform a new guideline, which will follow the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach,18,19 concerning the support of students with ASD in the tertiary education environment of the Czech Republic.

The objective of this systematic review is to explore the impact of educational and psychological interventions on educational, social, behavioral, and mental health outcomes in students with ASD in the tertiary education environment. The review question, interventions, and outcomes of this protocol were selected and confirmed in collaboration with the guideline development group that the review will serve to create specific recommendations for.

Review question

What are the effects of educational and psychological interventions provided in the tertiary education environment (including accommodations, meta-cognitive and self-regulation training, psychological counseling, social skills training, and peer-mentoring/academic coaching) compared with standard care on academic drop-out rate and evaluations, learning skills (in the area of executive functions, meta-cognition, and self-regulation), social skills and social engagement, behavior, mental health (including anxiety, stress, and depression) and finding a job after graduation in students with ASD?

Inclusion criteria

Participants

This systematic review will include university/college students with ASD with or without psychiatric comorbidity, with no exclusions related to sex, race, ethnicity, or culture. The age of the students will not be limited, but it is expected that the majority of participants will be above 18 years, which is generally considered the lowest age one can apply to college/university in many countries (although exceptions exist, eg, for gifted and talented students). The students must be studying for a bachelor’s, master’s, or doctoral degree in a college or university. All possible branches, disciplines, and faculties, including full-time, distance, and other forms of study, will be included. Short university courses without qualification at the bachelor, master, or doctoral level will be excluded. The demographic area and nationality of students will not be limited.

Interventions

We will include educational and psychological interventions or whole intervention programs focused on students with ASD where the intervention is offered in the college or university environment. The selected interventions include the set of accommodations (eg, individual terms and extended time for tasks), training of meta-cognitive and self-regulation strategies according to Education Endowment Foundation (EEF), psychological counseling (including short psychotherapy intervention, programs focused on the problem, and task-solving), social skills training programs, and peer-mentoring/academic coaching. These interventions are usually provided by university counseling centers or special needs student support centers. However, because of differences in the ways and levels of support of students with ASD in different countries, it is not required that the strategies are integrated into the institutional support or offered by employees with special training.

Concerning dosage, frequency, and duration of interventions, no limits will be imposed; however, based on current counseling practice, it is expected that providing these interventions will last at least 1 semester, with a minimum of 6 sessions for 30 minutes. Interventions provided outside the college/university environment (eg, in health care institutions) or multimodal treatment programs that don’t make it possible to identify the effect of a particular intervention will be excluded.

Comparators

Standard care (which typically means no specific intervention provided by a college/university) will be used as a comparator in this systematic review; however, we will consider other comparators if the study meets the eligibility criteria of this review.

Outcomes

The effects of interventions will be evaluated using measurements of outcomes related to academic drop-out rate and evaluations, learning skills (in the areas of executive functions, meta-cognition, and self-regulation), social skills and social engagement, behavior (problem, challenging, and aggressive behavior), mental health (including anxiety, stress, and depression), and finding a job after graduation.

Dichotomous data are expected for drop-out rate, social engagement (usually measured by attendance at classes or other social events20), behavior (the presence or absence of concrete social skills or concrete behavior is determined in prospectively defined time intervals), and finding a job after graduation (working/not working, full-time/part-time, major-related job or not). Observation and evaluation of statistical data are expected to be the main methods of data collection.

For other outcomes that present themselves as continuous or ordinal data (eg, academic evaluation, which may be measured on a 5-point scale), we will focus on studies that are using validated measures to assess these outcomes. Self- and caregiver-report measures will be analyzed separately. For each outcome, the priority of outcome measures was determined a priori, based on their psychometric qualities and social relevance:

• Social skills:

• Social Skills Rating System

• Test of Young Adult Social Skills Knowledge

• Emotion Regulation and Social Skills Questionnaire

• Executive functions:

• Behavior Rating Inventory of Executive Function for Adults

• Delis-Kaplan Executive Function System

• Color-Word Interference subtest (with traditional Stroop test)

• Meta-cognition: the Metacognition Index

• Self-regulation: the Behavioural Regulation Index

• Psychopathology (depression and anxiety):

• Beck Anxiety Inventory and the Beck Depression Inventory16

• 21-Item Depression, Anxiety, and Stress Scales

• Hamilton Depression Scale (Hamilton 1960) and Hamilton Anxiety Scale17

• Stress:

• Perceived Stress Scale-10 and -14

• The Ardell Wellness Stress Test

• Stress Coping Resources Inventory: A Self-Assessment

• Behavior:

• Aberrant Behaviour Checklist

• The Problem Behaviour Scale of the Scales of Independent Behaviour—Revised

• The Challenging Behaviour Scale (a 25-item closed-ended questionnaire for service providers)

• The variant of the competitive reaction time task, aggressive behavior measurement.

We will consider outcome measures that are self-developed only when data from validated outcome measures are not available in the included studies. Outcomes related to costs and adverse events of interventions (eg, interventions that are not comfortable for students, which may result in early termination of the intervention) will also be included, if available.

Types of studies

This review will consider all quantitative primary studies. We will include both experimental and quasi-experimental study designs, including randomized controlled trials, non-randomized studies of interventions, before-and-after studies, and interrupted time-series studies. In addition, analytical observational studies (including prospective and retrospective cohort studies, case-control studies, and analytical cross-sectional studies) and descriptive observational studies (including case series, case reports, and descriptive cross-sectional studies) will be included if they inform about the effects of interventions. Based on our preliminary search, a lack of experimental studies is expected and results from observational studies may be useful if relevant to the outcomes of this study. We will exclude bachelor theses, text and opinion papers, and all types of reviews. Mixed method studies will be considered if relevant data from the quantitative sections can be extracted.

Methods

This systematic review will be conducted following JBI methodology for the systematic review of effectiveness21 and will be reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist.22

Search strategy

The search strategy aims to locate both published and unpublished studies. A 3-step search strategy will be utilized in this review. An initial limited search of MEDLINE was undertaken to identify articles on the topic. The text words contained in the titles and abstracts of relevant articles, and the index terms used to describe the articles were used to develop a full search strategy for MEDLINE (Ovid; Appendix I). The search strategy, including all identified keywords and index terms, will be adapted for each included database and/or information source. In the end, the reference list of all identified papers and reports, and articles retrieved for critical appraisal will be searched for additional studies.

The databases to be searched will include CINAHL (EBSCOhost), MEDLINE (OvidSP), APA PsycINFO (EBSCOhost), SocINDEX (EBSCOhost), and Web of Science Core Collection. The search for unpublished studies and gray literature will include Clinical Trials, ERIC (EBSCOhost), Google Scholar (only the first 100 results), Open Dissertations (EBSCOhost), ProQuest Dissertations and Theses (ProQuest), and WHO ICTRP (International Clinical Trials Registry Platform). There will be no date or language limitations. For studies where the full text is not in English, the complete manuscript will be translated.

Study selection

Following the search, all identified citations will be collated and uploaded into the JBI System for the Unified Management, Assessment and Review of Information (JBI SUMARI; JBI, Adelaide, Australia).23 After duplicates are removed, titles and abstracts will be screened by 2 independent reviewers for assessment against the inclusion criteria of the review. Potentially relevant studies will be retrieved in full for assessment against the inclusion criteria by 2 independent reviewers (see Appendix II for criteria that will be observed during both phases of study selection). Reasons for the exclusion of papers at full-text screening that do not meet the inclusion criteria will be recorded and reported in the systematic review. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion or with an additional reviewer. The search results and details of the screening process will be presented in a PRISMA flow diagram.22

Assessment of methodological quality

Eligible studies will be critically appraised by 2 independent reviewers at the study level for methodological quality in the review, using the standardized JBI critical appraisal instruments, including the checklist for randomized controlled trials, quasi-experimental studies, cohort studies, case series, and case reports.21 Any disagreements that arise between the reviewers will be resolved through discussion or with a third reviewer. The results of the critical appraisal will be reported in narrative format and a table. Risk of bias will be conducted and presented in a Summary of Findings. Studies will not be excluded based on their methodological quality or risk of bias; however, the results of the critical appraisal will be considered in the synthesis of the evidence by sensitivity analysis and reported in narrative and tabular format. Therefore, all studies, regardless of the results of their risk of bias, will undergo data extraction and synthesis, where possible.21

Data extraction

Data will be extracted from studies included in the review by 2 independent reviewers using a data extraction tool created by the authors (see Appendix III). JBI SUMARI will be used for data extraction and no pilot phase is planned. The extracted data will include specific details about the study, study design, country, population, providers of the intervention, study methods, type of intervention, harm effects, and outcomes of significance to the review objectives and interventions. Any disagreements that arise between the reviewers will be resolved through discussion or with a third reviewer. Authors of papers will be contacted to request missing or additional data, where required.

Data synthesis

Studies will, where possible, be pooled in a statistical meta-analysis using RevMan v5.4 (Copenhagen: The Nordic Cochrane Centre, Cochrane). Outcome measures in many studies are based on the observation of behavior and will provide dichotomous data. For dichotomous data (expected for drop-out rate, social engagement, social skills, behavior, and finding a job after graduation), we will calculate effect sizes as relative risk and present these with 95% CIs. When there are zero events in an intervention or a control arm, RevMan will add a fixed value of 0.5 to the empty cell. If there are no events in either treatment arm, the study will not contribute to the pooled relative estimate of effect from the meta-analysis; however, we will keep these results to inform baseline risk for absolute as opposed to relative comparisons, and use risk difference instead of relative risk. Where possible, adjusted estimates will be extracted for any synthesis from non-randomized studies.

Continuous data from standardized scales is expected in studies measuring executive functions and meta-cognition, self-regulation, anxiety, stress, and depression.13,14 In this case, effect sizes will be expressed as weighted (or standardized) final post-intervention mean differences and their 95% CIs will be calculated for analysis. If standard deviations are missing, we will attempt to calculate or impute these to facilitate meta-analysis according to guidance in the Cochrane handbook.24 A single analysis for each outcome (univariate meta-analysis) will be conducted. Heterogeneity will be assessed statistically using the standard χ² and I ² tests. Statistical analyses will be performed using a fixed effect model where there are a low number of studies, and we will use random effects models whenever there are more than 5 studies.25 Data from single group observational studies will be synthesized using proportional meta-analysis in JBI SUMARI.23,26

Subgroup analyses will be conducted to examine different types of interventions, modes of intervention delivery, dose, geographic region, and degree type. Sensitivity analyses will be conducted to assess the influence of methodological quality on results; the statistical model for meta-analysis; and to test the robustness of data, such as the demographic heterogeneity (most studies are expected to be from the USA and Europe). Where statistical pooling is not possible, the findings will be presented in synthesis (without meta-analyses) format, including tables and figures to aid in data presentation, where appropriate. A funnel plot will be generated using RevMan to assess publication bias if there are 10 or more studies included in a meta-analysis.27 Statistical tests for funnel plot asymmetry (Egger test,)28 will be performed, where appropriate. Missing data will be requested from the authors of the studies, where required. If authors cannot be contacted or do not respond to requests, then the study data will not be included in the analysis.

Assessing certainty in the findings

The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach for grading the quality of evidence for all selected outcomes will be followed.19 An interactive Summary of Findings will be created using GRADEPro GDT (McMaster University, ON, Canada) for each type of intervention. The Summary of Findings will present the following information where appropriate: absolute risks for the treatment and control; estimates of relative risk; and a ranking of the quality of the evidence based on the risk of bias, directness, heterogeneity, precision, and risk of publication bias of the review results. The outcomes reported in the Summary of Findings will be drop-out rate, academic evaluations, learning skills (in the area of executive functions), social skills, challenging behavior, anxiety, and depression.

Funding

This research was funded by the Ministry of Education Youth and Sports, grant number DSGC-2021-0145, and by Support of Mobility at Palacký University Olomouc II., grant number CZ.02.2.69/0.0/0.0/18_053/0016919. The funders will have no role in content development.

ZM is supported by an NHMRC Investigator grant, APP1195676.

Author contributions

JK: Conceptualization, methodology, writing (original draft preparation), review and editing. JL: Writing, reviewing, and editing. JD: Writing (original draft preparation), review and editing, funding acquisition, and project administration. ZS: Preliminary search, preparation of the search strategy, database searches, writing (review and editing). MK: Methodological guidance, writing (review and editing), supervision. HS-P: Conceptualization, methodology, writing (original draft preparation). DH and LK: Writing (review and editing). MS and JM: Conceptualization. DZ: Writing (original draft preparation). ZM: Writing (review and editing), supervision.

Appendix I: Search strategy

APA PsycINFO

Search conducted: November 1, 2022

1. DE “Autism Spectrum Disorders” OR autism* [Ti/Ab] OR ASD [Ti/Ab] OR autistic spectrum disorder* [Ti/Ab] OR Asperger syndrome* [Ti/Ab] OR Asperger? [Ti/Ab] OR pervasive developmental disorder* [Ti/Ab] OR PDD [Ti/Ab] OR PDD-NOS [Ti/Ab] OR high function*[Ti/Ab] OR HFASD [Ti/Ab] (71,750)

2. DE “Colleges” OR DE “Students” OR universit* [Ti/Ab] OR college* [Ti/Ab] OR higher education [Ti/Ab] OR tertiary [Ti/Ab] OR post-secondary [Ti/Ab] OR postsecondary [Ti/Ab] OR post secondary [Ti/Ab] TAFE [Ti/Ab] OR further education [Ti/Ab] OR undergraduate [Ti/Ab] OR student* [Ti/Ab] OR campus [Ti/Ab] (713,660)

3. DE “Social Skills” OR DE “Social Interaction” OR DE “Coaching” OR DE “Counseling” OR DE “Psychotherapy” OR counselling [Ti/Ab] OR counseling OR [Ti/Ab] mentoring [Ti/Ab] OR couching [Ti/Ab] OR social interaction* [Ti/Ab] OR social skill* [Ti/Ab] OR self-regulation [Ti/Ab] OR self regulation [Ti/Ab] OR psychotherap* [Ti/Ab] OR accomodation [Ti/Ab] (327,736)

4. #1 AND #2 AND #3 (1212)

Appendix II: Criteria for study selection at the level of title/abstract and full text screening

Study selection according to:	Title/abstract	Full text
Population (P)	• Are there students with ASD? • Was the study done in the college/university environment?	• Are there other populations or types of special needs in the sample? • Are there only students in the qualification type of study program?
Intervention (I)	• Do the interventions applied in the study fall into the scope of selected interventions?	• Description of the intervention components
Comparator (C)	---	• Comparators will be evaluated only at the level of full text
Outcomes (O)	• Do the interventions applied in the study fall into the scope of selected outcomes?	• What outcome measures were applied?

Appendix III: Draft data extraction instrument

Study	Country	Study design	Participant characteristics	Interventions	Outcome	Description of main results
Author, year, citation			Number Age Gender Diagnosis and comorbidities Type of tertiary education study program Control sample (if applicable)	Provider: university counseling centers/external staff employed by the university Type of intervention: Details description will be elaborated on in the appendix Course (or frequency) of the intervention	Measurement: continuous/dichotomous; meaning of the higher score Harm effects (if applicable)	Continuous data M±SD Group size Dichotomous data Events and participants