Assistive Technology Interventions for Adolescents and Adults with Learning Disabilities: An Evidence-Based Systematic Review and Meta-Analysis

2. Introduction

People with specific learning disabilities (LD) have learning difficulties that are specific to reading, writing or mathematics, and are not explained by another condition (APA, 2013). Students with LD are increasingly seeking postsecondary education (Joyce & Rossen, 2006). In fact, the largest group of incoming postsecondary students with disabilities are students with LD, not students with sensory or motor disabilities (Gregg, 2009). Creating inclusive learning environments for students at all educational levels is an issue at the forefront of discourse today. In countries like the United States, there are also legal requirements for inclusion at the postsecondary level, as mandated by the Americans with Disabilities Act of 1990 (42 U.S.C.§12101). Therefore it is important to investigate all aspects of inclusive learning environments, including how assistive technology (AT) can serve the needs of adolescent and adult learners - our focus in this paper.

AT options have proliferated in recent years, due to the greater availability of ever more powerful computers and other forms of technology (Alper & Raharinirina, 2006). Survey studies like Abreu-Ellis & Ellis (2006) have documented the use of several types of AT accommodations for higher education students with LD: text-to-speech and voice recognition software, outlining programs, and various word-processing-based accommodations. But do these accommodations improve learning for students with LD, and how do they impact students’ educational experience?

The meta-analysis is the primary tool of choice for collating results from disparate studies in a systematic, quantitative way (Borenstein et al., 2009). Currently, research on AT supports reveals mixed success of AT for students with LD (Holmes & Silvestri, 2012; Lewandowski et al., 2016). However, there is a lack of meta-analyses and systematic evaluations of this work. This situation impedes students and educational technologists from making evidence-based decisions about AT interventions.

Relatedly, although quantitative research can tell us about the effectiveness of AT, we need qualitative research to find out about the lived experience of participants (Van Manen, 2015), a crucial consideration for inclusive education. If AT helps learning purely in a numerical sense, but is uncomfortable or socially stigmatizing to use, then advocating for it might cause more harm than good. Reviews including qualitative research are becoming widespread, especially in the health sciences (Andrew et al., 2009). However, we could not locate any research synthesis incorporating relevant qualitative studies.

To address these limitations, we present a systematic review of both quantitative and qualitative work, with meta-analyses where appropriate, to help us determine which particular AT interventions are effective for adolescents and adults with LD – a question which cannot be answered on the basis of previous reviews.

Meta-analyses help address a major issue in both learning disabilities research and education research with atypically developing groups: the issue of small sample sizes. Meta-analyses provide a formal framework of collating data from separately conducted, preexistent studies. Their primary limitation is that often they compare studies that have many differences in design, with the assumption that those differences are not central (Borenstein et al., 2009). Therefore, in every meta-analysis, a case-by-case decision must be made about inclusion criteria related to study design, outcome variables and sample characteristics. Here we will explain our decisions by including data about each paper in detailed tables, and discussing commonalities in outcome variables and design in each category.

2.1. An overview of previous work

The current paper builds on previous work on pedagogical uses of digital technology for students with disabilities by providing a review of the effectiveness of AT for adolescent and adult students with LD. The earliest relevant reviews, like Brown et al. (1989), did not focus specifically on LD, but surveyed all disabilities. A more recent review, Alper & Raharinirina (2006), included more papers about students with LD than any other disability, but only 25% of those papers featured students with LD who were over 21 years of age.

The use of AT in writing interventions for students with LD has received attention in the research literature. Some authors considered specific interventions in their reviews. Peterson-Karlan (2011) examined writing interventions for students of all ages with LD. That review was one of the very few that explicitly followed evidence-based guidelines, but it also concluded that in this specific area, there were an insufficient amount of studies at that time to draw any systematic conclusion (even with this broader age range). Freeman et al. (2005) reached a similar conclusion about keyboarding interventions. Wanzek et al. (2006) reviewed spelling interventions for K-12 students. This was another of the few relevant evidence-based publications, and the only one where effect sizes were calculated; however, the authors did not provide forest plots or meta-analyses of these effect sizes, possibly due to the heterogeneity of studies. Together, these reviews yielded support for the use of AT for students who struggle with writing, but the researchers noted many limitations – like the large amount of non-standardized outcome measures used in the papers –, and a lack of generalizability to older students.

The research team of MacArthur also produced a series of reviews about writing interventions for students of all ages (MacArthur, 1996; Macarthur, 1999; MacArthur et al., 2001), predominantly focusing on their own studies. Other researchers like Raskind & Higgins (1998) also primarily reviewed their own work, and did not seek to provide a systematic survey of the field. Their findings were mixed, with AT interventions often beneficial, but in limited ways.

Some articles focused on specific LDs, only finding a handful of papers. Lindstrom (2007) discussed accommodations specifically for postsecondary students with reading and written expression disorders, finding that there was little empirical evidence to guide judgment. We also located an amount of articles enumerating the various extant AT devices and approaches, without systematically assessing their effectiveness – like Lewis (1998a), Mull & Sitlington (2003). These were usually aimed at educators, though some of them appeared in academic journals with an audience mostly composed of researchers. Some articles for educators, like Martinez-Marrero & Estrada-Hernandez (2008), discussed both reviews and individual studies.

We also located review papers with broader foci. Rath & Royer (2002) surveyed all accommodations for college students with LD, including assistive technology; with the conclusion that at that date, very few relevant studies existed. More recently, Gregg (2011) focused on all sorts of accommodations for adolescent and adult students with LD – including and emphasizing testing accommodations like extended time, which do not fall under the AT umbrella. There was insufficient information to draw conclusions; even though the authors had previously been able to conduct a meta-analysis on the effectiveness of extended time for adolescents and adults with LD (Gregg, 2009). Lang et al. (2014) was the closest to our review, but it summarized various AT interventions in only postsecondary students with LD, and with a non-systematic literature search. The authors argued that AT was a viable option. Another recent publication, Lewandowski et al. (2016) discussed AT options both for students with LD and those with attention deficit hyperactivity disorder (ADHD). Its authors likewise did not perform a systematic search and assessment, and did not focus on any particular age group. They also concluded AT was helpful, but did not attempt to draw firm conclusions about the type of AT that was useful, and the extent of its usefulness.

As the above demonstrate, systematic evaluations have been lacking to this date. Most reviews were limited in scope. Often they listed studies with no search process described, and no attempt to produce a meta-analytic summation. Holmes & Silvestri (2012) criticized this lack and pointed out the mixed success of previous work. Furthermore, few reviews used an evidence-based framework (Wanzek et al., 2006; Peterson-Karlan, 2011), and none of them used formal assessments of study quality or meta-analytic tools – even though many of these articles were produced after evidence-based practice has developed reviewing standards; see for example Dollaghan (2004). Finally, qualitative and survey-based studies in particular have not been reviewed at all, to our knowledge. Some have argued that investment in AT is justified despite the meager systematic evaluation of the evidence base (McIntosh, 2009).

Considering these factors, we deemed that a systematic review of all available work was warranted, and would in fact be necessary to guide pedagogical decisions and disability support services in an evidence-based way.

2.2. Questions and hypotheses

We were interested in knowing whether AT interventions are effective for adolescents and adults with LD in secondary and post-secondary settings. On the basis of the above literature, we predicted mixed results, with some interventions more useful than others, and some possibly detrimental to learning. We sought to find out exactly which interventions were likely to be helpful.

We were likewise interested in the lived experience of people with LDs using these technological supports, and the broader social context of AT intervention. Therefore, we also assessed available qualitative and survey-based literature.

Our systematic review and meta-analysis addressed the limitations of past research by:

1. Conducting a broad search of research on the effectiveness of AT for students with LD and providing a description of our search.

2. Focusing on students at the secondary and post-secondary levels.

3. Focusing on students with LD.

4. Identifying which ATs are effective at enhancing learning for students with LD.

5. Providing both an analysis of the quality of the studies reviewed as well as a meta-analysis of the study results.

6. Including both qualitative and quantitative studies.

By focusing our review in this way, we aimed to provide the most useable information for students with LD as well as individuals who support their learning. Likewise, this systematic review and meta-analysis allowed us to identify limitations in the type of studies that are designed and conducted on this topic.

3. Methods

3.1. Inclusion criteria

Our goal was to locate AT interventions for students with LD in secondary and post-secondary settings; therefore, we searched for studies that primarily included participants who were in Grade 9 or higher or who were 14 years or older. To be as comprehensive as possible, we included peer reviewed, published studies as well as Ph.D. theses that were available on-line. We included both quantitative and qualitative work. We did not place geographic, cultural or time limitations on studies. We focused on research published in English, but we did not restrict our search procedure by language, meaning a study in potentially any language could have been included if the English-language papers cited it, or if it was indexed in the search engines we used by English keywords. During our queries we located and included only one non-English publication, a paper in Swedish (Milrad, 2010).

We included studies where some kind of AT was used. We used the definition of assistive technology from the Technology-Related Assistance for Individuals with Disabilities Act of 1988, commonly found in the literature: ”Any item, piece of equipment, or product system, whether acquired commercially off the shelf, modified or customized, that increases, maintains, or improves functional capabilities of individuals with disabilities.” (Sec. 3.) We excluded prototype design studies.

The definition of learning disabilities can vary from publication to publication; American studies often use the federal legal definition of LD:

”General. The term means a disorder in one or more of the basic psychological processes involved in understanding or in using language, spoken or written, that may manifest itself in an imperfect ability to listen, think, speak, read, write, spell, or to do mathematical calculations, including conditions such as perceptual disabilities, brain injury, minimal brain dysfunction, dyslexia, and developmental aphasia.

Disorders not included. The term does not include learning problems that are primarily the result of visual, hearing, or motor disabilities, of mental retardation, of emotional disturbance, or of environmental, cultural, or economic disadvantage. (34 C.F.R. § §300.7 and 300.541)”

We did not favor a specific definition and included studies with participants with ”learning disabilities” or ”learning difficulties” in general, alongside papers where participants had a specific disability usually considered to fall under LDs, like dyslexia. Studies of participants with attention deficits or autism spectrum conditions but no concurrent LDs were not included. Studies where the term ”learning disability” was used in a different way, e.g., to refer to general intellectual impairment, were not included. Studies were participants of multiple disabilities were grouped together were likewise not included, unless at least two thirds of the group had a diagnosis of LD. LD frequently co-occurs with other conditions, and thus we did not want to exclude studies with participants who also have further diagnoses.

3.2. Search method

We reported our literature search and data analysis process in the systematic reviews prospective database PROSPERO, registration number: CRD42015015918. The search was conducted in January 2015 and updated with 2015–2017 data due to peer reviewer requests in February 2017. Here we present the updated results.

For finding relevant literature, we used a combination of PubMed, Google Scholar (multiple methods), ERIC, manual citation search, and journal-based search. We provide the number of works yielded by each search method, with the caveat that these numbers can also overlap in some cases; better-cited articles can be located in multiple ways. In PubMed, we used the queries ”((”learning disabilities”) AND ”assistive”) AND ”technology”” which produced 18 articles, and ”((”learning disabilities”) AND ”assistive”)” (PubMed format), which produced 24 articles.

In Google Scholar, we used the queries ”learning disab* technolog*” and ”learning disab* technolog* college” (in 2015) and ”learning disabilities technology” and ”learning disabilities technology college” (in 2017, as the asterisk feature had been removed). The first twenty pages were evaluated (200 articles each page).

We also used another method of literature search: we chose older papers in the field, and examined the more recent papers that cited them using Google Scholar’s Cited By function. This allowed us to locate articles that either had low citation numbers, or were too recent to be indexed properly in topical databases, and thus might not be found using other methods. The articles we used as our starting point in Google Scholar Cited By were Day & Edwards (1996), which yielded 88 articles, and Yau et al. (1990), which yielded 6 articles. (Note that Yau et al. (1990) itself used a younger age group than our area of interest and thus was not included in this review.)

In the educational database ERIC we searched for the descriptor ”Learning Disabilities” with the field ”Assistive Technology”, an education level of ”Postsecondary Education”, a descriptor of ”College”, and we restricted the search to ”Peer Reviewed Only”. This yielded 113 articles. Since many articles in ERIC mention LD in passing, and more topical articles are not prioritized, less restricted search queries yielded thousands of results.

We also examined the citations of related review papers. Alper & Raharinirina (2006) yielded 96 articles. Bryant & Bryant (1998) yielded 98 articles. Freeman et al. (2005) yielded 96 articles. Peterson-Karlan (2011) yielded 123 articles. Holmes & Silvestri (2012) yielded 54 articles. Lang et al. (2014) yielded 55 articles.

To cover gaps in our more recent coverage, we also perused January 2012 – March 2017 articles in journals which had published at least two relevant articles in the past, based on the list of articles acquired during the previous steps. These journals included: the Journal of Learning Disabilities (248 articles), Learning Disability Quarterly (108 articles), Assistive Technology (195 articles), Annals of Dyslexia (70 articles), Assistive Technology Outcomes and Benefits (13 articles), Journal of Postsecondary Education and Disability (149 articles), Computers & Education (1339 articles), Journal of Vocational Rehabilitation (275 articles), and the Journal of Special Education Technology (94 articles).

We excluded five studies on the basis of them being prototype design studies aimed at developing a device or an interface – Lannen et al. (2002), Savidis et al. (2007), Mendi & Bayrak (2013), Williams & Hennig (2015a), Williams & Hennig (2015b).

The full text of all possibly relevant articles could be located after using our university’s InterLibrary Loan, with only a single exception where the citation itself was incomplete and was missing the publication name.

3.3. Data extraction

Author1 extracted multiple parameters of each article manually: the full citation, study method, diagnostic criteria used for inclusion, study context, sample size, sample gender distribution, sample age range, and geographic location of the study (on the country or where available, state or region level). Study methods were categorized as survey-based, qualitative, single-subject intervention, or group-design intervention. Study contexts were categorized as university, high school, special high school (for students with LD and/or other disabilities), or other.

3.4. Quality assessment

3.4.1. Assessing survey studies

There is a limited amount of research on the formal assessment of survey quality in systematic reviewing. Systematic reviews often explicitly exclude survey studies. General-purpose quality checklists used in evidence-based research do not tap the specific characteristics of survey studies. Although there is some precedent for using the Modified Downs-Black Checklist (Justice et al., 2008) for assessing survey studies, we have opted not to do this for multiple reasons. First, as we have used the same checklist for quantitative intervention studies, readers might conclude that the numbers would be comparable. However, as the survey and qualitative categories include theses, which often have lengthier methods sections, the quality ratings would probably be inflated compared to the quantitative studies. This means that even inside the survey studies and qualitative categories, ratings for articles and theses cannot be compared. (We did not locate any theses that featured quantitative studies.)

3.4.2. Assessing qualitative studies

Qualitative studies were assessed in a purely qualitative manner. The methodology of qualitative reviewing is much less defined at present than that of quantitative reviewing. Study quality assessment checklists like Tong et al. (2007) or Anderson (2010) generally have not been tested psychometrically – for example, metrics like inter-rater reliability are not provided. Therefore we cannot assess the usefulness of these checklists sufficiently.

After consultations with qualitative methods expert Renita R. Schmidt (personal communication), we designed the following procedure. Each paper was read, in temporal order of publication, and analytic notes were written by Author1 after each article – not upon finishing the entire set of papers, so as to allow what has been learned from each paper to influence the interpretation of the next. This is similar to the constant comparative method of analysis that is pervasive in qualitative research (for a recent description, see (Merriam, 2009)). We did not code articles line by line, as we deemed this unfeasible with the amount of data at hand. We performed two readings of each paper: the first in chronological order, then we sorted publications into topics and read them by topic group in chronological order.

3.4.3. Assessing group-design and single-subject studies

Two of us (BP, KRG) rated all group-design and single-subject intervention studies using the Modified Downs-Black Checklist (Justice et al., 2008). Both of us have a background in quantitative study design and developmental disabilities research.

The Modified Downs-Black Checklist was designed to assess the quality of treatment research in speech-language pathology. It has 25 binary yes-no items, for example Does study provide clear description of actual probability values of main outcomes or Are participants randomized to treatment conditions. We decided to add another item to this scale. We split the item Does study provide clear description of participant characteristics into demographic characteristics (age, gender, ethnicity etc.) and diagnosis characteristics. Therefore our maximum score for a given paper was 26.

To estimate inter-rater reliability, we calculated the intraclass correlation coefficient for average measures using a two-way random effects model. We determined inter-rater reliability for average quality scores was high (ICC = 0.828, 95% CI: 0.672–0.910, p-value for the F test < 0.001.). After this phase, we built item by item consensus, and the consensus ratings were used in the following analyses.

4. Results

We will proceed in this section in the following order: survey studies, qualitative studies, single-subject and group-design intervention studies (these last two grouped together). This way we proceed from studies with a broader focus toward studies with more circumscribed goals. Survey studies tend to have broad research questions, which is also true of some qualitative studies examining the lived experience of students with LD using assistive technology, whereas some qualitative studies have a narrower goal of assessing specific devices. The experimental and quasi-experimental publications tend to have very circumscribed aims. Publications with multiple components will be discussed in multiple sections.

4.1. Survey studies

4.1.1. Inclusion

Based on abstracts, we located 9 studies with a marked survey component; 5 met inclusion criteria. Table 1 presents information on the included survey studies. We will describe them in chronological order.

Table 1.

Survey-based studies in our review

Author	Year	n	Sample characteristics	Group	Gender	Country	Age	Journal
Abreu-Ellis C, Ellis JB	2006	17	”Individuals working at Disability Services Offices at Ontario universities representative of the 20 universities in Ontario”	Coordinators	N/A	Canada (Ontario)	Adults	Journal of Teaching and Learning
Gaiters-Fields	2005	10	”Psychological and educational assessment information was validated”	University	7 M, 3 F	US (Georgia)	21–26	THESIS
Klemes J, Epstein A, Zuker M, Grinberg N, Ilovitch T	2007	24 LD, 16 controls	”Previously diagnosed by the CSLD as students with LD”	University	4 M, 20 F	Israel	Undergrads	Open Learning: The Journal of Open, Distance and e-Learning
Parker DR, Banerjee M	2007	44 LD /ADHD, 98 controls	”Documented LD or ADHD”	University	18 M, 26 F	US	Undergrads	Assessment for Effective Intervention
Heiman T, Shemesh, DO	2012	363 LD, 601 controls	All the participants with LD were registered at the university’s Disability Support Center	University	188 M 175 F LD, 347 M 254 F controls	Israel	17–57	Journal of Learning Disabilities

4.1.2. Summary

Only two of the survey studies we located used statistical hypothesis testing: Parker & Banerjee (2007) and Heiman & Shemesh (2012). Other survey studies like Gaiters-Fields (2005), Abreu-Ellis & Ellis (2006), Klemes et al. (2006) reported only descriptive statistics.

The studies were often dated, presenting information from a time before major changes in university disability services. Further, the publications did not form a cumulative body of science and did not build on each other: each discussed a separate question, and they did not tend to cite each other. This restricted the conclusions we could draw from them.

Based on the two studies that contained inferential statistics, we can state that there is tentative data about the following: students with LD show a different technology use profile than students with ADHD and typical development (for both assistive and general-use technology); for example, students with ADHD are more comfortable with presentation software than either the LD or TD groups. Data are mixed on whether students with LD are more likely to use internet-based technology than the other two groups – this might depend on the particular kind of technology and how much reading it requires. There is a small amount of data related to students’ well-being and hope: for example, there is some evidence that students with LD might self-report better well-being and a more hopeful outlook, connected to AT use. This latter result might be culture-specific to Israel, as we have seen conflicting data from the United States in less controlled studies and a large-scale quantitative survey from our laboratory (McGregor et al., 2016).

The three studies that included only descriptive statistics also provided information on disparate topics. Gaiters-Fields (2005) queried African-American students with LD at a Historically Black University, as part of a larger project. All students (n = 10) used AT, but self-funded their AT due to little support from the university. Abreu-Ellis & Ellis (2006) surveyed professionals who worked in disability services offices in 17 universities of Ontario, Canada. Most participants strongly agreed that incoming students with LD needed to be trained in using AT and indicated that their university provided this service on a one-on-one basis. About half of the respondents stated that their office also provided training in small group settings. Challenges the participants listed in response to open-ended questions included ”consistency in assistive technology use by the students, effective training while semester coursework is in progress, and fitting unique individuals with very unique needs to the available technology” (p. 39). Klemes et al. (2006) examined postsecondary students with LD using a distance-learning course with electronic units that featured multimedia presentation of the subject matter. Students self-reported in the survey that they spent less time studying with these materials than with conventional materials. They also claimed to use many of the electronic components (like search or copy-paste). Most students stated they enjoyed the electronic units, though a minority expressed dislike, primarily about the inconvenient format.

4.2. Qualitative studies

4.2.1. Inclusion

We located 15 primarily qualitative studies based on their abstracts, and excluded two after reading them. One thesis, Roberts (2003) was also published in shortened form as an article Roberts & Stodden (2005). Table 2 shows descriptive information.

Table 2.

Qualitative studies in our review

Author	Year	n	Sample characteristics	Group	Gender	Country	Age	Journal
Anderson-Inman L, Knox-Quinn C, Szymanski M	1999	3	“students who participated in one or more of the above projects”	High school, University	1 M, 2 F	US (Oregon)	High school sophomore, ?, 22	Career Development for Exceptional Individuals
Bradshaw YM	2001	2	“LD student with the defining categories of auditory processing deficits and related language difficulties in spelling and reading and did not have any cognitive impairments“	University	1 M, 1 F	US (Virginia)	Adults	THESIS
Chiang H-Y, Liu C-H	2011	15	All of the participants were diagnosed as having LD and dyslexia.	High school	15 M	Taiwan	High school grades 1–3	Assistive Technology
Dodge KM	2012	8	All participants selected for this study had been diagnosed with a reading comprehension learning disability, had used accommodations at King Community College, and had attended the college for at least one semester	University	2 M, 6 F	US (Midwest)	18–40	THESIS
Dziorny MA	2012	92 survey (inc. TD), 3 case study (+ 5 TD)		University	3 M, 5 F (LD 1 M 2 F)	US (Texas)	19–60	THESIS, Proceedings of SITE
Gaiters-Fields	2005	3	“Psychological and educational assessment information was validated”	University	1 M, 2 F	US (Georgia)	21–26	THESIS
Graves L, Asunda PA, Plant SJ, Goad C	2011	11LD / ADHD	“provided documentation of either an LD and/or ADHD to the Office of Disability Services“	University	9 M, 3 F	US	Adults	Journal of Postsecondary Education and Disability
Milrad MB	2010	9 (students)	dyslexia	University / Other	3 M, 6 F	Sweden	20–50	THESIS
Roberts KD, Stodden RA	2003, 2005	15	“receiving services under the category of learning disabled”	University	9 M, 6 F	US (Hawai’i)	19–56	THESIS, Journal of Vocational Rehabilitatio
Woodfine BP, Baptista Nunes M, Wright DJ	2006	12 (1), 20(2) – does not say how many LD and controls	“The students who have dyslexia were selected using their Adult Dyslexia Diagnostic (ADI) score (the dyslexia assessments carried out by the University of Sheffield gives an ADI score).“	University	N/A	UK (Sheffield)	N/A	Computers & Education
Young G	2013	12 students, 12 parents	“Formally diagnosed with a learning disability, with or without attention deficit hyperactivity disorder“	Special high school / other	N/A	Canada (Ontario)	Grade 10?	Technology-mediated Learning

4.2.2. Studies: AT as one component of accommodations

Four of the qualitative studies had a broader focus, with AT only one of the areas investigated: Bradshaw (2001), Gaiters-Fields (2005), Milrad (2010), Dodge (2012). These studies examined the impact of all accommodations on the lived experience of university students with LD, primarily using interviews, but also with various forms of participant observation.

Bradshaw (2001) provided two detailed case studies of university students with LD in Northern Virginia, to identify which factors were important for their academic success. One student mentioned using technological accommodations, but stated he did not benefit from most of them, with the exception of a tape recorder. Accommodations and support strategies other than AT were much more helpful in his university studies; he especially benefited from anxiety-related counseling, and reading remediation so that he eventually no longer needed to rely on books on tape. The second student benefited more from AT than the first student. Although her university had accommodations for students with LD, these were not technological. Instead, she tape-recorded her classes and learned how to use a laptop computer with the help of her husband. The laptop was also useful because she could look up resources for people with LD, including information about her legal rights as a student.

Gaiters-Fields (2005) provided case studies of three African-American undergraduate students with LD at a Historically Black University, who were chosen from the ten survey respondents mentioned above. One of the students, Kensley, was described as heavily relying on technology ”to assist him in his studies and personal management.” The university did not provide students with AT at the time of the study, and the researcher mentioned this as an area of concern: if Kensley had not been able to afford his AT supports himself, he would have had great difficulty coping with the university environment. The specific details of Kensley’s technology use were not described. The other two students’ AT use was not emphasized in the study.

Milrad (2010), focused on Swedish higher education students with dyslexia. Out of nine students interviewed, two did not use any AT. All others used audiobooks, although one student stated he tried this type of AT, but it did not work well for him. The other students spoke highly of audiobooks. Students often used spell checkers, either Word’s built-in spell checker or Stava Rex / Stava Rätt, a spell checker developed in Sweden specifically for people with dyslexia that can be integrated into various word processing programs (including Word). Two people had problems with Stava Rex: one student liked it but could not get it on his computer, and the other did not find it good enough for her purposes. Some students also used text-to-speech. Two people mentioned Quicktionary, a handheld scanner that translates English words into Swedish. Students reported mixed experiences with AT; they tried forms of AT that they ultimately did not adopt, and they sometimes abandoned AT altogether.

The two students who claimed they had never used AT received an intervention where they were provided a speech-to-text program, Voice Xpress. Both students used Voice Xpress extensively to write essays, but both reported that it often had trouble understanding their speech. They found it especially problematic that the errors it produced were of the type not recognized by spell checkers. Eventually, both students abandoned the software.

Dodge (2012) interviewed eight students with reading comprehension disabilities at a US Midwestern community college. Students received a variety of accommodations, including AT: for example, books on CD, smart pens, laptops, and recorded lectures. All students used multiple accommodations. Overall, they perceived the non-AT accommodations to be more valuable than the AT accommodations. A participant mentioned that books on CD were often hard to use. Other non-AT accommodations like reading the test out loud, dictating answers, and extended time on tests were described as more useful. In general, testing accommodations were more appreciated than classroom accommodations, but testing accommodations were also primarily not technological.

4.2.3. Studies: Perspectives about AT

Young (2013) also addressed a broader theme: what do high school students and their parents think about AT? Twelve Canadian students who each had LD with or without ADHD and their parents participated in the study. All students attended a special school for people with LD. Both groups were asked about their experience of AT use prior to and during their attendance of the special school. The overall tone of the responses was positive, but some drawbacks of AT were noted. Students and parents reported that AT helped the students finish tasks, demonstrate their academic ability, and improve their writing. It let them compensate for their difficulties, and increased their confidence. Some students felt AT increased motivation, but others felt it decreased motivation, because it was often a hassle. Two students also reported they felt stigmatized by their use of AT. Three parents and four students mentioned that they found AT frustrating.

4.2.4. Studies: Technological course supports for students with LD

Four qualitative studies examined technology-mediated course design or complex technologically supported studying for students with LD.

Anderson-Inman et al. (1999) featured three case study vignettes of students with LD who were enrolled in two intervention programs in a high school, community college or a university focusing on computer-assisted studying. Some of the students received a laptop and participated in a course titled ”Computer-Based Study Strategies.” Instruction was personalized to specific students’ needs. Other students participated in a networked note-taking intervention helping students with LD learn efficient note-taking by sharing a virtual workspace with a note-taker. Out of the three vignettes presented, one student received both the laptop and strategies intervention and the networked note-taking intervention, one received only the laptop and strategies intervention, and one received only the networked note-taking intervention. All three students achieved more academic success than before the intervention, but some initially struggled with the technology.

Woodfine et al. (2008) investigated text-based synchronous e-learning in university students with dyslexia. The authors found that students with dyslexia struggled more in text-based synchronous e-learning contexts than students with typical development. Participants had to collaborate in groups of three to solve a survival scenario: one student with dyslexia and two without. The interactions took place in a WebCT environment, which had descriptions of the problem to be solved and a chat room where participants could collaborate. Students were interviewed after the problem-solving exercise. Participants with dyslexia reported difficulty with typing and spelling and also with reading other students’ responses. They also experienced negative emotions related to their performance. They felt isolated and embarrassed, especially by their spelling, and they thought they often failed to convey their meaning. Participants who were typically developing noticed and remarked on the lack of participation on the part of the students with dyslexia, but they sometimes misinterpreted it, for example by assuming that the student was not computer literate.

Graves et al. (2011) examined asynchronous online access. The asynchronous online component involved recordings of class presentations, specifically intended to be an accommodation for students with disabilities. Postsecondary students with LD or ADHD who were enrolled in STEM courses which had an asynchronous online component were interviewed about their experience. Students were overall favorably disposed toward this accommodation. They claimed that having access to course recordings increased clarity, convenience and comfort, and helped them study at their own pace. Participants anticipated higher grades for themselves and felt that the accommodation helped them cope with their disability. On the other hand, some students found the structure of the downloadable materials confusing. They also experienced some technological issues.

Dziorny (2012) designed a course module in Second Life™ (Linden Labs) with the assumption that learning in a virtual spatial environment would be beneficial for students with dyslexia. This study contained a lengthy survey segment, but survey data were presented with all participants grouped together, the majority of whom did not have a learning disability. In its qualitative segment, participant data were presented individually, with students with dyslexia clearly identified. Eight people participated in a single module of an introduction to communications course, three of whom had dyslexia and five who were typically developing. They were observed while participating, both inside and outside the virtual environment, and they were also interviewed twice about their experiences, learning history and preferences. Participants both with and without dyslexia liked the course, and most of them felt it had met their needs. However, all of them experienced technical difficulties with Second Life, and one participant (without dyslexia) was unable to use it.

4.2.5. Studies: Specific assistive supports

Two studies investigated specific kinds of AT: Roberts & Stodden (2005), Chiang & Liu (2011). Roberts (2003)’s thesis, later published as an article in Roberts & Stodden (2005) examined the use of a voice recognition system, Dragon Naturally Speaking™ (Nuance), in a sample of 15 students with LD at different postsecondary institutions. Dragon™ is the current market leader in voice recognition. Data were gathered using a variety of methods including interviews, focus groups, participant observation, and writing samples. Participants received training in the use of the software as well as ongoing support. The main research questions involved the continued use of the system, and the variables influencing it. Only two students continued to use Dragon after their training was completed. Important variables related to continued use were ”time, access to a personal computer, ease of use, personal issues, use of standard English, the specific limitations associated with a person’s disability, whether or not the subjects had any other compensatory strategies in place, and the acquisition of skills necessary to use the software” (p. iv).

Chiang & Liu (2011) sampled Taiwanese high school students having both LD and dyslexia diagnoses, who were studying English as a second language. During their English classes, they had the opportunity to use Kurzweil 3000™ (Kurzweil Educational Systems) text-to-speech software, and after two weeks, they were interviewed about their experience. They liked that the software could be customized to their own preferences (for example, in reading speed) and that they could ”use this software to read repetitiously” (p. 202). Several participants reported that they also used Chinese-English electronic dictionaries, and they all preferred the Kurzweil 3000 to their electronic dictionary. However, they also complained that it was very hard to use it to look up dictionary definitions, as the software did not have Chinese-English vocabulary definitions, unlike their preexistent devices. Students also felt that the software helped them both with spelling and with pronunciation, but they did not report an improvement of their general academic performance. The authors speculated this was probably due to the short length of the study. Readers should note that the authors also had a separate publication with quantitative results from this intervention, included in our group designs section Chiang et al. (2012).

4.2.6. Quality assessment

The quality of the qualitative studies summarized above was mixed due to multiple factors. Participant recruitment and follow-up was a notable issue. For instance, in Graves et al. (2011), even though many more students participated in a larger implementation study, only about 25% of them were available for interview; and in Dziorny (2012), the author could locate only three participants with dyslexia, and they were grouped together with typically developing students (for this reason we could not use the survey segment from this publication, only the qualitative segment). This difficulty in recruitment resulted in a lack of data saturation – participants each had markedly different reactions, and we could not estimate whether additional participants would follow similar patterns or provide altogether different data. Participant attrition was likewise problematic. For example, in Roberts & Stodden (2005), acquiring quantitative data of improvement proved largely unsuccessful due to difficulties in reaching participants for follow-up.

Triangulation was sometimes lacking due to reasons not related to recruitment: for example in Graves et al. (2011), there was no independent assessment of whether students’ performance improved, or whether there was a relationship between positive experiences with the technology and performance improvement. Further, data were on occasion selectively reported: for example, Anderson-Inman et al. (1999) described ”stories of successful transition from secondary to postsecondary education” in the course of a complex, federally funded intervention project. Unsuccessful stories were not mentioned.

Theoretical grounding was also a relative weakness. Sometimes the reason for performing a qualitative study was unclear in itself: intervention projects like Woodfine et al. (2008) or Dziorny (2012) seemed to lend themselves more to quantitative research. In some cases – for instance, Woodfine et al. (2008) – the conclusions sections were brief and non-analytical.

Despite these deficiencies, we found valuable data in the qualitative literature, often focusing on aspects of the academic experience of students with LD that went unreported elsewhere. An important conclusion we could draw was whenever triangulation was performed, results almost always supported the students’ own claims. This was true even when students offered strongly negative opinions of official support. For example, in Bradshaw (2001), one of the participants claimed that he did not benefit from most AT at the university: the institution either did not have current computer software or his counselor did not know how to use it. The counselor independently confirmed this.

4.2.7. Summary

Qualitative researchers often explicitly claimed that they had no intent to provide generalizable data (for example, Dziorny (2012)) or stated that they would have liked to provide generalizable data, but they were not able to do so (for example Dodge (2012)). Yet we saw many commonalities in the papers we described.

More recent studies revealed that higher education institutions provided more AT supports than older studies – this could be expected. However, even in recent studies, it was a consistent theme that students with LD did not necessarily appreciate or utilize AT supports provided by the institution. Some students were heavy users of AT, but others found it a hassle. Those who were AT users often acquired and set up AT independently of their institution.

Many participants tried but subsequently abandoned AT. In some cases, AT became unnecessary with practice of the original skill it supported. Even on the post-secondary level, we saw mentions of reading remediation being beneficial, somewhat contrary to the view that these kinds of interventions are no longer useful with adult students and AT use should be preferred (Reis et al., 2000). In other cases, AT was abandoned because of frustration. Technical difficulties were reported with every device or software, and different participants had different difficulties with the same intervention. Often, university professionals were unable to provide effective technical support for AT. Negative emotions associated with AT were mostly connected to these technical issues, but some students also felt stigmatized because of their AT use.

Even when participants enjoyed using their AT devices, it was hard to tell from these studies whether this resulted in a performance improvement; in any case, the improvement in subjective well-being was marked (also in line with the small amount of quantitative survey data relevant to this question). AT use also had advantages beyond academic support, for example by helping students advocate for themselves.

We can conclude that, based on extensive and varied qualitative data, some students clearly benefited from AT use, but AT use should be custom-tailored to the individual, and technical support should be provided. Even when technical support was available, some people failed to respond to AT interventions. Negative emotions about AT were expressed by many students, even students who benefited from AT. Finally, some kinds of AT could be harmful to students with LD; most notably, synchronous online course components, as these required rapid reading and writing. In contrast, asynchronous online course components enabled students to work through the material at their own pace and were perceived by students to be more useful.

4.3. Group-design and single-subject intervention studies 4.3.1. Inclusion

After reading the abstracts, 45 papers were included as either single-subject (n = 9) or group-design (n = 36) interventions. Upon reading the full text of articles, 7 single-subject publications and 31 group-design publications met criteria for inclusion. These were then grouped by topic on the basis of which AT device/s they used for intervention. Six groups were produced:

1. Text-to-speech, including complex computer-based interventions with a primary text-to-speech component – 11 publications (13 studies)

2. Speech-to-text – 6 publications (6 studies)

3. Word processing, including spell or grammar check – 5 publications (5 studies)

4. Multimedia and hypertext – 7 publications (8 studies)

5. Smart pens – 4 publications (4 studies)

6. Other computer-based interventions – 4 publications (4 studies)

We used these groups to present and analyze our data. Most publications contained only a single study. For the exceptions, publication quality ratings refer to the entire paper, not to individual studies.

Despite categorization by AT type, some categories were too heterogeneous to support a metaanalysis. Table 3 shows the analyses we applied by topic. We calculated effect sizes in all cases where sufficient data were available from the original publications, and where the outcome variable was suitable. (In the case of single-subject studies, we used Beeson & Robey (2006) where the design permitted it, with adjustment for bias as in the group-design Hedges’ g; we note where we departed from this.)

Table 3.

Analyses performed on topic groups

	Effect summaries	Forest plot	Metaanalysis	Outcome variable
Text to speech	Yes	Yes	Yes	Reading comprehension / score change
Speech to text	Yes	No	No	N/A
Word processing	Yes	Yes	Yes	Error rate change
Multimedia	Yes	Yes	No	Score change
Smart pens	Yes	Yes	Yes	Reading comprehension
Other	Yes	No	No	N/A

For each study, we will proceed to report the major significant differences found, with p-values, and the size of said differences in the format provided in the publication itself (for example, raw means difference or percentage change). We provide standardized effect sizes (Hedges’ g) for our chosen outcome variable, calculated with the software package Comprehensive Meta-Analysis, version 3.

Where multiple comparisons or contrasts existed, we used only an intervention / no intervention measure, and where there were both pre- and post- intervention and no intervention measures, we used the post-measures. We did not use data from typically developing controls or controls with other disabilities or remedial education. Where there were different kinds of interventions (for example, a word processor with various features turned on/off), we used a contrast between no intervention and maximal intervention.

In all cases we used random effects models for metaanalyses due to heterogeneity in the data. Because the number of studies in each subgroup was small, we opted not to report funnel plots for publication bias.

4.3.2. Text-to-speech (speech synthesis, computer-assisted reading, screen reading)

Text-to-speech (TTS) interventions use software or, in early cases, combined software/hardware devices to provide synthesized speech. The computer reads out text to the user with reading difficulties.

Table 4 provides demographic and other background information on the studies, while Table 5 features methodological data and the results of the quality assessment.

Table 4.

Text-to-speech publications – demographic and other background information

Authors	Year	Studies in paper	Journal	n	Diagnosis	School	Gender	Location	Age
Calhoon MB, Fuchs LS, Hamlett CL	2000	N/A	Learning Disability Quarterly	81	identified as having LD, who were receiving math and reading instruction in special education resource rooms and had reading and math IEP goals	High school	61% M	US	Grades 9–12
Chiang H-Y, Liu C-H, Lee S-J, Shih Y-N	2012	N/A	Work: A Journal of Prevention, Assessment and Rehabilitation	29	All of the participants were diagnosed as having LD and dyslexia.	High school	29 M	Taiwan	High school
Elkind J, Sandperl Black M, Murray C	1996	Study 1	Annals of Dyslexia	50	“Criteria used for diagnosis varied”	University	N/A	US (California)	Adults
Elkind J, Sandperl Black M, Murray C	1996	Study 2	Annals of Dyslexia	29	“Criteria used for diagnosis varied”	University	N/A	US (California)	Adults
Elkind J, Sandperl Black M, Murray C	1996	Study 3	Annals of Dyslexia	8	“a formal diagnosis of learning disability”	Adults	N/A	US (California)	Adults
Elkind J, Sandperl Black M, Murray C	1996	Study 4	Annals of Dyslexia	12	“diagnosed as dyslexic with one of the Slingerland Screen Tests”	Adults	N/A	US (California)	Adults
Falth L, Svensson I	2015	1	International Journal of Teaching and Education	5	”performed one SD below mean on a word decoding test and on a phonological decoding test […] fulfilled the criteria for a dyslexic profile”	High school	N/A	Sweden	Grade 10
Floyd KK, Judge SL	2012	N/A	Assistive Technology Outcomes and Benefits	6	Inclusion in STEPP program		4 M, 2 F	US (Southeast)	18–20
Higgins EL, Zvi JC	1995	Study 1	Annals of Dyslexia		See Higgins & Raskind 1997
Higgins EL, Zvi JC	1995	Study 2	Annals of Dyslexia		See Raskind & Higgins 1995
Higgins EL, Raskind MH	1997	N/A	Learning Disabilities	37	All students had previously been identified as having a learning disability according to State University system-wide criteria	University	21 M, 16 F	US (California)	19–37
Lange AA, McPhillips M, Mulhern G, Wylie J	2006	N/A	Journal of Special Education Technology	93	“at least one year behind in reading age”	High school	47 M, 46 F	Northern Ireland	14.5–15.8
Lewis RB	1998	Study 1	Report		See Lewis et al 1998 (Word processing section)
Lewis RB	1998	Study 2	Report		See Lewis et al 1998–1999 (Word processing section)
Lewis RB	1998	Study 3	Report	103 LD, 120 non-LD	Chosen by “teachers serving students with learning disabilities)	High school	59% LD M	US	Grades 4–12
Olson R, Foltz G, Wise B	1986	N/A	Behavior Research Methods, Instruments and Computers	11 High school, 15 PS	“referred by their reading teachers”	High school	N/A	US (Colorado)	15–18
Raskind MH, Higgins E	1995	N/A	Learning Disability Quarterly	33	All students had previously been identified as having a learning disability according to State University system-wide criteria	University	19 M, 14 F	US (California)	University

Table 5.

Text-to-speech publications – design and quality

Authors	Year	Studies in paper	Intervention	Design	Outcome measure	Study groups	Rating
Calhoon MB, Fuchs LS, Hamlett CL	2000	N/A	TTS: TTS+video on math exam	Group design	School testing scores	No assistance, teacher reading, computer reading, computer reading with video	21
Chiang H-Y, Liu C-H, Lee S-J, Shih Y-N	2012	N/A	TTS: Kurzweil 3000	Group design	Testing scores EWRT, GEPT	LD students pre/post	18
Elkind J, Sandperl Black M, Murray C	1996	Study 1	TTS: BookWise	Group design	Reading speed, timed and untimed comprehension scores	Intervention, no intervention / Timed and untimed test	13
Elkind J, Sandperl Black M, Murray C	1996	Study 2	TTS: BookWise	Group design	Reading speed, timed and untimed comprehension scores	N/A	13
Elkind J, Sandperl Black M, Murray C	1996	Study 3	TTS: BookWise	Single subject	Reading speed, comprehension scores	N/A	13
Elkind J, Sandperl Black M, Murray C	1996	Study 4	TTS: BookWise	Single subject	N/A	N/A	13
Falth L, Svensson I	2015	1	Prizmo TTS app	Group design	Word decoding skill	Grade 5 and 10 students pre/post	11
Floyd KK, Judge SL	2012	N/A	TTS: ClassMate Reader	Single subject	Reading comprehension quiz	N/A	18
Higgins EL, Raskind MH	1997	N/A	TTS: SoundProof	Group design	Silent reading score	Computer, human, no assistance	18
Higgins EL, Zvi JC	1995	Study 1	See Higgins & Raskind 1997				9
Higgins EL, Zvi JC	1995	Study 2	See Raskind & Higgins 1995				9
Lange AA, McPhillips M, Mulhern G, Wylie J	2006	N/A	TTS: Read & White Gold + Computer: spellcheck, dictionary, homophone tool	Group design	Reading comprehension, word meaning, homophone error detection, spelling error detection	Assistive software, MS Word assistance control, MS Word no assistance control	20
Lewis RB	1998	Study 1	See Lewis et al 1998 (Word processing section)				16
Lewis RB	1998	Study 2	See Lewis et al 1998–1999 (Word processing section)				16
Lewis RB	1998	Study 3	TTS: Write:OutLoud	Group design	Writing quality, error rates, attitude toward writing, error types	Speech for text entry, revising, text entry + revising / LD computer, LD handwriting, TD no intervention	16
Olson R, Foltz G, Wise B	1986	N/A	TTS: DECTalk	Group design	Oral errors, targeted oral errors, recognized words, comprehension	With and without computer feedback, oral vs silent reading	8
Raskind MH, Higgins E	1995	N/A	TTS: SoundProof	Group design	Errors found, errors by category	Computer, human, no assistance	19

We chose reading comprehension as our outcome variable. When some academic examination or testing score of the material was provided instead of a direct reading comprehension metric, we took that also as a measure of reading comprehension.

The large number of studies is deceptive: out of the 13 unique studies, several reported no statistics useful for calculation of effect sizes – Olson et al. (1986), Studies 3–4 of Elkind et al. (1996), Lange et al. (2006), Lewis (1998b) and Floyd & Judge (2012). A further study (Study 2 of (Elkind et al., 1996)) was an expansion of a previous one. In the remaining studies, outcome variables were often hard to compare – for example, Fälth & Svensson (2015) did not provide a reading comprehension metric per se, only a word boundary detection metric (though with a moderately positive effect size of g = 0.716).

Selective presentation of data was also an issue: for instance, Olson et al. (1986) only presented data from 9 out of 26 participants, as other participants either experienced a floor or ceiling effect, and Lewis (1998b) and Lange et al. (2006) did not provide standard deviations or enough data to calculate them.

Table 6 shows the forest plot with all studies we could include. The overall effect was small (Hedges’ g) of 0.445 (p = 0.06, just over the common threshold of significance), and this effect further decreases if we exclude the one large effect, Chiang et al. (2012) as an outlier (0.167; p = 0.094) (Table 7). With the removal of that study, the null effect falls into the 95% confidence interval.

Table 6.

Forest plot and metaanalysis of text-to-speech studies

Table 7.

Results with one study excluded

At the group level, text-to-speech interventions have small effects. That said, there are patterns within groups that suggest benefits for some individuals. Therefore, more data should be gathered. To ensure maximum comparability with previous results, a reading comprehension measure should be included among the set of outcome measures, and the size of improvements should be correlated with unassisted / baseline performance.

Some studies also investigated which variables can lead to intervention success. Elkind et al. (1996) and Higgins & Raskind (1997) reported large negative correlations where the lower the initial score, the more the improvement from TTS. However, Calhoon et al. (2000) reported higher initial reading levels associated with more improvement. There can also be an age-dependent effect. Lewis (1998b) found that in secondary school students, as opposed to primary school students, performance decreased in response to intervention. These issues can all potentially produce the relatively small effect size in our metaanalysis, because the interactions might obscure the main effect.

A large amount of papers in this category featured very old papers and/or papers with low quality ratings; this happened primarily because for historical reasons, text-to-speech was one of the first forms of AT applied in LD. The question arises whether we are able to draw any conclusion from this data set. TTS systems have undergone vast improvement in the last two decades, and this might also mean present-day TTS interventions could be more effective. This is very tentatively supported by the fact that the most recent cases where we were able to calculate effect sizes from papers were also higher (Chiang et al., 2012; Fälth & Svensson, 2015) – though one needs to note that Chiang et al. (2012) found a null effect on one of the three metrics the authors used.

Given the small positive effect, we recommend more causal experimentation, with a specific focus on performance interactions. We hypothesize that older learners and/or more advanced readers are less likely to benefit from TTS.

4.3.3. Speech-to-text (speech recognition, computer-aided dictation)

Speech-to-text interventions use software to recognize the user’s voice and translate it into computer commands. Speech-to-text is used as AT by people with various disabilities, including motor conditions which prevent users from typing, but this kind of software can also facilitate writing and computer use in LDs. All studies in this category used a version of the Dragon™ voice recognition software.

Table 8 provides demographic and other background information on the studies, while Table 9 features methodological data and the results of the quality assessment.

Table 8.

Speech-to-text publications – demographic and other background information

Authors	Year	Journal	n	Diagnosis	School	Gender	Location	Age
Higgins EL, Raskind MH	1995	Learning Disability Quarterly	29		University	17 M, 12 F	US (California)	Adults
Higgins EL, Raskind MH	2000	Journal of Special Education Technology	52	“previously identified as having a LD”	Special high school	31 M, 21 F	US (California)	9 to 18
Litten M	1999	Dyslexia	N/A	N/A	Special high school	N/A	N/A	Grade 10 (14–15)
MacArthur CA, Cavalier AR	2004	Exceptional Children	31 (21 LD, 10 TD)	“identification by the school district as a student with a learning disability”	High school	14 M, 6 F (LD), 3 M, 7 F (TD)	US	High school
Millar DC, McNaughton DB, Light JC	2005	Journal of Postsecondary Education and Disability	3	“had a documented learning disability, specifically in writing”	University	3 F	US (Northeast)	22–24
Raskind MH, Higgins EL	1999	Annals of Dyslexia	See Higgins et al 2000

Table 9.

Speech-to-text publications – design and quality

Authors	Year	Intervention	Design	Outcome measure	Study groups	Rating
Higgins EL, Raskind MH	1995	STT: DragonDictate	Group design	Exam scores	Computer, Human, No assistance	22
Higgins EL, Raskind MH	2000	STT: DragonDictate, IBM VoiceType	Group design	Word recognition, spelling, reading comprehension, phonological deletion, orthographic choice, semantic choice, metacognitive choice, WM	Continuous speech recognition vs discrete vs none	17
Litten M	1999	STT: DragonDictate Classic + Keystone	Group design	N/A	N/A	2
MacArthur CA, Cavalier AR	2004	STT: Dragon Naturally Speaking	Group design	Quality, length, vocabulary, total errors, unknown words, time spent	Computer, Human, No assistance	18
Millar DC, McNaughton DB, Light JC	2005	STT: Dragon Naturally Speaking	Single subject	Accuracy and rate of transcription	N/A	19
Raskind MH, Higgins EL	1999	STT: DragonDictate+IBM VoiceType	Group design	See Higgins et al 2000		17

Overall, the speech-to-text studies were surprisingly heterogenous considering they used the same software. Designs were different and outcome variables were not directly comparable. Even total error rate, a very straightforward measure, was only reported by two studies. Therefore, we opted not to include a forest plot or to conduct a meta-analysis.

One further issue with the studies reviewed in this section is that Dragon underwent great improvement in the time span of these studies (1995–2005) and since then (Huang et al., 2014). Many of the studies contained remarks about how the technology is often unreliable and does not recognize voices of certain people. The field of machine learning has seen a great deal of growth in the past decade, so we have no way of knowing if we can extrapolate from any of the earlier technological failures and shortcomings to present-day speech recognition technology. In any case, the studies we located seemed to have positive outcomes, but research was not cumulative, and two studies out of a total of five did not present enough information for us to evaluate. Still, we can a fortiori assume that if a technology was successful in its previous iterations, it is likely to be at least as much, and probably more successful in current, more improved iterations. Thus we cautiously recommend both the use of STT to assist in learning, and more investigation as it is likely to lead to further positive results.

4.3.4. Word processing, including spell and grammar check

The built-in features of modern word processing software like spelling or grammar checks, or composition aids, are usually designed for typically developing users. However, many people with LDs report that they use these software features extensively and find them beneficial. Interventions in this category examine the effect of word processing aids on the learning outcomes of the LD population. Table 10 provides demographic and other background information on the studies, while Table 11 features methodological data and the results of the quality assessment.

Table 10.

Word processing publications – demographic and other background information

Authors	Year	Journal	n	Diagnosis	School	Gender	Location	Age
Collins, T.	1990	Computers and Composition	57	Assessments of their disability on file	University	N/A	US (Minnesota)	Adults
Lewis RB, Ashton TM, Haapa B, Kieley CL, Fielden C	1998–1999	Learning Disabilities	106 LD, 97 NLD	“Special education teachers provided results of recent assessments”	High school	68% LD M	US	Grades 4–12
Lewis RB, Graves AW, Ashton TM, Kieley CL	1998	Learning Disabilities Research & Practice	108 LD, 22LD controls, 132 TD controls	“identified by the district as having a specific LD”	High school	63% LD M	US	Grade 4–12
McNaughton D, Hughes C, Clark K	1997	Journal of Learning Disabilitiesil	11	“identified as learning disabled according to the federal guideline adopted by the university’s Program for Learning Disabled Students (i.e., a difference between intelligence and achievement percentile rank in reading, math, written language, general knowledge, or foreign language of at least 40 points); and (c) identified as having a functional difficulty in spelling	University	9 M, 3 F	US (Northeast)	19–32
McNaughton D, Hughes C, Ofiesh N	1997	Learning Disabilities Research & Practice	3	“identified as having a specific LD as defined by the Commonwealth of Pennsylvania)	High school	1 M, 2 F	US (PA)	16–18

Table 11.

Word processing publications – design and quality

Authors	Year	Intervention	Design	Outcome measure	Study groups	Rating
Collins, T.	1990	Computer: Word processing	Group design	Course completion rates, course grades, Daly-Miller Scale of Writing Apprehension, fluency of writing samples	LD, nonLD	12
Lewis RB, Ashton TM, Haapa B, Kieley CL, Fielden C	1998–1999	Computer: spell check, grammar check	Group design	Writing quality, error rates, attitude toward writing, error types	3 experimental groups (spell check, grammar check, TTS in different combinations) + no intervention / LD, nonLD	19
Lewis RB, Graves AW, Ashton TM, Kieley CL	1998	Computer: Word processing + word prediction	Group design	Writing speed, accuracy, quality	5 experimental groups / LD, nonLD	23
McNaughton D, Hughes C, Clark K	1997	Computer: spell check + TI LM-6000 handheld spell checker	Group design	Spelling error rates, detection of errors, correction of errors, errors in the final text, time to correct an error	5 proofreading conditions, repeated measures in same group	18
McNaughton D, Hughes C, Ofiesh N	1997	Computer: spell check	Single subject	Strategy use, percentage of errors corrected, final error rate	N/A	15

There are many possible outcome measures in this avenue of research, which adds heterogeneity. Four out of the five studies reported some kind of final error rate measure, and three out of the five reported some kind of quality measure. Quality measures varied, so we opted to use error rate changes in response to interventions as our outcome variable. We produced effect sizes where possible; in the single subject design study McNaughton et al. (1997), for one student (Case 2), there was only one post-intervention measurement, and thus we could not produce a standard deviation to calculate an effect size.

Even though there were only a handful of studies, we still decided to run a metaanalysis, as the beneficial effects were large. As shown in Figure 12, the overall effect is a large −1.626 (Hedges’ g), with a p-value of 0.002. All studies in the plot show a negative effect because the error rate decreases; meaning the intervention was successful.

These interventions have been successful, but we must note their limitations regardless. We did not manage to locate many studies, and many that we did find were dated, a worry given that word processing technology has changed rapidly in the past decades. Still, we can safely assume that if this technology was highly effective in the past, its effectiveness is unlikely to decrease.

4.3.5. Multimedia and hypertext

Multimedia and hypertext interventions are more heterogenous than either speech-to-text or text-to-speech. They usually use some form of multimedia, such as illustrated hypertext or audiovisual presentations, to facilitate the learning of people with LDs. 7 papers in this category presented information about 8 distinct studies. Table 13 provides demographic and other background information on the studies, while Table 14 features methodological data and the results of the quality assessment.

Table 13.

Multimedia publications – demographic and other background information

Authors	Year	Journal	n	Diagnosis	School	Gender	Location	Age
Higgins K, Boone R	1990	Journal of Learning Disabilities	10 LD, 15 remedial, 15 regular	by Washington state definition	High school	29 M, 11 F	US (Seattle, WA)	Grade 9
Higgins K, Boone R	1990	Journal of Learning Disabilities	5 lowest achieving from 1	by Washington state definition	High school	3 M, 2 F?	US (Seattle, WA)	Grade 9
Higgins K, Boone R, Lovitt TC	1996	Journal of Learning Disabilities	13 LD, 12 remedial	“according to school district guidelines”	High school	19 M, 6 F	US (WA)	Grade 9
Kennedy MJ, Newman Thomas C, Meyer JP, Alves KD, Lloyd JW	2014	Journal of Learning Disabilities	32 SWD (27 LD, 3 behavioral, 2 MR), 109 students without disailities	“All SWD in this study had an Individualized Education Plan (IEP) on file with the school and received special education services”	High school	76% of 32 M (?)	US	Grade 10
Kennedy MJ, Deshler DD, Lloyd JW	2015	Journal of Learning Disabilities	249 NLD, 30 LD	“had an IEP stemming from a diagnosis of specific learning disability related to reading,“	High school	24 M, 6 F	US	Grade 9 – 12
Satsangi R, Bouck EC	2015	Learning Disability Quarterly	3”	diagnosed with a learning disability in mathematics”	High school	3 M	US (Midwest)	14, 16, 18
Satsangi R, Bouck EC, Taber-Doughty T, Bofferding L, Roberts CA	2016	Learning Disability Quarterly	3	”identified with a learning disability in mathematics”	High school	3 M	US (Midwest)	17, 18, 19
Straub C, Vasquez E III	2015	Journal of Special Education Technology	4	”prior diagnoses with an LD that affected their language skills”	High school	3 M, 1 F	US (Florida)	13–16

Table 14.

Multimedia publications – design and quality

Authors	Year	Intervention	Design	Outcome measure	Study groups	Rating
Higgins K, Boone R	1990	Computer: Hypertext study guides	Group design	Quiz scores	Lecture, Hypertext guide, Lecture + Guide / LD, Remedial, Regular Ed	18
Higgins K, Boone R	1990	Computer: Hypertext study guides	Single subject	Quiz scores	N/A	18
Higgins K, Boone R, Lovitt TC	1996	Computer: Hypertext study guides	Group design	Quiz scores	Lecture, Hypertext guide, Lecture + Guide / LD, Remedial	20
Kennedy MJ, Newman Thomas C, Meyer JP, Alves KD, Lloyd JW	2014	Computer: multimedia-based content acquisition podcasts	Group design	Researcher-created test	Vocabulary podcasts, no intervention / Students with / without disabilities (84% LD)	18
Kennedy MJ, Deshler DD, Lloyd JW	2015	Computer: multimedia-based content acquisition podcasts	Group design	Researcher-created test	4 different vocabulary learning conditions / LD, no LD	20
Satsangi R, Bouck EC	2015	Virtual manipulatives to teach area and perimeter	Single subject	Percentage of correctly solved area and perimeter problems	N/A	21
Satsangi R, Bouck EC, Taber-Doughty T, Bofferding L, Roberts CA	2016	Virtual and concrete manipulatives to teach single-variable linear equations	Single subject	Percentage of correctly solved equations	N/A	21
Straub C, Vasquez E III	2015	Synchronous online collaborative writing software	Single subject	Holistic quality of writing score	N/A	20

Interventions took disparate forms in this category. Quite a few studies used multimedia presentations, but even these were designed differently. In the Higgins studies (Higgins & Boone, 1990; Higgins et al., 1996), the multimedia presentations were interactive ”hypertext study guides”, but in the Kennedy studies (Kennedy et al., 2014, 2015), they were non-interactive ”content acquisition podcasts”; essentially slide-based presentations. But these studies were otherwise relatively homogenous: they all included high school students, outcome variables were similar to or identical with preexistent school testing, and even the subject material was similar (history lessons).

Two interventions by another research group, Satsangi & Bouck (2015) and Satsangi et al. (2016) took an entirely different approach. Both studies examined whether students with a mathematics disability could benefit from learning about geometry using virtual, computer-based manipulatives - both compared to no intervention and to physical manipulatives (the physical manipulatives were slightly more effective.) A further, unique intervention (Straub & Vasquez III, 2015) examined whether synchronous online collaborative learning could help students with LD in learning writing strategies.

Multiple publications also investigated the performance of remedial education students as distinct both from students with LD and typical development: the two studies reported in Higgins & Boone (1990), and Higgins et al. (1996). We did not consider data from remedial education students.

Although these studies were thematically similar, their interventions were too different to include in one meta-analysis. Therefore we only opted to produce a table of effect sizes, as seen in Figure 15. Effects tended to be strongly positive where they were possible to calculate; in some case we could not calculate them, usually due to too few or no outcome measurements in a single-subject trial. Some of the very large positive effects were due to participants not being able to perform the experimental task at all before the intervention, and able to perform perfectly after the intervention.

The only article we could not include in the table of effect sizes was Kennedy et al. (2015), because it did not feature a no-intervention condition, therefore we describe it separately. This study compared ”content acquisition podcasts” (similar to Powerpoint presentations) produced with different methods. There were four kinds of podcasts: explicit instruction only (with adherence to Mayer’s Cognitive Theory of Multimedia Learning), keyword mnemonics only, explicit + keyword, and explicit instruction (with no adherence to Mayer’s Cognitive Theory of Multimedia Learning). Effect sizes were provided with Cohen’s d. In a 4 × 2 split-plot, fixed-factor repeated measures ANOVA, group or time effects were not significant, but the group × time interaction was significant at p < 0.001. Post hoc pairwise comparisons showed that students with LD in the explicit + keyword group had significantly higher scores than students with LD in the non-Mayer group, even after Bonferroni correction (Cohen’s d = 1.97). Students with LD in the explicit + keyword group scored higher than students with LD in the explicit only and keyword only groups, but these results did not reach significance after correction (Cohen’s d = 1.09 and 1.40, respectively.)

4.3.6. Smart pens

Smart pens are handheld devices with built-in scanning and character recognition features. Users can scan individual words or lines of text with the pen, and the device can provide speech synthesis, dictionary definitions, translations or syllabification, depending on model and make. Smart pens are mostly used as AT for people with dyslexia, or for typically developing learners of a foreign language.

Table 16 provides demographic and other background information on the publications, and Table 17 features methodological information and the results of the quality assessment.

Table 16.

Smart pen publications – demographic and other background information

Authors	Year	Journal	n	Diagnosis	School	Gender	Location	Age
Belson SI, Hartmann D, Sherman J	2013	Journal of Special Education Technology	10	“language-based learning disabilities, attention deficit hyperactivity disorder (ADHD), visual and spatial disorders, and other specific learning disabilities”	Special high school	4 M, 6 F	US (“Mid Atlantic”)	14 – 18
Higgins EL, Raskind MH	2005	Journal of Special Education Technology	30	“previously identified as having a LD” + as having a severe reading disability by scoring 2 years or more below expected grade level in reading comprehension as measured by the Woodcock Johnson Reading Mastery, passage comprehension subtest	Special high school	20 M, 10 F	US(California)	10 – 18
Johnson I	2008	Kairaranga	4	“already participating in reading remediation programmes“	High school	3 M, 1 F	New Zealand	10 – 15
Schmitt AJ, McCallum E, Hennessey J, Lovelace T, Hawkins RO	2012	Assistive Technology	3	“recognized by the school’s disability concerns office as being students with reading disabilities“	University	2 M, 1 F	US (Mid-Atlantic)	20 – 21

Table 17.

Smart pen publications – design and quality

Authors	Year	Intervention	Design	Outcome measure	Study groups	Rating
Belson SI, Hartmann D, Sherman J	2013	Smart pen: LiveScribe Echo + notetaking strategies	Group design	Organization, Content, Selectivity, Potential of notes	LD students with and without pen (pre/post)	15
Higgins EL, Raskind MH	2005	Smart pen: Quicktionary Reading Pen II	Group design	Formal Reading Inventory(comprehension) Standard / Raw scores	LD students with and without pen	19
Johnson I	2008	Smart pen: Oxford Reading Pen	Group design	Reading accuracy and comprehension scores	N/A	14
Schmitt AJ, McCallum E, Hennessey J, Lovelace T, Hawkins RO	2012	Smart pen: Readingpen Advanced Edition	Single subject	Number correct, comprehension rate	LD students with and without pen (pre/post)	22

Most studies of smart pen interventions included some kind of reading comprehension measure as their outcome variable, with the exception of Belson et al. (2013) focusing on the quality of notes that the students produced. Therefore we opted for reading comprehension as our outcome variable. Two studies used group designs: Higgins & Raskind (2005), Johnson (2008); and one study used a single-subject design: Schmitt et al. (2012). As this latter study had no A-B-A phases, we calculated effect sizes not using the method in Beeson & Robey (2006), but rather by recording the individual data points from the graph, and producing their means and standard deviations. The effect sizes we gained this way were similar to those provided by the authors.

Belson et al. (2013) used a different outcome measure than the above publications: the quality of notes taken by high school students with LD. The content and selectivity of notes significantly improved in response to using the Livescribe Echo pen combined with notetaking instruction, with a difference of 0.56 and 0.64 on a scale of 1–5 (p = 0.0499 and 0.0209, respectively). Unfortunately, notes were rated by non-blinded observers.

Figure 18 shows our forest plot and meta-analysis. The combined effect size (Hedges’ g) of the studies was 0.449, quite small, but significant at p = 0.029.

In this subset of studies, more research is likewise warranted, but results so far allow us to be optimistic. Again not everyone responds to this kind of intervention favorably, but the overall effect is positive.

4.3.7. Other computer-based interventions

These interventions are too diverse to divide into further subgroups. Due to the heterogeneity of the research, no meaningful quantitative summation is possible; therefore we will discuss each study separately. Table 19 presents demographic and other background information of these studies, while Table 20 provides information on study design and quality.

Table 19.

Miscellaneous publications – demographic and other background information

Authors	Year	Journal	n	Diagnosis	School	Gender	Location	Age
Berninger VW, Nagy W, Tanimoto S, Thompson R, Abbott RD	2015	Computers & Education	35	Dysgraphia, dyslexia, OWL LD/SLI	High school	80% M	US(Washington)	10y4m – 14y9m
Inman Anderson L, Knox Quinn C, Horney MA	1996	Journal of Learning Disabilities	32	MS, High school LD students “using the eligibility criteria established by the state of Oregon”	High school	23 M, 9 F	US (Oregon)	12.2 – 16.8
Lin P-Y, Lin Y-C	2016	Research in Developmental Disabilities	12690 LD, 208289 total sample	”students with individual education programs (IEPs) and/or formally identified by the Identification, Placement, and Review Committee in Ontario”	High school	N/A	Canada (Ontario)	Grade 10
Okolo CM, Hinsey M, Yousefian B	1990	Learning Disabilities Research	18	“school-identified LD”	High school	10 M, 8 F	US	High school

Table 20.

Miscellaneous publications – design and quality

Authors	Year	Intervention	Design	Outcome measure	Study groups	Rating
Berninger VW, Nagy W, Tanimoto S, Thompson R, Abbott RD	2015	Computer: iPad based writing instruction	Group design	Multiple normed writing performance tests	Pre/post	19
Inman Anderson L, Knox Quinn C, Horney MA	1996	Computer: Complex laptop study strategies	Group design	WRMT-R, DATA-2, Study Skills Test, LASSI-High school, Writing speed	Post hoc grouping	13
Lin P-Y, Lin Y-C	2016	”Assistive technology” and/or ”Computer”	Group design	OSSLT literacy test performance	TD, LD, emotional or behavioral disorders, multiple disabilities	17
Okolo CM, Hinsey M, Yousefian B	1990	Computer: Typing games	Group design	Typing speed and accuracy	Drill-and- practice and game learning conditions	14

Okolo et al. (1990) examined which formats of keyboarding instruction are most helpful for students with LD. Good keyboarding skills are a prerequisite of using many of the above forms of AT, but students with LD can also struggle with learning to type. Two keyboard teaching interventions were used in the study: a more conventional drill, and game-based learning. Students’ typing speed increased in both conditions (from 5.44 to 8.25 wpm in the drill, and from 6.45 to 8.60 in the game, at p < 0.001), although their typing accuracy did not change significantly. Their attitudes toward computers also became more positive. There was no significant performance difference between the two interventions. However, students in the game-based learning condition completed fewer training sessions on their own after the intervention had concluded.

Anderson et al. (1996) taught complex laptop-based study strategies to high school students with LD. Students were sorted into three post hoc groups based on their technology adoption: power users, prompted users and reluctant users - this related both to the frequency of their technology use and their attitudes toward it. Using one-way ANOVA, lower IQ scores on verbal, performance and full-scale measures were associated with lower levels of adoption at p < 0.001. There was also a difference in some of the recorded measures of literacy and skill test scores, usually disfavoring reluctant users, but no difference in others. The post hoc grouping seemed less suited to interpreting the results than for example, correlation-based reporting would have been, so these data were not straightforward to evaluate.

Berninger et al. (2015) investigated tablet-based writing instruction for students with LD. The intervention used researcher-designed interventions and measured outcomes with a variety of standardized tests like the Clinical Evaluation of Language Fundamentals-4 conducted pre- and post-intervention. Effect sizes (Cohen’s f²) were reported ranging from 0.17 to 0.40 (p. 9), in the medium to large range. An analysis of individual students by their specific LD also showed that most, but not all of them responded to instruction aimed at their specific areas of concern. These results were quite favorable, but the intervention itself was quite sparsely described, and only named in figure captions, as ”HAWK (Help Assistance for Writing Knowledge)” (p. 6–7).

Lin & Lin (2016) was a study with an epidemiological approach. All grade 10 students taking the Ontario Secondary School Literacy Test in a year (n = 208,289) were subdivided post hoc into groups based on whether they had learning disabilities, ”emotional or behavioral exceptionalities” or ”multiple exceptionalities”, or none of these conditions. Then the researchers assessed whether accommodations they received on the test resulted in better performance. Many of the accommodations were outside our scope (e.g., extended time), but both ”computer” and ”assistive technology” accommodations were listed. (The further ”scribe” category combined both STT software and human scribes, so we could not make use of it.) As the precise nature of these accommodations was not further explained, we could not sort this study into our above categories. Computer-based and other AT accommodations were helpful in LD, with the accommodation combinations most likely to be positively related to performance being computer + setting-based accommodations (e.g., quiet room) and computer + extended time. Overall the accommodation combination with the highest rate of success was also the most resource-intensive: scribe + setting-based, followed by the two computer-based accommodations we mentioned. The researchers reported multiple methods of calculating odds ratios and also used the data for a methodological discussion on effect sizes.

4.4. Summary of the experimental and quasi-experimental intervention studies

We located studies of various kinds of AT. Whereas to our knowledge, there has been no recent research on how frequently students with LD use these kinds of AT, we found experimental or quasi-experimental studies related to five different types of AT: text-to-speech systems, speech-to-text systems, word processing interventions, multimedia interventions, smart pens; and some miscellaneous studies using computers and tablets that did not fit into these five categories.

The numerous text-to-speech studies demonstrated a small overall effect, likely due not to the lack of effectiveness of the technology itself, but because of an interaction with baseline reading ability. The fact that some students found this intervention helpful, but for others it was not helpful or even detrimental, obscured the effect in positive responders. Some studies found that students with lower ability (Elkind et al., 1996; Higgins & Raskind, 1997) or younger students (Lewis, 1998b) benefited more from this technology, but there was also a contrary result from Calhoon et al. (2000) where better initial performance also predicted better response to the intervention. We would need more information on the nature of this interaction to make a recommendation. Students with LD might try this accommodation, but the students and the university disability services personnel who work with them are likely to find it helpful for some but not others.

Speech-to-text studies were fewer in number and overall quite heterogenous, despite using the same software, Dragon. There was not enough information for us to produce a meta-analysis. Studies did tend to present positive effects, so we would recommend more investigation of this type of AT, and making it available to students.

Word processing interventions focused on different features of word processors (for example, spell check or grammar check). Word processing seemed to lead to better writing outcomes, often with large effect sizes. We located four studies using a comparable outcome metric, error rates, and found a large positive effect in our meta-analysis. But this result unfortunately has limited usefulness: first, because the meta-analysis only included a small number of studies; and second, because the research was conducted in the 1990s. Since then, word processing aids have become extremely commonplace in higher education, to the extent that they might not be seen as AT accommodations anymore. Nevertheless, these supports are clearly helpful and, thus, should be considered when planning accommodations for students with LD. They can also be targets for further study, and application development, to determine exactly which aspects are helpful.

We further recommend that modern word processors with all their features enabled serve as a control condition in future studies, as a baseline against which other AT interventions are measured – similar to the ”treatment gold standard” in medical intervention trials. Lange et al. (2006) was the only study we could locate that used a similar control.

Multimedia and hypertext interventions had a strong positive effect, across all subtypes of these very varied interventions. These presentations of subject material seemed to lead to better learning. These types of presentations are also becoming common in higher education, and making headway in secondary school settings, so what needs to be researched is not whether multimedia is helpful at all, as in practice, its use is a given. Rather, what we need to know is which kinds of presentations we should use. One recent study, Kennedy et al. (2015), did compare different kinds of multimedia presentations, and another (Satsangi et al., 2016) compared virtual and physical presentations of the same objects.

Smart pens seemed to have a small, but significant effect on reading comprehension. As in other computer-reading interventions (text-to-speech above), some students responded unfavorably to this kind of AT, but the overall effect was positive. In addition to more research, we recommend that smart pens be added to the repertoire of possible secondary and postsecondary education accommodations for LD.

Even though our meta-analyses could be best described as tentative due to the small number of studies, we believe they do offer one additional considerable benefit. By assembling comparable studies and choosing outcome variables that are most frequent in the literature, we provided a framework for future studies. If outcome variables already present in previous studies are chosen for new projects, the results will fit into preexistent data, enabling researchers to produce cumulative metaanalyses at later points.

5. Conclusions

Our study was the first to formally survey and evaluate AT interventions for adolescents and adults with LD. We located a sizable body of research that has not been previously evaluated in this manner, and could draw both quantitative and qualitative conclusions, in addition to providing future directions.

Word processing-based AT interventions had a large positive effect on writing error rates. Text-to-speech systems had a small positive effect on reading comprehension, with some evidence that an interaction with baseline reading ability was obscuring a larger effect. Smart pens also had a small positive effect. The use of speech-to-text systems led generally to positive outcomes, but outcome variables were too different for meta-analysis. In the case of multimedia / hypertext interventions, effects were very strongly positive, but the interventions themselves were too disparate to perform a meta-analysis.

Table 21 shows a brief summary of these results. Where we could not perform a meta-analysis, we provide only the range. (Note: speech to text effect sizes are reported by the original authors in various formats; as we could not find comparable outcome variables in that case, we did not standardize effect sizes.)

Table 21.

Summary of effects

Intervention type	Outcome	Hedges’ g (p)	Interpretation	Evidence basis
Text to speech	Reading comprehension	0.445 (0.06)	Moderate positive effect	Meta-analysis
Speech to text	Various	0.42 to 1.125 (N/A)	Moderate to large positive effect	Systematic review
Word processing	Error rate change	1.626 (0.002)	Large positive effect	Meta-analysis
Multimedia	Various	0.376 to 27.800 (N/A)	Small to large positive effect	Systematic review
Smart pens	Reading comprehension	0.449 (0.029)	Moderate positive effect	Meta-analysis

Convergent results from both quantitative and qualitative data demonstrated that AT supports can be effective, but they need to be customized to the person. Some forms of AT could be unhelpful or harmful for some participants; for example, text-to-speech systems tended to hinder students who had relatively high baseline reading ability. Negative emotions were predominantly connected to frustration with the technical aspects of specific AT solutions, though a minority of participants also reported social stigmatization. Often, students did not receive sufficient support from their educational disability services personnel, especially with the technical aspects of AT. Many students who reported success with AT used systems they had set up at their own time and cost.

We can conclude that AT use is helpful to adolescents and adults with LD in multiple ways: it can produce better quantitative educational outcomes, and it can also lead to increased satisfaction with learning and improved quality of life. But to reach these goals, certain conditions need to be met. The systems need to be customized to the individual, and educational professionals should not expect that one-size-fits-all solutions will be suitable for everyone in the LD population. The largest drawbacks of AT are abandonment due to technical issues, and the lack of suitability of specific interventions to specific students. These drawbacks can both be mitigated with increased institutional support, and we also found evidence that support has indeed been increasing over the past decades.

6. Limitations

A systematic review can only draw conclusions from the available publications. Many assistive devices or methods had little to no effectiveness testing in the preexistent literature-The interventions were quite disparate even when the AT systems tested were similar, and effectiveness was often measured using metrics that were not comparable. This meant that we were not able to include several of the available studies in the meta-analyses.

Overall, research was often not cumulative: authors either did not refer to other scholarly efforts, or deliberately chose different paradigms and designs for their own projects. Many of the articles were published in low or no impact factor journals and venues with little visibility. Several relevant papers were published in defunct journals and/or journals without DOI; thus it is not surprising that other scholars were less aware of them.

It is imperative to produce quantitative summations of AT effectiveness in adolescent and adult samples, as an evidence-based approach can be used to argue for higher-quality supports in secondary and postsecondary education, and also in the workplace. Unfortunately, at present, data primarily provide evidence in favor of AT supports that have already been widely acknowledged by the public to be effective (for example, word processing).

An especially large gap remains related to individualized supports. Although it is apparent that some people with LD do not benefit from the same kinds of AT that are effective for others, we know little about which factors influence AT effectiveness. Both qualitative and quantitative studies have tried to address this: for instance, Elkind et al. (1996), Higgins & Raskind (1997), Lewis (1998b), Calhoon et al. (2000), Roberts & Stodden (2005), but so far there have been few of them, and results are sometimes conflicting.

Another gap relates to the development of new technologies. Many kinds of AT currently in use have been in development for decades, and thus it is relatively easy to generalize from earlier studies to current technology - especially considering that the effects we found tended to be positive overall. If even the more rudimentary forms of AT tested in older articles were usually found to be effective, then all the more so for more advanced forms of the same technology. For instance, speech synthesis in TTS systems has progressed from monotoneous, robotic computer voices to natural-like speech in the present day.

By contrast, there are very few technologies that are currently in use and are entirely new. Tablets are probably the devices that differ the most from AT widely available a decade or more ago. Tablets are handheld computers with a touch interface. Research about tablets currently mostly focuses on typical development (for a systematic review, see Haßler et al. (2016) with 23 studies) or other conditions like autism (Lorah et al. (2015) with 17 studies). We managed to locate very few studies using tablet apps for adolescents and adults with LD, and sometimes even those apps were used for purposes already described in the previous literature - like Fälth & Svensson (2015) featuring a text-to-speech app.

It is unknown how likely are students with LD to use tablets for academic purposes outside a research context - in our mixed methods study in preparation, very few young adult college students with language-related LD reported using tablets for studying, while most of them reported using laptops. Mobile phones were mostly mentioned in the context of scheduling (Google Calendar, etc.) and only infrequently as a study aid (e.g., as a spell checker or dictionary).

Unfortunately, tablets and other mobile devices running apps have not yet had substantive effectiveness testing in our age and diagnostic groups. Many tablet studies with atypically developing groups focus on children, and LD research in general has a similar age bias (Conti-Ramsden & Durkin, 2016). When these devices run software that fulfils a similar function as preexistent AT software, it is not a stretch to assume that effectiveness can be similar, but it is important to know that this has seldom been empirically tested.

It is a possible major criticism that the data reported above often stem from older studies. Should earlier studies be incorporated into meta-analyses when technology is rapidly developing?

Including older data is common in meta-analyses. Patsopoulos & Ioannidis (2009) empirically examined the issue of older studies in meta-analyses of healthcare interventions. They gathered a random Cochrane sample of 157 meta-analyses, and found that only a quarter of the studies included in these meta-analyses were published in the last five years of the literature search period. Only 8% of all reviews discussed the age of studies as a methodological concern. In 82.1% of studies, post hoc excluding studies older than 10 years did not change the p-value of the effect; in 10.1% of studies, the effect lost significance and in 7.8%, it gained significance. The authors could not compare effect size changes between older and newer studies, because usually the more recent publications were so few that such a comparison was underpowered. According to Dechartres et al. (2016)’s meta-meta-analysis of which variables affect meta-analysis treatment effect estimates, study age was one of the very few variables which did not seem to influence results.

Patsopoulos & Ioannidis (2009) concluded that ”The amount of data, regardless of year of publication, is limited for most health care topics […] and we do not have the luxury of discarding trials simply because of their calendar year.” They also pointed out that excluding older studies can be a form of selective reporting that introduces positive bias. Further, they tentatively noted that when a comparison is possible, older studies tend to have slightly larger effect sizes. In our meta-analyses, technology has improved considerably and therefore we would expect newer studies to have larger effect sizes, so these biases are likely to cancel each other out.

Therefore we opted not to exclude older data, with the caveat that more research would definitely be welcome. By gathering all available studies, our review hopefully also serves to point out the gaps in literature and inspire more AT intervention research.

Despite these limitations, we successfully conducted meta-analyses and draw conclusions from both quantitative and qualitative studies. The quality of these studies was also similar to that of studies on other types of interventions (Justice et al., 2008).

7. Future directions

This summary of the literature on the effectiveness of AT interventions for adolescents and adults with LD reveals a number of gaps. Methodologically, there is little quantitative survey-based research, research that could document type, frequency, purpose, and satisfaction with AT use. Outcome measures should be carefully considered in all types of research designs. The more we, as a field, can select common outcomes, the more we will enable meta-analytic conclusions. Gaps in the types of questions we ask are also worth considering. For example, questions concerning the effectiveness of tablets or contemporary word processing are rare, despite the ubiquity of these products in high school and post-secondary classrooms. Finally, it would be useful to ask how the utility of AT accommodations vary depending on the purpose of the intervention: to compensate for a disability, to scaffold coursework, or to aid skill development. This issue bears further investigation.

Table 12.

Forest plot of error rate effects

Table 15.

Testing score changes in response to multimedia interventions

	Hedges’ g	Standard error	Lower limit	Upper limit	Interpretation
Higgins 1996 lecture vs combined	0.81	0.11	0.59	1.02	Large positive effect
Kennedy 2014 RR CAP vs no CAP	1.82	4.50	−6.99	10.63	Large positive effect
Kennedy 2014 EE CAP vs no CAP	1.30	4.77	−8.06	10.65	Large positive effect
Higgins 1990 Case 1	0.38	14.14	−27.36	28.09	Small positive effect
Higgins 1990 Case 3	2.39	13.75	−24.56	29.34	Large positive effect
Satsangi 2015 Case 2 area	6.67	1.72	3.29	10.05	Large positive effect
Satsangi 2015 Case 3 area	1.74	0.85	0.07	3.41	Large positive effect
Satsangi 2015 Case 2 perimeter	6.71	1.73	3.32	10.11	Large positive effect
Satsangi 2015 Case 3 perimeter	11.33	2.92	5.60	17.06	Large positive effect
Satsangi 2016 Case 1	5.13	1.31	2.56	7.70	Large positive effect
Satsangi 2016 Case 2	22.28	5.02	12.44	32.13	Large positive effect
Satsangi 2016 Case 3	27.80	6.25	15.55	40.05	Large positive effect
Straub 2015 Case 1	3.67	1.23	1.26	6.07	Large positive effect

Table 18.

Forest plot and metaanalysis for smart pen studies

Highlights.

• ✧Assistive technology can be useful for adolescents and adults with LD
• ✧Word processing, multimedia and hypertext interventions were the most effective
• ✧Speech-to-text interventions had a small effect
• ✧Smart pens and text-to-speech systems had mixed results
• ✧Interactions with baseline ability or age can obscure intervention success