Artificial Intelligence in Communication Sciences and Disorders: Introduction to the Forum

The excitement around artificial intelligence (AI) in health care and education is palpable. In speech-language pathology, a wide range of AI-driven tools are being developed to enhance the efficiency, accessibility, and effectiveness of patient care (Deka et al., 2024; Suh et al., 2024). These tools include virtual therapists (e.g., Kim et al., 2024; Richter et al., 2023); interactive games (e.g., Ganzeboom et al., 2022); chatbot conversational partners (e.g., Belda-Medina & Calvo-Ferrer, 2022); personalized therapy systems (e.g., Kim et al., 2024); intelligent assistants (e.g., Santa Barletta et al., 2024); and AI-driven diagnostics to assess speech, language, and swallowing (e.g., Brahmi et al., 2024; Jauk et al., 2023; Neumann et al., 2024; Privitera et al., 2024). The efficiency provided by these systems, many of which include automation, will be particularly advantageous in light of the increasing demand for speech-language pathology services (U.S. Bureau of Labor Statistics, 2023) and the persistent nationwide shortage of speech-language pathologists (Farquharson et al., 2022). Some of the most anticipated AI-driven technological breakthroughs include advancements in communication-assistive devices, such as brain–computer interfaces (e.g., Card et al., 2024), personalized automatic speech recognition (ASR; e.g., Green et al., 2021), voice cloning (e.g., Anand & Reji, 2024), and eye-tracking technology (e.g., Peters et al., 2024). These advancements are expected to significantly enhance the ability of individuals with communication disorders, including those with even very severe speech impairments, to interact more effectively with both people and technology.

Despite the growing interest in AI-driven clinical tools, the technology is still in its early stages, with much of its promise yet to be realized. Researchers in communication sciences and disorders (CSD) and other disciplines are now diligently working to evaluate the accuracy, clinical utility, limitations, and potential risks of these emerging technologies (Al-Ali et al., 2024; Privitera et al., 2024; see Google, n.d., for a broad discussion of responsible AI practices). One significant challenge at the time of writing is the lack of large, diverse data sets (see Hasegawa-Johnson et al., 2024). This limitation restricts AI's ability to learn effectively and generalize across various impairments (Abràmoff et al., 2023). Furthermore, concerns about data privacy and the security of patient information add complexity to research efforts and hinder the seamless integration of AI into clinical practice (Zhang & Zhang, 2023). This dynamic research landscape necessitates the development of new paradigms, including novel methodologies for validating AI systems, frameworks for ethical AI deployment, and innovative interdisciplinary collaboration among practitioners, clinical scientists, speech and language scientists, computer scientists, bioengineers, data privacy specialists, and user experience experts.

The 2023 ASHA Research Symposium, co-sponsored ASHA and the National Institutes of Health's National Institute on Deafness and Other Communication Disorders (NIH-NIDCD), was held on November 18, 2023, in Boston. The day-long conference, titled “Artificial Intelligence in Communication Sciences and Disorders (CSD),” fittingly took place in 2023—a landmark year when commercial-grade technologies, such as large language models (LLMs) and other generative AI tools, became widely available and swiftly achieved mainstream adoption. The symposium brought together experts from diverse fields to share their insights on how AI applications are driving the next generation of speech analytic and speech recognition tools.

The day's sessions were organized into four thematic areas:

1. Speech Disorder Research in the Big-Data Era: Large-scale databases and speech analytics

2. Advances in Speech Biomarkers: Monitoring neurological and mental health

3. Personalized Speech Recognition and Vocal Synthesis: Advancing clinical care for individuals with speech impairment

4. Updates on Brain–Computer Interfaces: Non-invasive and invasive approaches

The symposium featured nine internationally recognized scholars, including Philip Nelson (Google), Mark Hasegawa-Johnson (University of Illinois at Urbana-Champaign), Julie Liss and Visar Berisha (Arizona State University), Vikram Ramanarayanan (Modality.AI, Inc), Emily Provost (University of Michigan), Richard Cave (MND Association), Rupal Patel (Northeastern University), Leigh Hochberg (Mass General Research Institute/Harvard Medical School), and Jun Wang (University of Texas at Austin). Their presentations can be viewed at https://www.asha.org/research/researchsymposium/2023-research-symposium/ (ASHA, 2023). The invited speakers, each with over a decade of experience in AI utilization, are widely recognized experts who are uniquely qualified to discuss real-world AI applications in CSD.

The event commenced with a keynote address by Philip Nelson, a director of engineering at Google Research and leader of Project Euphonia, an initiative focused on improving accessibility for individuals with communication impairments. With a career spanning several decades, Philip has been at the forefront of utilizing AI and advanced technologies to address critical societal and healthcare challenges. His talk, “Breaking Barriers With AI: Google's Programs for Advancing Accessibility, Communication, and Social Inclusion,” set the stage for the day-long symposium.

This forum of the Journal of Speech, Language, and Hearing Research (JSLHR) includes the following articles co-authored by seven presenters.

The article by Hasegawa-Johnson et al. (2024), titled “Community-Supported Shared Infrastructure in Support of Speech Accessibility,” addresses one of the most significant obstacles to developing robust AI-driven assistive technologies for individuals with speech disabilities: the scarcity of disordered speech samples for model training. Their article introduces the Speech Accessibility Project, an innovative and large-scale initiative fostering collaboration among academia, industry, and patient advocacy groups to enhance speech technology accessibility. This project aimed to create a vast, diverse, and openly shared speech corpus, including recordings from approximately 2,000 participants with various neurological conditions affecting speech. The authors contextualize the initiative historically and emphasize the right to equal access, underscoring the essential role of community involvement and resource sharing in creating rare yet highly impactful speech corpora. Additionally, the article outlines the guiding principles for constructing disordered speech corpora and thoroughly examines the logistical considerations involved in their annotation processes.

The article “Automatic Speech Recognition of Conversational Speech in Individuals With Disordered Speech” by Tobin et al. (2024) addresses the critical need for ASR systems that can be effectively used in real-time conversations by individuals with speech disabilities, including dysarthria. Despite these advancements, commercial ASR systems often struggle to accurately recognize unscripted speech produced by individuals with atypical speech patterns. Therefore, enhancing conversational ASR could be a transformative step toward improving social participation for these individuals. The authors studied the speech recognition performance disparity between disordered speech scripted (read aloud) or produced in conversation. They identified several factors contributing to the degraded performance of conversational ASR, including those related to the speaker, speech stimuli, and the recording environment. The researchers also explored whether training ASR systems specifically on conversational speech samples could improve performance. This study highlights the urgent need for conversational ASR systems to handle disordered speech effectively and suggests directions for future research and development to pursue this objective.

The article “How People Living With Amyotrophic Lateral Sclerosis Use Personalized Automatic Speech Recognition Technology to Support Communication” by Cave (2024) identifies several challenges developers need to overcome to advance ASR communication tools for those with speech impairments, as evidenced by detailed case studies. His study suggests that laboratory-based benchmarks for personalized ASR may overestimate their effectiveness in real-world scenarios. The findings also highlight the crucial role of incorporating feedback from all stakeholders in the development process. Although the user-centric evaluation approach advocated by Cave might seem impractical to scale, it is necessary to create more inclusive designs with real-world applications that improve ASR systems designed to enhance real-time social interactions.

The study “FluencyBank Timestamped: An Updated Data Set for Disfluency Detection and Automatic Intended Speech Recognition” by Romana et al. (2024) focuses on enhancing speech recognition and identifying disfluencies in stuttered speech. People who stutter often encounter difficulties with ASR systems, which frequently misrecognize or reject their speech. This significantly compromises the accessibility and usability of these systems for people who stutter. The insufficient performance of ASR systems in processing disfluent speech is partly due to the limited availability of diverse and complex data sets that accurately represent stuttering for model training. To address this pressing need, the authors developed the FluencyBank Timestamped data set, an enhanced version of the original FluencyBank dataset (Bernstein Ratner & MacWhinney, 2018). This new resource comprises transcripts from 37 speakers, featuring aligned audio and detailed labels that pinpoint the type and location of disfluencies. The study establishes benchmarks for speech recognition and disfluency detection, underscoring substantial gaps and areas for improvement. The study is notable for its rigorous evaluation pipeline, which positions the FluencyBank Timestamped data set as an invaluable resource for enhancing the robustness of disfluent speech processing in a range of critical applications.

The study “Neural Decoding of Spontaneous Overt and Intended Speech” by Dash et al. (2024) explores the potential of decoding spontaneous and intended speech directly from brain signals that were recorded using magnetoencephalography (MEG). The research team collected MEG signals from participants spontaneously producing the words “no” and “yes.” The researchers utilized advanced machine learning models to decode these words from recorded signals corresponding to segments associated with either overt speaking or just the intention to speak. Although the study focused on predicting only two words, it demonstrates the feasibility of non-invasive neural speech decoding during spontaneous speech—a critical advancement toward developing brain–computer interfaces for individuals with speech impairments.

In the article “Operationalizing Clinical Speech Analytics: Moving From Features to Measures for Real-World Clinical Impact,” Liss and Berisha (2024) present insights from their extensive research on developing and validating digital speech analytics for clinical use. Central to their work is the premise that speech patterns hold significant information about a speaker's neurological health as well as the potential barriers to communication participation and social engagement. This article outlines the challenges of traditional machine learning models in clinical settings and advocates the use of clinically validated speech measures, rather than relying solely on high-dimensional feature representations from standard acoustic speech extraction algorithms. This article presents a case study of the development of a speech measure for articulatory precision in amyotrophic lateral sclerosis (ALS). The process from ideation to Food and Drug Administration (FDA) breakthrough status designation is detailed, highlighting the importance of moving from theoretical constructs to clinically validated, objective measures. This review encourages future research to explore the generalizability of this approach across different clinical populations, noting that, while ALS is a suitable starting point, other conditions may present unique challenges.

The review article “Multimodal Technologies for Remote Assessment of Neurological and Mental Health” by Ramanarayanan (2024) introduces the Modality platform, a patient-driven system for monitoring speech and other domains, facilitating mental and neurological health assessment. This system employs a virtual agent utilizing chatbot technology to deliver standardized test instructions and elicit conversational speech samples from patients. The platform is designed to integrate information from various modalities such as speech acoustics, natural language processing, orofacial and limb movement, eye gaze, respiration, and skin conduction. The author emphasizes the role of speech as a vital sign and biomarker in medical applications, particularly for assessing conditions such as ALS, Parkinson's disease, and schizophrenia. This comprehensive platform was designed to meet the growing demand for reliable, accurate, low-burden, at-home speech monitoring solutions that can be applied across a broad spectrum of neurological and mental health disorders. The author also addresses the challenges associated with deploying multimodal technologies in real-world settings, such as the need for robustness against diverse conditions, privacy concerns, and generalizability of models. While acknowledging these challenges, the author presents a compelling case for how multimodal data and advancements in AI can accelerate progress toward personalized precision health, digital clinical trials, and remote health monitoring.

In summary, this JSLHR forum presents an exciting exploration of AI tools for individuals with speech disabilities, highlighting the potential of AI to enhance patient care and outcomes. However, as many of the authors in this forum emphasize, despite AI's transformative potential, the path to achieving clinical-grade AI, including obtaining FDA approval, is far from seamless and requires addressing complex issues beyond the technical hurdles of model development. The authors collectively identified several substantial challenges: (a) the need for speech data sets that reflect a diverse range of speech disabilities, demographic backgrounds, languages, and dialects across various functional communication contexts; (b) the importance of incorporating clinical expertise and end-user feedback throughout the design process; and (c) the necessity of rigorous, systematic validation efforts to establish their accuracy and clinical utility, particularly in real-world scenarios.

In their book on the clinical implementation of AI, Simon and Aliferis (2024) emphasize the importance of developing AI systems that are both theoretically robust and practically applicable: “Even if a clinical AI system meets or exceeds expert-level performance in the lab, this does NOT mean that (i) the system can be readily adopted into clinical practice, (ii) it will perform similarly when deployed, or (iii) the evaluation metrics used accurately reflect clinically impactful use of the AI model” (p. 528). Equally important to clinical validation is the need to address concerns related to privacy, bias, fairness, security, transparency, and accountability. Tackling these challenges is essential for the ethical and effective deployment of AI in health care (Chen et al., 2023), which requires substantial intellectual and financial investment.

In conclusion, this is an exciting time for CSD practitioners and their patients, as we anticipate AI-driven breakthroughs that could revolutionize our research and practice. These advancements have the potential to significantly enhance the quality of life of millions of people with communication impairment worldwide. While AI offers tremendous transformative possibilities, effectively applying it across the diverse spectrum of communication disabilities will require large-scale, coordinated research efforts that are deeply informed by existing scientific and clinical knowledge as well as the specific needs articulated by clinicians and other users of the technology. As we advance, it is essential to avoid conflating computer algorithms with clinical algorithms, and to balance enthusiasm with skepticism and caution.

Artificial Intelligence Statement

This introduction includes text copyedited by an AI language model. The author has used OpenAI's (2024) GPT-4 model to assist in editing portions of the manuscript. The generated content was subsequently reviewed and revised by the author to ensure accuracy and alignment with the research objectives.

Publisher Note: This article is part of the Forum: Research Symposium on Artificial Intelligence.

Funding Statement

This article stems from the 2023 Research Symposium at ASHA Convention, which was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under Award R13DC003383. Research reported in this publication was partially supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under Award K24 DC016312 to Jordan R. Green. The content is solely the responsibility of the author and does not necessarily represent the official views of the National Institutes of Health.