Diagnosing Apraxia of Speech in Children and Adults When There Is No Gold Standard: A Scoping Review About the Diagnostic Process

Abstract

Purpose:

Childhood apraxia of speech (CAS), stroke-induced apraxia of speech (AOS), and progressive apraxia of speech (PAOS) cannot be diagnosed with an error-free gold standard test or measure. Reference standards are sensible approximations but are vulnerable to bias. In this scoping review, we characterize the diagnostic process for reference standards that have been used in recent diagnostic literature and evaluate the extent to which bias-reducing strategies have been implemented.

Method:

We extracted data about the diagnostic process for studies of speech characteristics in CAS, AOS, and PAOS that were published between 2014 and 2024. Observational studies published in peer-reviewed journals and in English were included, whereas treatment studies, surveys, case studies, and reviews were excluded.

Results:

Of 198 included studies, the most common diagnostic method was consensus or confirmation by two to three diagnosticians (38.4%), followed by checklist verification (25.3%), and simple clinical impression (19.2%). Testing or measurement techniques were used less frequently for diagnostic purposes (9.6%). The diagnostic process varied by etiology, and strategies to reduce bias were minimally implemented across the board.

Conclusions:

There is a need to accept that current diagnostic reference standards are inherently imperfect and to reduce risk of bias by using converging evidence, defining syndrome features operationally, standardizing consensus formation, and estimating reliability of classification. A focus shift to how speech profiles, tests, and measures relate directly to prognosis and treatment response may expedite scientific discovery, as may strategy-sharing among those who work with children, adult stroke survivors, and adults with progressive disease.

Supplemental Material:

https://doi.org/10.23641/asha.31173799

Childhood apraxia of speech (CAS), stroke- or trauma-induced apraxia of speech (AOS), and progressive apraxia of speech (PAOS) are all defined conceptually as disorders that have their origin in faulty speech motor planning or programming (American Speech-Language-Hearing Association [ASHA], 2007; Duffy, 2020; McNeil et al., 2009). For convenience, we refer to them collectively as “speech apraxia syndromes” in this review. Since there is presently no way to verify faulty motor planning or programming neurobiologically (Anastasopoulou et al., 2025; Hickok, 2012; Miller & Guenther, 2021), diagnosticians must instead infer the level of impairment from indirect behavioral correlates. While such inferences may have logical and coherent elements, their validity cannot presently be evaluated due to the lack of an error-free truth about the occurrence of speech motor planning or programming inefficiency. Instead, the three speech apraxia syndromes are designated by constellations of observable features that are defined only in the broadest sense (Haley & Jacks, 2023; Miller & Wambaugh, 2017; Ziegler, 2002).

Considerable research has been devoted to clarifying the speech features that best characterize the CAS, AOS, and PAOS syndromes. It is recognized that the following features co-occur with some regularity and, while not necessarily diagnostic, in one way or another are relevant to impressions about diagnosis: perceptible phonemic errors, consonant and/or vowel distortions, distorted and substituted/added consonants and vowels, increased difficulty with multisyllabic words and consonant clusters, visible groping for articulatory postures, audible articulatory restarts and revisions, attempts to self-correct, difficulty achieving articulatory configurations, disrupted coarticulatory transitions, inappropriate prosody, inconsistent errors, syllable segregation/segmentation, lengthened intersegment durations, intrusive schwa, slow speech rate with increased errors as rate increases, slow and/or off-target diadochokinetic rate, effortful speaking, lengthened segments within words, and equalized stress or errors of stress assignment (ASHA, 2007; Duffy, 2020; Duffy et al., 2023; Jacks & Haley, 2021; Shriberg et al., 2011). None of the features in this extensive, yet nonexhaustive, list is unique to the speech apraxia diagnoses. They are also present to varying degrees in aphasia, dysarthria, stuttering, and developmental phonological disorders. The extent to which subsets of features are useful for differential diagnosis depends on the severity of the disorder; the stage of development, recovery, or progression; degree of individual variability; what diagnoses are to be differentiated (e.g., CAS vs. dysarthria, AOS vs. aphasia with phonemic paraphasia); and how each feature is defined, recognized, and weighted by diagnosticians. The education diagnosticians receive, the colleagues with whom they collaborate, the dialects and accents with which they are familiar, and the type of patients they see all influence what they look for, register, and consider as they develop or confirm auditory impressions about diagnosis. Add to that presumed biological interactions among neural networks for speech and language (Hickok et al., 2011; Levelt et al., 1999; Miller & Guenther, 2021; Tourville & Guenther, 2011) and it should come as no surprise that the problem of equivocal apraxia diagnoses has persisted for decades and generated a robust literature attempting to improve the situation.

In medicine, diagnostic accuracy research is conducted to ensure that new tests are adopted clinically only when functional utility has been demonstrated and characterized (Umemneku Chikere et al., 2019). This is traditionally accomplished by comparing the discrimination qualities of new tests to a “diagnostic truth” that states whether the disease (or condition) is present or absent. The benchmark test that determines this diagnostic truth is known as the “reference standard,” and if the reference standard has been determined to approximate error-free diagnosis, it is further accepted as a mostly objective “gold standard.” Custom statistics, such as sensitivity, specificity, positive or negative predictive values, and area under the receiver operating characteristic curve are calculated by comparing new tests to this gold standard (Carter et al., 2016; Santini et al., 2021). New tests and measures perform differently on diagnostic accuracy metrics, and, therefore, the balance of over- versus underdiagnosis for a given test or measure determines the purpose for which it may be useful to the clinical quest of reducing suffering and improving outcomes (Duffy et al., 2023; Laganaro et al., 2021; Strand & McCauley, 2019).

There are, however, conditions where it is impossible to identify a test or procedure that yields a “true” diagnosis. There may be a significant risk of misclassification, or it may be generally established among experts that no reference standard meets diagnostic needs for the condition (Umemneku Chikere et al., 2019). The medical research community recognizes, for example, that no gold standard exists for conditions such as carpal tunnel syndrome or corticobasal syndrome (Bachmann et al., 2005; Mathew et al., 2012). In these cases, diagnostic accuracy research relies instead on imperfect reference standards while precautionary steps ideally are taken to minimize the bias that is inherent to them (Rutjes et al., 2007).

The speech apraxia syndromes are prime examples of conditions for which a gold standard for diagnosis cannot, at the present time, be identified. The problem is twofold. First, no agreed-upon collection of features defines speech apraxia the same way across clinics and research groups. This issue has been illustrated in previous work (e.g., Allison et al., 2020; Molloy & Jagoe, 2019; Murray et al., 2021) and will not be addressed in the present review. Second, and more importantly for our purposes, there is no agreed-upon process for determining whether the diagnostic features of choice are sufficiently present to warrant a diagnosis in any given person. This latter issue has received far less attention, potentially because it has been assumed that experienced clinicians recognize a behavioral phenotype when they encounter it. This is not a valid rationale. The required observations and interpretations are inherently subjective and vulnerable to bias stemming, in addition to the background elements we mentioned, from expectations about prevalence, occurrence of coexisting traits, physical limitations, communication patterns, and other known and unknown patterns diagnosticians observe in their training and practice (Haley et al., 2012; Molloy & Jagoe, 2019; Murray et al., 2021; Ziegler et al., 2022). Despite occasional claims to the contrary (including previously by our own team; Duffy et al., 2023; Haley et al., 2012; Mailend & Maas, 2020), we conclude that, while it is justified to characterize speech-language pathologists' (SLPs) impressions as the “clinical standard” for diagnosing speech apraxia, these impressions—by definition—cannot also be considered the diagnostic gold standard.

To the point, several studies show that independent and experienced diagnosticians disagree about whether an apraxia diagnosis is appropriate when they consider speech samples of persons with diverse speech sound disorders (Haley et al., 2012; Maas et al., 2015; Murray et al., 2023; Shriberg et al., 2003). Unless they are calibrated to a reference standard (or with each other), it is unavoidable that personal experiences determine what diagnosticians detect and prioritize. They must use thresholds to determine whether individual features are present, decide what to do if the combination of features is equivocal, and refer to a base rate of occurrence to adjust expectations. If this information is not provided, they have no choice but to make their own assumptions, though they may not be aware of doing so. Calibration is an option to establish reliability, but if those involved do not have a history of working together, this can take considerable time and effort. More importantly, the question remains what should be the standard for calibration and whether that standard is acceptable to other research teams, because training for reliability alone does not ensure validity. In the ideal case, the calibration standard is clearly defined and broadly accepted. In other words, it is a strong and nearly error-free reference standard. Thus, we arrive again at the problem of conducting diagnostic research without such a reference point—without a gold standard.

Once we accept that there is no gold standard for diagnosing any of the speech apraxia syndromes, it comes as no surprise that exceedingly few diagnostic accuracy studies have been conducted to improve effectiveness of clinical resources and treatment triaging by evaluating sensitivity, specificity, or predictive value of new tests or measures (Allison et al., 2020; Murray et al., 2021). Instead, diagnostic research about speech apraxia has been concerned with identifying features that are sufficiently stable and measurable to define the CAS, AOS, and PAOS syndromes in the first place. A robust literature remains devoted to identifying diagnostic markers with the anticipation that collections of such markers will one day yield objective, quantitative, and agreed-upon reference standards for diagnosis. In the meantime, to evaluate the utility of potential markers, researchers must form comparison groups of participants who likely do and do not have the target condition, which paradoxically means that the work that strives to develop diagnostic markers must also rely on some type of diagnostic reference standard.

A recent scoping review considered what diagnostic markers had been studied for AOS and CAS so far and characterized the evidence supporting them (Allison et al., 2020). Results showed that progress has been made refining viable markers in the areas of speech symptoms, quantitative speech measures, impaired linguistic–motor processes, and neuroimaging. However, the review did not consider on what bases the diagnostic comparison groups were formed in this literature. To our knowledge, there has been no previous review of methods for translating speech features into diagnostic decisions, or what steps might have been taken to reduce bias. Therefore, the purpose of this study was to characterize the diagnostic process used in published research articles addressing the diagnosis and clinical presentation of CAS, AOS, and PAOS.

In addition to clinical impression, we considered two reference standard categories that are often used in medical and psychiatric research when a gold standard cannot be identified. The first option is to combine results of several relevant but imperfect quantitative tests or measures to form a more robust composite measure. Here, the risk of bias is in the selection of tests and the way in which they are combined. To minimize this risk, researchers are advised to demonstrate analytic rationales that are broadly accepted by the field or rely on statistical modeling to identify optimal combinations (Naaktgeboren et al., 2013; Umemneku Chikere et al., 2019). The second option is to rely on consensus diagnosis by a group of experts. While the risk of error is moderated by relying on experienced rather than novice clinicians, the risk of bias is, as we have observed, unavoidable. Therefore, its management warrants systematic steps, such as recruiting from diverse clinics, presenting cases in a standardized manner, and synthesizing divergent views about diagnosis as impartially as possible (Handels et al., 2014).

Review Questions

Our review was focused on two primary questions:

1. To what extent have clinical impression, tests or measurements, expert consensus, and other diagnostic methods been used to form comparison groups in recent diagnostic research involving CAS, AOS, and PAOS?

2. What steps have been taken to reduce the bias that is inherent to diagnosing with imperfect reference standards?

Method

Our review followed the Arksey and O'Malley (2005) scoping study framework for expressing the research question, identifying relevant studies, selecting studies, extracting data, and summarizing the results (Arksey & O'Malley, 2005). One departure was that we anticipated a database search would identify studies of sufficient diversity to answer our questions and did not deem it necessary to also hand-search reference lists and journals or solicit literature recommendations from the community. Similarly, we did not follow the optional recommendation to interview or survey stakeholders for further interpretation and contextualization of the study results. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (Tricco et al., 2018) was used to ensure adequate process and reporting.

The first step was to conduct a literature search inclusive of CAS, AOS, and PAOS research studies published in scientific journal issues between January 2014 and December 2024. A search of three electronic databases (PubMed, CINAHL, and PsychINFO) using the search term “apraxia of speech” generated 2,450 studies (see Figure 1). After removing 786 duplicates, the first author reviewed the title and abstract for 1,664 studies to identify those primarily focused on syndrome characteristics and comorbidities. Because the screening criteria were straightforward and inclusive of most descriptive research, we deemed it sufficient for one of us to be responsible for the initial abstract and title review. The inclusion criteria were as follows: publication in a peer-reviewed journal and in the English language; diagnosis of three or more participants with CAS, AOS, or PAOS; empirical evaluation of one or more clinical, experiential, or experimental responses to stimulus conditions during a single session; or descriptive evaluation of features associated with the diagnosis. Prospective, retrospective, and longitudinal studies were included. Since our research questions were about diagnostic research, we excluded articles using treatment designs, surveys or interviews, commentaries, tutorials, and reviews. If the relevance of a study was unclear from the abstract, the full article was reviewed for eligibility by all three authors at the time data were to be extracted. The initial abstract screening eliminated 1,434 studies. Following a full-text review of the remaining 230 articles, another 35 were eliminated by author consensus based on using one of the excluded study designs or addressing the wrong participant population. The screening resulted in a total of 195 articles for full review. A reference list of these is included as Supplemental Material S1.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram of the study selection process. One thousand four hundred thirty-four articles were excluded based on a screening of titles and abstracts, and 35 articles were eliminated after a full-text review based on using one of the excluded study designs or not addressing childhood apraxia of speech, apraxia of speech, or progressive apraxia of speech.

All three authors participated in the review of each article. Data extraction was completed using an online form that the authors developed (see the Appendix). Extracted data included the type of speech apraxia that was the focus of the article, the number of participants, and details about the process for diagnosing CAS/AOS/PAOS. Given our research questions, we were not concerned with what criteria were mentioned, but rather with the process for interpreting criteria to make a diagnosis. The extraction form was piloted with 69 research articles on all three speech apraxia syndromes and revised to improve clarity and correspondence with how information was presented in the literature. Based on this pilot work, we settled on three categories for the diagnostic process: (1) unverified clinical impression as diagnosis, (2) consensus or confirmation diagnosis by two or more clinicians, and (3) quantitative support for the diagnosis, which was subsequently divided into (a) verification of observed speech features on a diagnostic checklist and (b) tests or measurements using perceptual or acoustic methods.

For consensus or confirmation diagnosis, we considered two strategies to reduce bias: (a) standardized information about task and participants and (b) explicit steps to resolve disagreement. For diagnosis with quantitative support, we considered a single bias-reducing strategy: combination of two or more imperfect measures. When combinations were used, we recorded whether they were determined analytically or statistically. Finally, with each process, we made note of whether observer reliability was estimated for the diagnostic judgments.

We considered only information in the published article; other studies were not consulted for background nor was procedural information considered if it was conveyed exclusively in the supplemental materials, because these sources were not consistently available to our team. Extracted information was restricted to the diagnostic process as presented before the study was conducted. Because the purpose of most studies was to identify presentation features and potential markers indicative of CAS, AOS, or PAOS, speech measures and ratings were routinely provided as study results. We did not include these results since they were not relevant to the study's sampling or grouping strategy.

Each article was randomly assigned to two of the authors, with all authors reviewing an equal number of articles. We completed independent reviews and then met regularly over a period of 6 months to discuss 10–15 articles at a time. The two assigned reviewers for each article arrived at consensus during these meetings, while the third author checked for adherence to definitions and helped identify relevant sections of the articles that verified the extracted data. Results were exported to an Excel spreadsheet.

Results

A similar number of studies enrolled participants from each of the three diagnostic categories (66 articles for CAS, 61 for AOS, and 66 for PAOS). Two additional studies included more than one diagnostic category (AOS and PAOS in Takakura et al., 2019; CAS, AOS, and PAOS in Shriberg et al., 2017). These were added to the analysis within each etiology, yielding a total of 67 study samples of CAS, 63 study samples involving AOS, and 68 study samples involving PAOS. Our reporting of results is, therefore, based on a total of 198 samples (henceforth referred to as “studies”). Summary tables are provided in Supplemental Materials S2–S4. The sample size varied widely across studies (three to 375 participants with CAS [Mdn = 15.0], three to 68 participants with AOS [Mdn = 14.0], and three to 126 participants with PAOS [Mdn = 28.5]). Though several articles were from the same laboratory or clinic and participant samples overlapped, each article was analyzed separately.

Most of the 198 studies (76.8%) described or referenced the speech sample that was used for diagnosis—usually a combination of connected speech, word/sentence repetition, and repetition of diadochokinetic syllable strings. Children often completed an articulation test, and adults were sometimes asked to read orally. Most studies (92.4%) provided some explanation of the process for diagnosing speech apraxia in study participants. However, the information was not always complete and was often distributed across sections of the described methods. A little over half (52.5%) of the studies that did explain the diagnostic process listed or referenced syndrome criteria (e.g., ASHA, 2007; Duffy, 2020; McNeil et al., 2009; Shriberg et al., 2011; Wertz et al., 1984). The four diagnostic processes we coded were used differently in the literature for the three speech apraxia categories (see Table 1). We discuss each process in turn.

Table 1.

Processes used to diagnose childhood apraxia of speech (CAS; n = 67), apraxia of speech (AOS; n = 63), and progressive apraxia of speech (PAOS; n = 68), reported as the number (and percentage) of study samples within each diagnostic category.

Diagnostic process (reference standard)	CAS	AOS	PAOS
Not explained	3 (4.5)	4 (6.3)	8 (11.8)
Clinical impression	10 (14.9)	13 (20.6)	15 (22.1)
Consensus or confirmation	7 (10.4)	25 (39.7)	44 (64.7)
Checklist verification	47 (70.1)	2 (3.2)	1 (1.5)
Tests or measurements	0 (0.0)	19 (30.2)	0 (0.0)

Diagnosis by Clinical Impression

The first diagnostic process we considered was simple clinical impression by one or more persons from the research team, usual care, or both. Studies were included in this category if a diagnostic decision was made by clinical judgment alone. Diagnostic criteria were sometimes listed or referenced, but the presence of these features was not explicitly verified, and there was no evidence that two or more clinicians reviewed every case and came to consensus about the diagnosis. A modest proportion of studies (19.2%) used a simple clinical impression strategy, and, as shown in Table 1, they distributed relatively equally across the three speech apraxia syndromes. In 39.5% of the studies that relied on clinical impression, it was unclear who made the diagnosis; in 31.6%, the diagnosis was made exclusively by members of the research team; and in another 28.9%, diagnosticians included one or more clinicians who were not on the research team.

We found reliability for diagnosis by clinical impression reported in five studies (13.2%). Ziegler et al. (2022) conducted a study to estimate the prevalence of AOS via converging evidence from clinical and research sources. Three experienced and calibrated researchers reviewed video recordings of 148 persons with aphasia who participated in a randomized controlled treatment study (Breitenstein et al., 2017). The diagnosticians provided impressions concerning AOS diagnosis/severity as well as confidence in these judgments. Between-raters reliability was moderate but significant. Another study from the same group (Aichert et al., 2016) used a validation process wherein one speech therapist provided the initial diagnosis and a second completed an independent diagnosis for 14 participants with AOS and four with aphasic-phonological presentation. The second rater was asked to classify each participant and to provide a judgment of certainty (certain, slightly uncertain, or very uncertain). The results showed agreement with certainty in 12 cases, agreement with slight uncertainty in five cases, and disagreement with high uncertainty in one case. In a study examining comorbidity of other conditions with CAS, Chenausky et al. (2023) relied on one of three SLP coauthors to provide a diagnosis by clinical impression. This diagnosis was verified for 23% of the sample seen for follow-up assessment by a second SLP, with no reported discrepancies. Additionally, two coauthors independently assessed 22% of the participant group using the Diagnostic Evaluation of Motor Speech Skill (DEMSS; Strand & McCauley, 2019), resulting in an intraclass correlation of .98 based on the DEMSS total score. Finally, a recent pair of studies reported reliability for diagnoses of progressive AOS (Illán-Gala et al., 2024; Lorca-Puls et al., 2024). In both studies, at least one certified SLP rated the clinical severity of AOS and dysarthria on a 0–7 scale. For 15% of the samples, a second independent rating was made, with intraclass correlations between the two ratings showing excellent agreement (.86 for AOS and .79 for dysarthria).

Diagnosis by Consensus or Confirmation

The second diagnostic strategy was by consensus or confirmation. Consensus diagnosis was defined as a strategy where every participant was diagnosed independently by two or more diagnosticians who subsequently arrived at a diagnosis on which they agreed. Confirmation diagnosis was defined as an initial diagnosis (typically from the usual care team) that was followed by an independent confirmation by a second diagnostician (usually from the research team). It was necessary to group these consensus and confirmation diagnoses together in the review because the exact procedure was often not explained. If a consensus or confirmation method was not described in the article or applied to all participants, the diagnostic process was by default classified as “clinical impression.”

Consensus or confirmation diagnosis was used in 38.4% of all the studies we reviewed. Two or three experts made judgments about the presence or absence of speech apraxia in all these studies. It was the leading strategy for both the AOS and PAOS diagnoses (39.7% and 64.7% of study samples, respectively). In 24.1% of the studies that relied on consensus or confirmation diagnosis, it was unclear who made the diagnosis; in 52.6%, the diagnosis was made by members of the research team; and in 22.4%, it included one or more clinician who was not on the research team.

Next, we considered procedures to reduce inherent bias in the consensus or confirmation diagnosis. The first strategy we looked for was the use of a standardized method for how information about each case was shared with diagnosticians and to what extent lab results, assessments, and medical history were made available. A standardized case presentation would include assurance that either none or all experts on the panel had met each person that was to be diagnosed and, ideally, information considered relevant to diagnosis would be presented in a consistent format to all diagnosticians. This level of standardization was not used in the speech apraxia syndrome literature. We identified only one study that detailed the information about the case that was shared and how. The study involved diagnosing children with CAS (Vuolo et al., 2023), and the standardization was relatively limited. Families provided assessment reports they had received about their children, then speech and language testing was completed and video-recorded, and finally both sources of information were reviewed independently by the diagnosticians.

Bias-reducing strategies can also be used to control how consensus is reached. Many studies did not explain the process for managing disagreement. For the 64% of AOS studies and 11.4% of PAOS studies that did explain the method, it was usually to simply use a majority vote or recruit a third rater as the tie breaker if the original assignment was different for two diagnosticians. One consensus study incorporated a degree of certainty in the rating (1 = no AOS, 2 = possible AOS, 3 = AOS) and operationalized the expert diagnosis for group formation based on a mean rating of 2.0 or greater across three diagnosticians (Maas et al., 2015).

Reliability among diagnosticians was estimated for 11 (14.0%) of the studies that relied exclusively on consensus or confirmation diagnosis. Two studies reported reliability using a kappa statistic, with results ranging from good to substantial agreement (Ballard et al., 2016; Scholl et al., 2017). Most other studies reported percentage of unanimous agreement between raters prior to discussion, with values ranging from 40% (Maas et al., 2015) to 100% (Bislick & Hula, 2019; Haley & Jacks, 2019; Takakura et al., 2019) and intermediate agreement from 67% to 85% (Maas et al., 2014; Mailend et al., 2019; Mailend, Maas, Beeson, et al., 2021; New et al., 2015; Vergis et al., 2014).

Diagnosis by Checklist Verification

The third diagnostic reference standard we identified was checklist verification, which we considered to be a form of quantitative perceptual assessment. It was typically combined with clinical impression or informal consensus or confirmation diagnosis by the same diagnosticians. In some cases, it was combined with tests or measurements. Thus, we refer to it with a plus sign. The role of one to three diagnosticians was to verify that a predetermined number of diagnostic features were present in each participant's speech samples. Checklist verification was used in 50 studies. Forty-seven of these studies involved CAS diagnosis, where it was the most common reference standard, used in 70.1% of the study samples.

The CAS checklists were sometimes customized by the research team, but usually they involved recommendations from the ASHA (2007) technical report (inconsistent errors in repeated productions of syllables or words, lengthened and disrupted coarticulatory transitions between sounds and syllables, and inappropriate prosody, especially in the realization of lexical or phrasal stress), four or five of any of the features on the so-called Mayo checklist (vowel distortions, difficulty achieving initial articulatory configurations or transitionary movement gestures, equal stress or lexical stress errors, distorted substitutions, syllable segregation, groping, intrusive schwa, voicing errors, slow rate, slow diadochokinetic rates, and increased difficulty with multisyllabic words; Shriberg et al., 2011), or a combination of these two checklists. Consistent with the ASHA guidelines, it was sometimes required that any documented features were observed in more than one speaking context. Some studies provided confirmation at the individual level about which features were present (e.g., Case & Grigos, 2020; Vuolo et al., 2023), whereas other studies did not document which specific features were observed or in how many participants, effectively reducing the distinction between listing diagnostic features and documenting the diagnostic process. Several recent studies defined the features operationally (e.g., Iuzzini-Seigel, 2021; Lewis et al., 2023; Zuk et al., 2018), and some features, such as phonemic inconsistency, difficulty with multisyllabic words, and poor verbal sequencing, were defined quantitatively (e.g., Iuzzini-Seigel, 2019; Miller et al., 2019; Preston et al., 2014), thus adding a level of direct measurement to the checklist verification.

Two studies used checklist verification to diagnose AOS, and one study also applied it to primary progressive apraxia of speech. Graff-Radford and colleagues (2014) did not require a specific number of features for AOS diagnosis; however, one or two diagnosticians checked the presence or absence of seven speech features that were deemed relevant (slow rate, distorted substitutions, vowel distortions, increased sound errors with increased complexity, segmentation of words and/or syllables, self-correction of sound level errors, and articulatory groping) in seven participants with suspected pure AOS. Shriberg et al. (2017) used the same Mayo checklist (Shriberg et al., 2011) to diagnose CAS, AOS, and PAOS in their study involving all three syndromes.

Reliability of diagnosis was estimated for 10 (20.0%) of the study samples using checklist verification—all studies of CAS. Six studies reported reliability of the resulting CAS classification diagnosis assignment based on the checklist. Interrater agreement was high (90%–100%), based on a second rating for a subset of participants (Chilosi et al., 2015; Ingram et al., 2019; Murray et al., 2015; Overby et al., 2019, 2020; Shriberg et al., 2017). Four additional studies reported strong reliability of identifying CAS features, with intraclass coefficients of .83 (Fiori et al., 2016) and .93 (Zuk et al., 2018), percent reliability of 92% (Iuzzini-Seigel, 2019) and 100% (Grigos et al., 2015), and a kappa statistic of .83 (Fiori et al., 2016). An additional study reported 100% reliability between two author raters who completed independent ratings; however, the nature of agreement was unclear whether pertaining to diagnostic assignment, rating of features, or both (Gold et al., 2022). Finally, rather than estimating diagnostic reliability, some studies (e.g., Vuolo & Wisler, 2024) used independent agreement about checklist features as a strategy for participant selection, resulting in a known or unknown number of potential participants not qualifying for enrollment.

Diagnosis by Test or Measurement

The fourth and final diagnostic strategy was to base the diagnosis directly on tests or measurements. This strategy was used as the reference standard for diagnosis in 30.2% of AOS study samples but in none of the CAS or PAOS study samples (see Table 1). The most common test instrument was the Apraxia of Speech Rating Scale (ASRS; Strand et al., 2014), which was used for diagnosis in eight studies (Basilakos et al., 2015, 2017; Chenausky et al., 2020; den Ouden et al., 2018; Hybbinette et al., 2021, 2022; Lancheros et al., 2023; Moser et al., 2016). Different versions of the scale have included 13–16 abnormal speech features, each of which is rated on a 5-point scale from 0 (not present) to 4 (nearly always evident and marked in severity). A summed score of 8 or higher was considered indicative of AOS, consistent with recommendations (Duffy et al., 2023; Strand et al., 2014). At least one so-called primary feature was required in some studies (i.e., distorted sound substitutions, distorted sound additions, increased sound distortions or distorted sound substitutions with increased utterance length or increased syllable/word articulatory complexity, increased sound distortions or distorted sound substitutions with increased speech rate, inaccurate speech alternating motion rates, or reduced words per breath group relative to maximum vowel duration) per the recommendations of Strand et al. (2014).

The words of increasing length, repeated trials, and inventory of articulation subtests of the Apraxia Battery for Adults–Second Edition (Dabul, 2000) were scored by diagnosticians in three AOS studies and used as quantitative evidence for AOS (Ballard et al., 2018; Hickok et al., 2014; Trupe et al., 2018). Six AOS studies (Galluzzi et al., 2015; Haley et al., 2021; Haley & Jacks, 2023; Haley, Jacks, Richardson, et al., 2023; Harmon et al., 2019; Romani et al., 2017) relied fully or partially on the proportion of perceptually coded “distortion” or “phonetic” errors on a word repetition task, and seven used acoustic measures of articulation rate or lexical stress as diagnostic evidence of abnormal prosody (Ballard et al., 2018; Cunningham et al., 2016; Haley et al., 2017, 2021; Haley & Jacks, 2023; Haley, Jacks, Richardson, et al., 2023; Harmon et al., 2019). Four of these studies required meeting quantitative criteria for both speech sound distortion and prosodic features. Ballard et al. (2018) required converging evidence of impairment based on a words-of-increasing-length test, an acoustic measure of lexical stress, and consensus diagnosis between two diagnosticians.

Reliability was reported in eight (42.1%) of study samples that diagnosed apraxia by tests score or other measurements, all studies of stroke AOS. Three studies using the ASRS total score for diagnosis obtained reliability ratings for a subset of participants (10%–23% of the sample; Basilakos et al., 2017; Chenausky et al., 2020; den Ouden et al., 2018), with coefficients ranging from 0.88 to 0.90, in the good to excellent range and another reported an intraclass coefficient of 0.95 for an ASRS average score (Moser et al., 2016). Trupe et al. (2018) diagnosed AOS based on a score greater than five on the Apraxia Battery for Adults–Second Edition inventory of articulation checklist; a second independent reviewer scored 29% of the sample, and this resulted in 100% agreement with the original diagnosis.

Haley and colleagues operationalized AOS diagnosis using measurement of average syllable duration for multisyllabic words (Haley et al., 2017, 2021) and distortion frequency (Haley et al., 2021). Reliability for the word syllable duration was obtained for a subset of samples in both studies (12%–30%) and compared using Pearson correlation, with values ranging from .93 to .96. In the 2021 article, the authors also reported reliability on coding words with distortion errors for 18% of the sample, with a Pearson correlation of .86. Galluzzi et al. (2015) diagnosed AOS for participants producing phonetic errors in 10% or more of segments. Reliability transcriptions were completed for 20% of participants. Overall scoring consistency was 90%, but reduced to 76% when only phonetic errors were considered.

It is important to acknowledge that though few studies in this review based the apraxia diagnosis on quantitative test scores, most of the 198 studies (70.2%) provided some descriptive information about participants' speech characteristics or severity, even if it was only a severity rating. In addition to the tests and measurements that were referenced for diagnostic purposes, instruments used for descriptive purposes included the Goldman-Fristoe Test of Articulation (Goldman & Fristoe, 2015), the DEMSS (Strand & McCauley, 2019), the Diagnostic Evaluation of Articulation and Phonology (Dodd et al., 2006), the Verbal Motor Production Assessment for Children (Hayden & Square, 1999), the Kaufman Speech Praxis Test for Children (Kaufman, 1995), intelligibility scores from the Assessment of Intelligibility of Dysarthric Speech (Yorkston et al., 1984) or the Chapel Hill Multilingual Intelligibility Test (Haley et al., 2011), perceptual measures of speech sound accuracy (percentage of consonants, vowels, or words produced correctly), and a variety of instruments for languages other than English.

Discussion

Our review of 11 years of descriptive diagnostic studies of CAS, AOS, and PAOS confirmed that differential diagnosis, with few exceptions, has relied on clinical expertise in one form or another. It is on the one hand sensible to use gestalt impressions by experienced clinicians to capture recurring patterns, especially considering that the syndromes are not yet fully characterized or defined quantitatively relative to their diagnostic neighbors. The approach is also practical and consistent with current clinical practice (Mailend, Maas, & Story, 2021; Wambaugh et al., 2019). On the other hand, unless calibrated to a standard, diagnosticians have no choice but to interpret speech features based on their personal experiences and beliefs (Haley et al., 2012; Murray et al., 2023; Shriberg et al., 2003). This is simultaneously a problem of validity and reliability, and it requires proactive action. The concern for this study, therefore, was to characterize the diagnostic process that underlies research on the features of the speech apraxia syndromes, where the goal is to develop valid and reliable markers and tests for CAS, AOS, and PAOS. We begin by considering our two research questions.

Question 1: Methods for Diagnosing Speech Apraxia Syndromes

We first asked to what extent clinical impression, tests or measurements, expert consensus, and other methods for diagnosing CAS, AOS, and PAOS have been used in recent diagnostic research. Results showed that explicit diagnostic procedures have been used extensively since at least 2014—the earliest publication year we considered. This is encouraging since the practice of referencing or listing features without connecting them to diagnostic decision making may promote unjustified assumptions of validity (Haley et al., 2012). It should, however, be noted that more than a quarter of these recent studies (26.8%) neglected to reveal the diagnostic process or relied on simple clinical impression that was not further described.

Most studies used either consensus/confirmation or checklist verification strategies. The most common process was to select and group participants based on the consensus or confirmation of two or three expert clinicians' subjective impressions. Usually, local clinicians or members of the research team qualified as experts. While the use of expert consensus is an important step to reduce misdiagnosis, the small number of raters is concerning, since there is risk for limited representation of alternative viewpoints. Moreover, if diagnosticians have experience working together outside the study, reliability estimates may be inflated by undisclosed or unknown variables that cannot easily be replicated elsewhere. When using members of the research team as diagnosticians—as was the case in over half of these studies—there is also risk of observer bias if they have knowledge of the speech features or other variables that are to be evaluated in the study.

Checklist verification was the simplest and most common form of measurement we identified. The approach requires a defined set of criteria and a process for verifying that a certain number of them are present in people diagnosed with one of the speech apraxia syndromes. Checklist verification was the dominant diagnostic strategy for contemporary diagnosis of CAS and had the additional advantage that most of those studies relied on the same checklists (ASHA, 2007; Shriberg et al., 2011). Note, however, that considerable subjective judgment is still required to determine whether there is sufficient evidence to rate listed features as present in a person's speech output. Take for example the feature “slow rate.” The diagnostician must decide what does and does not count as “slow.” For example, do pauses and hesitations count? How slow is slower than normal? What type of speech sample would be appropriate for evaluating slowness?

Operational definitions are meant to reduce uncertainty by describing behaviors in such a way that they can be observed and measured (Haley et al., 2012; Terband et al., 2019). For example, Iuzzini-Seigel and colleagues defined “slow rate” in CAS as follows: “Speech rate is not typical. It is slower during production of part (e.g., zzziiiiiiper/zipper) or the whole word (e.g., tooommmmaaatoooo/tomato)” (Iuzzini-Seigel et al., 2015). This definition guides diagnosticians to focus on slowness within words and clarifies that sound prolongation and overall slowness are both valid examples; however, it does not clarify whether pauses within or between words might also exemplify slowness.

Given that operational definitions define behaviors in ways that are also measurable, recent CAS studies have begun relying on measurement to decide whether certain checklist items should be considered present or absent. For example, difficulty with multisyllabic words has been defined as scoring 2 SDs below the mean for age on formal assessment (e.g., Miller et al., 2019) and phonemic inconsistency has been defined as scoring above cutoff on a calculated measure (e.g., Iuzzini-Seigel, 2019). Well-crafted operational definitions and other quantitative metrics can reduce the risk of confirmation bias as diagnosticians review the checklist, so they do not subconsciously give greater weight to observations that support their immediate “gestalt” impression.

In a sense, the ASRS may be viewed as a checklist with integrated magnitude ratings from 0 to 4. In the present review, the ASRS was used for diagnostic purposes in eight studies of AOS in stroke, using the recommended cutoff score of 8. Counterintuitively, even though it was developed for PAOS, it was not used for diagnostic decision making in any diagnostic studies involving that population. Although the scale authors have demonstrated diagnostic validity for PAOS (Duffy et al., 2023; Strand et al., 2014), so far all studies of PAOS from the Mayo Clinic have used consensus or clinical impression for diagnosis, whereas the item ratings from the ASRS, individually or collectively, have been used as supplementary information about severity. Recent versions of the scale have been used by the authors to evaluate the relative prominence of phonetic versus prosodic feature clusters and to differentiate phonetic versus prosodic subtypes of primary PAOS (itself a subtype of PAOS; Utianski, Duffy, Clark, Strand, Botha, et al., 2018), but not to diagnose the presence or absence of PAOS.

Our review found no evidence of studies relying on comprehensive syndrome quantification, which is understandable given that agreed-upon collections of diagnostic markers have yet to be identified and are the envisioned outcome of much diagnostic research on the speech apraxia syndromes. Tentative diagnostic groups have occasionally been formed in the AOS literature based on one to three measures linked to the syndrome definition. Most of these studies are from our own laboratory. For example, we have used slow production of multisyllabic words, perceived speech sound errors at the phonemic level, and perceived sound distortions to form groups of participants who likely have AOS, participants who likely have aphasia with phonemic paraphasia, and participants who have no or minimal problems with speech sound production. The choice to use one to three metrics has been strategic for our group to avoid circularity in the study design, whereby participants are grouped on the same feature that is to be evaluated as a potential diagnostic marker (see the discussion in the later section on diagnostic fallacies). For example, when our team was interested in whether the frequency of perceptible distortion errors differentiates between AOS and aphasia with phonemic paraphasia (e.g., Haley et al., 2017), we first ensured that all eligible participants did make sound errors (by requiring a certain frequency of phonemically evident errors) but formed our comparison groups based on articulation rate alone (operationally defined as abnormally long mean duration of syllables in multisyllabic words and therefore including both sound prolongation and pauses; Haley, Jacks, Kim, et al., 2023). Though this strategy has worked reasonably well for our research program, it obviously serves only as an approximation of syndrome diagnosis. It will be important for future research to use more complete combinations of quantitative measures and to develop norms from childhood to old age (Haley et al., 2012; Haley, Jacks, Kim, et al., 2023; Holm et al., 2007; Murray et al., 2021; Terband et al., 2019). A comprehensive set of norm-referenced measures can then be combined either logically (through a version of the checklist approach—ideally based on professional consensus) or with weights derived from statistical modeling in a large data set (Naaktgeboren et al., 2013; Umemneku Chikere et al., 2019).

Like operational definitions, quantitative assessments help reduce the risk of confirmation bias whereby diagnosticians unknowingly notice features that corroborate their initial diagnostic impression. It also offers transparency of diagnosis and longevity of study impact, given lingering agreement limitations concerning how to best identify and interpret diagnostic speech features. Even if some features that are considered valid today turn out to have limited value for CAS, AOS, or PAOS diagnosis when studied in greater depth, classifications that include them can still be interpreted if relevant tests, measures, or checklist verifications are provided for all participant groups. The same cannot be said for diagnostic impressions or even consensus/confirmation diagnosis that is not supported by documented observations.

Question 2: Steps to Reduce Diagnostic Bias

Having established that most diagnostic research on CAS, AOS, and PAOS rely on a defined but imperfect reference standard, we turn now to strategies for minimizing bias. Our second question was what steps have been taken to reduce the bias that is inherent to diagnosing with imperfect reference standards. One approach is to combine reference standards. While checklist verification was frequently combined with consensus/confirmation diagnosis and to a limited extent with tests or measurements, it was far less common to combine consensus/confirmation diagnosis with tests or measurements. Combinations of individual tests or measurements were, as mentioned, restricted to a small number of AOS studies and not intended to capture the full syndrome. All combinations were made by reasoning rather than statistical methods. As diagnostic research becomes increasingly precise, we encourage researchers to incorporate converging evidence through more than one diagnostic strategy when assigning participants to comparison groups and when planning a series of diagnostic studies. It is our hope that future studies may consider a broad repertoire of consensus, checklist, and measurement techniques by consulting literature beyond a single speech apraxia etiology. For example, AOS and PAOS researchers may consider adding checklist verification and, when using the ASRS, report scores for individual features to allow evaluation independently of gestalt impression. Conversely, CAS researchers may wish to consider adding magnitude ratings to the checked items and expanding options for quantitative measures. Additionally, consensus building across research groups (and potentially speech apraxia syndromes) can increase the likelihood that similar reference standards are adopted and scientific progress is expedited.

We also found almost no evidence for bias-reducing strategies in studies using diagnosis by consensus or confirmation. In medical diagnostic research, consensus diagnosis is strengthened by recruiting a panel of expert raters from different work settings, confirming that they are unfamiliar with study participants, taking explicit steps to calibrate them to the task, and standardizing the case presentation and process to build consensus (Handels et al., 2014). It is advisable that all experts on the panel receive the same information about social and medical history, test results, and observations from any physical and behavioral examinations. Given recent advances in diagnostic research on the speech apraxia syndromes (Allison et al., 2020), this background information might include neuroimaging results for adult-onset disorders (Basilakos et al., 2015; Botha et al., 2015; Chenausky et al., 2020; Moser et al., 2016; Wilson et al., 2023) and genetic test results for children (Lauretta et al., 2024; Peter et al., 2016). Details about case presentations were absent in the consensus/confirmation studies we reviewed. Moreover, though it was often clarified that a majority vote determined consensus, the process for arriving at this agreement was not detailed. This introduces a risk that decision making was driven by one or more team members with, for example, distinct authority, extensive experience, and/or persuasive assertiveness. Delphi methodology has been proposed to counteract the problem by structuring the consensus process (Jones & Hunter, 1995) and could potentially be used in larger studies of CAS, AOS, and PAOS.

Reliability estimation for diagnostic decisions is used to evaluate the degree to which classifications are consistent and replicable. This information helps clinicians and researchers determine whether study results are trustworthy and interpretable. Attention to reliability also works proactively by holding researchers accountable to use strong definitions and replicable methods, thereby improving the quality of the research. Our results suggest that reliability estimation for differential diagnosis is an area for strategic improvement in diagnostic research involving CAS, AOS, and PAOS, as only 17.7% of the studies we reviewed estimated diagnostic reliability among diagnosticians or raters. Levels of reliability or agreement ranged from fair to excellent. This does support feasibility for the varied diagnostic processes we reviewed. Three studies (< 2%) accounted for the possibility of diagnostic uncertainty by capturing degree of confidence in diagnosis for study participants (Aichert et al., 2016; Maas et al., 2015; Ziegler et al., 2022). We applaud this effort and transparency, given the known difficulty of differential diagnosis and the diagnostic fallacy of a false dichotomy (discussed in the next section) that appears to disguise true performance variations within CAS, AOS, and PAOS.

Diagnostic Fallacies

The lack of a gold standard introduces risk for logical error in diagnostic research. Here, we consider two fallacies. The first—circular reasoning—applies most obviously to diagnosis by checklist verification and diagnosis by tests or measurements. It occurs when the evidence used to support a claim is the same as the claim itself, for example, when certain speech features are found to be more common in CAS, AOS, or PAOS than in a comparison group, but those same features were relied upon to differentiate the apraxia group from the control group in the first place. It also occurs in diagnosis by clinical impression or consensus/confirmation when the feature to be evaluated is interpreted as evidence of diagnosis. Diagnostic research that is limited by circular reasoning would, for example, show that people who are perceived to have slow speech and abnormal word prosody do indeed produce words slowly. While this knowledge may help clarify measurement options, it does not evolve our understanding of the condition. We caution that a variant of the circular reasoning fallacy applies when the comparison group produces very few speech sound errors—a form of sampling bias that exaggerates the difference between diagnostic groups. Here, we learn how those who have a speech sound production disorder differ from those who do not, but not about what is specific to the speech apraxia syndrome. Whereas sampling bias for the comparison group has been addressed in the CAS literature by screening participants on an articulation test, it has not been addressed systematically in AOS and PAOS research. Consequently, participants in comparison groups with aphasia tend to produce fewer speech sound errors—sometimes dramatically so.

The second fallacy is the false dichotomy of diagnosis—the illusion that only two options (present/absent) are possible (Jenniskens et al., 2019). The premise of diagnosing CAS, AOS, or PAOS as simply present or absent is inconsistent with evidence showing multidimensional quantitative continua or clusters of relevant speech features (Haley & Jacks, 2023; Mailend & Maas, 2020; McCabe et al., 2024; Takakura et al., 2019; Utianski, Duffy, Clark, Strand, Botha, et al., 2018). This introduces the reality of performance profiles for which diagnosis is uncertain. For example, in a study of 318 participants with progressive disease, Duffy et al. (2023) concluded that 14% of participants obtained ASRS scores in a range that could not be interpreted with a high degree of confidence concerning a PAOS diagnosis referenced against the researchers' own consensus diagnosis.

In our own research on the AOS syndrome, we have responded to this dilemma by acknowledging the presence of both prototypical and equivocal profiles (e.g., Haley et al., 2012; Haley, Jacks, Richardson, et al., 2023). The practice of refraining from making a conclusive diagnosis appears to be more common in clinical practice than in research (Iuzzini-Seigel et al., 2022; Miller et al., 2019) but may paradoxically bring clarity to diagnostic research. Related to this issue, presentation profiles change as children mature (Lewis et al., 2004; Shriberg et al., 2003), as adults recover from stroke or trauma (Haley et al., 2016; Polanowska & Pietrzyk-Krawczyk, 2016; Wilson et al., 2023), and as neurodegenerative diseases progress (Duffy et al., 2021; Josephs et al., 2014; Utianski, Duffy, Clark, Strand, Boland, et al., 2018). Diagnostic markers must adapt to this change and allow for profiles that evolve over time toward or away from recognizable speech apraxia syndromes. Documenting this evolution in longitudinal studies should prove valuable for both prognostication and treatment. Similarly, between-syndromes comparisons, simulations, longitudinal studies, and evaluation of the relationship between speech and language processes may help elucidate underlying principles and contribute to theory building and testing.

The Clinical Reason for Diagnosis

From a clinical perspective, the primary reason to value diagnostic accuracy is that it has the potential to improve treatment and counseling (Maas, 2024; Murray & Iuzzini-Seigel, 2017; Wauters et al., 2023). After all, behavioral syndromes are designated not only by constellations of observable features that co-occur with some regularity but also by presumed differences in prognosis and response to intervention. This presents an alternative strategy to the missing gold standard dilemma. Given the current state of the science, researchers might spend less energy on the quest for diagnostic accuracy and more energy on validating the clinical relevance of diagnostic tests, ratings, and measurements that are linked to the disorder of interest. This pragmatism has been a long-term research strategy in medical research lacking a diagnostic gold standard, where it is implemented with statistical methods, such as event rates, relative risks, and correlation analyses (Bossuyt et al., 2012). Interdisciplinary research on PAOS has already demonstrated differences in mortality and rate of decline for those with predominant sound distortions, those with mostly slow and segmented speech, and those who also present with prominent agrammatic aphasia (Utianski, Duffy, Clark, Strand, Boland, et al., 2018; Whitwell et al., 2017, 2021). Diverse speech profiles in children and stroke survivors (including but not restricted to CAS and AOS) may likewise have prognostic implications. Conversely, whereas advantages of dynamic assessment have been demonstrated in CAS management (Strand et al., 2013), there appears to be missed opportunities in applying similar inquiries to AOS and PAOS.

There is surprisingly limited empirical information about diagnostic relevance for treatment outcomes, as reflected by the dearth of comparison groups consisting of other diagnostic groups with speech sound disorders in CAS and AOS treatment research (Munasinghe et al., 2023; Murray et al., 2014, 2021; Wauters et al., 2023). By applying the same treatment conditions to study participants with sound production disorders of varied types, it will be possible to clarify the connection between diagnostically relevant tests or measurements and treatment response. Over time, such research should help validate practical utility of the speech apraxia syndromes and potentially adjust or refine their boundaries.

Recommendations and Future Directions

This scoping review demonstrated that recent diagnostic research on CAS, AOS, and PAOS has used systematic strategies to form relevant comparison groups, but that there is room for improvement in managing the bias that comes with relying on imperfect reference standards. We offer the following recommendations for future research: (a) Report both the features and the process that were used to form comparison groups; (b) accept that there are presently no gold standards for diagnosing CAS, AOS, or PAOS, and that clinical standards for diagnosis, while convenient and clinically feasible, are inherently subjective; (c) minimize diagnostic bias by using standardized and replicable processes and converging evidence from complementary diagnostic processes; (d) document evidence for diagnostic reasoning with transparency, so it can be interpreted independently by other research teams both at the present time and in the future; (e) estimate diagnostic reliability; (f) invest in longitudinal research and design treatment studies to evaluate candidacy based on varied speech sound disorders; (g) coordinate diagnostic reference standards across research groups and speech apraxia syndromes and share resources to improve efficiency and rate of discovery.

Data Availability Statement

Most data generated during this study are included in this published article and its supplemental material files. Any additional data will be shared with interested researchers by the corresponding author upon reasonable request.

Appendix

Data Extraction Instrument

1.  First Author, Year, Title:

2.  What type of apraxia of speech is this article about?

•    □  CAS (including suspected CAS)

•    □  AOS (including suspected AOS; stroke, TBI, tumor)

•    □  PAOS (progressive AOS: AOS with PPA, PPAOS)

•    □  This study should be excluded (No apraxia diagnosis was given in the participant description, or it was a treatment, survey, or review study. Exclude if apraxia diagnosis was an outcome of the study). Also exclude case studies (n < 3)

3.  How many participants with (any form of) apraxia of speech were included?

4.  Was the process for determining CAS/AOS/PAOS group assignment or eligibility for diagnosis explained?

•    ○  Yes

•    ○  No

                  If NO skip to # 16

                 YES continue:

5.  What was the process for determining CAS/AOS/PAOS group assignment or eligibility for diagnosis in the methods section?

•    □  Clinical impression by one or more person (known or unknown) or usual care (per chart or referral), and it is not stated that each participant was evaluated by two diagnosticians

•    □  Consensus or confirmation diagnosis (two or more clinicians diagnosed each participant. This can include one person from the research team and one from the referral source)

•    □  Quantitative and replicable inclusion criteria from one or more test, checklist, or measures (e.g., cutoff score, decision tree, number of features observed)

6.  IF you selected that the apraxia diagnosis was made by CLINICAL IMPRESSION, CONFIRMATION, or CONSENSUS: Who made the diagnosis?

•    □  One or more persons who appeared to be from the research team

•    □  One or more persons who appeared to be from education/health care

•    □  One or more professionals; unclear whether research team or education/health care

•    □  Not applicable

7.  IF diagnosis was made by CONSENSUS or CONFIRMATION; How many clinicians or experts were involved in the consensus?

•    □  2–3 persons

•    □  > 3 persons

•    □  Not applicable

8.  IF diagnosis was made by CONSENSUS or CONFIRMATION; What was the procedure for reaching consensus? Note that the question asks for a procedure for how real or potential disagreements were handled.

•    ○  There were explicit steps for reaching consensus (at a minimum the process for resolving disagreement was stated)

•    ○  Consensus was implied or reached informally, or the research team confirmed diagnosis by referring clinician/s

•    ○  Not applicable

9.  IF diagnosis was made by CONSENSUS (not confirmation); Did one or more diagnosticians meet the client?

•    ○  Yes (only if explicitly stated)

•    ○  No (only if explicitly stated)

•    ○  Unknown

•    ○  Not applicable

10.  IF diagnosis was made by CONSENSUS (not confirmation); Did all diagnosticians have the same exact information about the client?

•    ○  Yes (only if explicitly stated)

•    ○  No (only if explicitly stated)

•    ○  Unknown

•    ○  Not applicable

11.  What speech material was used to evaluate the speech quality for the apraxia diagnosis?

•    ○  Connected speech

•    ○  Word repetition

•    ○  Repetition of sounds or syllables

•    ○  AMRs, SMRs

•    ○  Motor Speech Evaluation (not described or referenced)

•    ○  Not mentioned

•    ○  Other

12.  Explain how any of the following TESTS were used for diagnosis or demographics involving ALL participants (to make a diagnosis, included descriptively, or not included). Do not rate assessments that are the study's dependent variable/s.

Instrument/measure	Use in the diagnostic process
ASRS full score (0–52)	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants
ASRS severity rating (0–4)	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants
ABA/ABA-2	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants
DEMSS	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants
DEAP	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants
Acoustic measures	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants
Perceptually coded/quantified sound errors (not within tests)	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants
Other	Clearly used to make diagnosis for all participants	Used for severity estimation or descriptively	Not included, not mentioned, or not used for all participants

13.  Was reliability estimated for the apraxia diagnosis (for a subset of participants is ok)?

•    ○  Yes

•    ○  No

14.  IF several diagnostic strategies or tests were used for diagnosis, was there a predetermined and replicable process or composite criterion to diagnose all participants? Do not consider items WITHIN tests or checklists as separate diagnostic strategies.

•    ○  YES, determined ANALYTICALLY for all participants (including cutoffs or number of features present for rating scales)

•    ○  YES, determined STATISTICALLY (including equation)

•    ○  NO but more than one strategy was used for a subset of participants to confirm diagnosis

•    ○  NO/Not applicable

15.  Were any of the following processes used as criterion for diagnosis (this is to supplement question #7)?

•    □  Listed criteria or features as criterion for apraxia (e.g., ASHA 3, Strand et al., McNeil et al., Wertz et al.). It is not enough to just reference a textbook. Do not count a restating of test results here if already reported (e.g., restating ASRS features that are indicative of AOS or quantifying features through measurements or tests)

•    □  Severe disorder or symptoms as criterion for apraxia (e.g., below threshold on phonology/articulation test)

•    □  Response to treatment as criterion for apraxia (e.g., lack of response or response to a specific treatment)

•    □  Exclusion criteria for apraxia (e.g., no receptive language impairment)

•    □  Qualitative description of all participants' speech to justify apraxia diagnosis

•    □  None of these

16.  Comments (include any relevant observations not addressed in previous questions) or justification for scoring if needed:

Funding Statement

This research was supported by the National Institutes of Health—National Institute on Deafness and Other Communication Disorders under Awards R01DC8569 to Katarina L. Haley and R21DC018893 to Marja-Liisa Mailend.