Systematic review of clinical guidance documents for autism spectrum disorder diagnostic assessment in select regions

Abstract

Clinical guidance documents play an important role in ensuring access to high-quality autism spectrum disorder diagnostic assessment practices. The objective was to perform a systematic review of professional association and government clinical guidance documents for autism spectrum disorder diagnostic assessment, analyzing their quality and content. The government search was limited to English-speaking, single-payer, publicly funded health systems. A quality appraisal was conducted by two appraisers using the Appraisal of Guidelines Research and Evaluation, second edition tool. A content analysis was conducted for recommended clinical personnel and psychometric tools. The 11 documents demonstrated higher quality in Scope and Purpose (mean: 90.1, standard deviation: 7.4) and Clarity of Presentation (mean: 82.8, standard deviation: 9.4) and lower quality in Applicability (mean: 43.3, standard deviation: 23.8) and Rigor of Development (mean: 52, standard deviation: 21.9). All documents either recommended multidisciplinary team assessment or stated it was ideal. The documents varied substantially in their recommended tools and personnel for diagnostic assessment. There was little supporting evidence for team and personnel recommendations. Multiple guidance documents exist for autism spectrum disorder diagnostic assessments, with varying quality and recommendations. The substantial variation likely stems from insufficient evidence supporting assessment practices. Research is required to close the evidence gaps and inform high-quality clinical guidelines.

Introduction

The rise in the prevalence of autism spectrum disorders (ASD), now estimated at 1 in 68 (Centre for Disease Control National Center on Birth Defects and Developmental Disabilities, 2014), has created an increased demand for ASD diagnostic assessments. Clinical guidance documents play an important role in shaping these diagnostic assessment practices. There have been numerous guidelines published with recommendations for diagnostic assessments for ASD, for governmental jurisdictions (Dua, 2003; National Collaborating Centre for Women’s and Children’s Health, 2011) as well as for professional associations (Filipek et al., 2000; Johnson et al., 2007). A recent review focusing on tools used in determining the diagnosis found that guidelines varied in their recommendations (Canadian Agency for Drugs and Technology in Health, 2013).

To date, a comprehensive review and critical analysis of ASD diagnostic guidelines evidence base, quality, and content, and discussion of implications for practice have not been performed. A critical analysis of both quality and content of these documents to expose variation and areas of disagreement and uncertainty is essential as these documents have a significant impact on clinical practice. Performing a systematic review helps to ensure that future clinical guidance for ASD is transparent, rigorous, and applicable. The objective of this study was to perform a systematic review of clinical guidance documents for ASD diagnostic assessment, comparing and contrasting their quality and content.

Methods

Literature search

The systematic review of guidance documents included clinical practice guidelines published by Canadian, American, and United Kingdom health and educational professional associations whose members may participate in the diagnosis of ASD, as well as guidance documents published by governments representing English-speaking single-payer publicly funded healthcare systems. The definition of a guidance document was taken from the Institute of Medicine definition of clinical practice guidelines, which are “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” (Institute of Medicine of the National Academies, 2011).

The systematic review extended from 1 January 2000, when the Diagnostic and Statistical Manual of Mental Disorders (4th ed., text rev.; DSM-IV-TR) was published (American Psychiatric Association (APA), 2000) to October 2015 although the emphasis was on reporting current documents that guide ASD diagnosis. The search terms used included ASD, which is the current diagnostic label under the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; APA, 2013) and diagnostic terms in the DSM-IV-TR (APA, 2000) including autism, autistic disorder, pervasive developmental disorder (PDD), Asperger’s syndrome, and PDD not otherwise specified (PDD-NOS). These were combined with the Medical Subject Heading “Diagnosis.” Where possible, the publication type was limited to Government Publication, Guideline, Legislation, or Practice Guideline.

Guidelines were identified by searching the websites and journals of relevant professional associations, searching health, psychology, and education citation databases (including MEDLINE, EMBASE, PsychINFO, Cumulative Index of Nursing and Allied Health Literature, and Educational Research Information Clearinghouse), as well as searching the National Guideline Clearinghouse and the Grey Matters grey literature search tool (Canadian Agency for Drugs and Technology in Health, 2014). A full search strategy is outlined in Supplementary Material 1. Titles and abstracts were screened for eligibility by a single reviewer (M.P.) based on their applicability to ASD diagnosis for pre-school aged children (children less than 6 years of age). Documents identified as relevant underwent full-text review by a single reviewer (M.P.) with application of the inclusion and exclusion criteria as specified in Table 1.

Table 1.

Inclusion and exclusion criteria.

Inclusion criteria English guideline documents for “diagnosis” of ASD for children under 6 years Documents related to services for older children that still include age group of children less than 6 years of age Documents pertaining to intervention if eligibility criteria for intervention programs required certain diagnostic tests or assessment models Documents pertaining to screening for ASD as screening processes are likely to influence the number of children requiring a diagnostic assessment

Exclusion criteria Documents that were not clearly identified as guidelines and did not include both a systematic review and recommendations for practice Documents specific to individuals aged 6 years or older Documents pertaining only to intervention and other ASD management issues Published literature reviews and commentaries, or summaries of other published guidelines were excluded to limit the scan to documents most likely to guide clinical practice

ASD: autism spectrum disorder.

Data extraction

Relevant information was extracted from each document using a custom Data Collection Sheet for: Government Documents (Supplementary Material 2) or Professional Associations (Supplementary Material 3). If elements of the policy were unclear to the reviewer, the document was reviewed with other team investigators and attempts were made to contact the relevant government agency or professional association.

Quality appraisal

The quality of the guidance documents was assessed for quality by two independent reviewers (M.P. and L.Y.A.) using the Appraisal of Guidelines Research and Evaluation, second edition (AGREE-II) tool (Brouwers et al., 2010). The AGREE-II is an internationally recognized tool that assesses the quality and reporting of practice guidelines. It has 23 items divided into six domains: Scope and Purpose, Stakeholder Involvement, Rigor of Development, Clarity of Presentation, Applicability, and Editorial Independence, along with two global rating scales. Each item is scored on a scale of 1 (strongly disagree) to 7 (strongly agree). Agreement between appraisers was calculated using an intra-class correlation coefficient. If scores assigned by the appraisers for an individual document were not within 2 points of agreement, a face-to-face discussion of the pertinent elements of the document occurred to reach consensus. The domain scores were summed and a scaled domain score was determined for each domain using the formula

$$\mathrm{Scaled\; domain\; score}=\frac{\mathrm{Obtained\; score}-\mathrm{minimum\; score}}{\mathrm{Maximum\; score}-\mathrm{minimum\; possible\; score}}$$

An overall score of 1–100 for each document was determined from the mean of the scaled domain scores and was used to rank the documents. Descriptive statistics were used to describe the quality of the documents. Comparisons were made between quality of documents based on associations and jurisdiction.

Content analysis of guidance document systematic review

A content analysis of the guidance documents was conducted and the process for diagnostic assessment and impact of a positive diagnosis on eligibility for publicly funded ASD services was delineated for each document. One reviewer (M.P.) reviewed each document and recorded relevant information on the data extraction sheets (Supplementary Materials 2 and 3). Key elements compared and contrasted included which personnel can diagnose ASD, the use of multidisciplinary team (MDT) assessment, recommended psychometric assessment tools, and eligibility requirements for funding or services, including whether the child can access services with a provisional diagnosis, defined as an unconfirmed diagnosis of ASD that is communicated to the family. These core elements were summarized and tabulated, stratified by organization type.

Results

A total of 839 unique documents were retrieved using the chosen search terms, and screened for relevance (Figure 1). Of those, 26 documents were determined to be eligible and underwent full-text review; 11 of those met the inclusion criteria (Table 2). Two documents were no longer endorsed by their professional associations. These included the 2001 document published by the American Academy of Pediatrics (AAP) which was replaced by a subsequent document in 2007, and the American Speech-Language-Hearing Association (ASHA) guideline which was rescinded in 2015.

Figure 1.

The initial search strategy identified 869 guidance documents, of which 30 were duplicates. The remaining 839 documents underwent screening based on review of the title and abstract. In all, 26 documents met the eligibility criteria. Upon full-text review, 15 were excluded because they did not contain clear guidelines, were summaries, or were irrelevant to diagnosing ASD in the targeted age group. The remaining 11 guidelines were included in the systematic review (n = number).

Table 2.

Title, year, and type of guidance documents used in review.

Guideline, year	Type of document	Abbreviation
American Academy of Neurology, 2000	Practice parameter	AAN
American Academy of Pediatrics, 2001	Policy statement	AAP 2001
British Columbia, 2003	Standards and guidelines	BC
American Speech-Language-Hearing Association, 2006	Guideline	ASHA
American Academy of Pediatrics, 2007	Clinical report	AAP 2007
Scottish Intercollegiate Guidelines Network, 2007	Guideline	SIGN
New Zealand Ministries of Health and Education, 2008	Guideline	NZ
Miriam Foundation, 2008	Best practice guidelines	Miriam
American Occupational Therapy Association, 2009	Practice guideline	AOTA
National Institute for Health and Care Excellence, 2011	Guideline	NICE
American Academy of Child and Adolescent Psychiatry, 2014	Practice parameter (patient-oriented)	AACAP

A list of the type and scope of guidance documents can be found in Supplementary Material 4. The identified guidance documents had a wide scope, which ranged from providing aims, ideals, and guidance for clinical practice (such as in the Miriam Foundation and AAP guidelines) to providing minimum standards required for diagnosis in a particular jurisdiction (such as the British Columbia (BC) guidelines). The main difference between a recommendation and a requirement in the guidance documents was that a requirement determines eligibility for ASD-related services and funding.

Quality appraisal

The intra-class correlation coefficient for the two independent reviewers was 0.73 representing strong agreement. The scaled domain scores and rankings of all included documents are presented in Table 3. The mean total score on the AGREE-II for all included documents was 66.8 (standard deviation (SD):12.8, range: 44.3–94.1). The three documents with the highest overall scores were the National Institute for Health and Care Excellence (NICE) guideline (National Collaborating Centre for Women’s and Children’s Health, 2011), New Zealand (NZ) guideline (New Zealand Ministries of Health and Education, 2008), and the Miriam Foundation Guidelines (2008).

Table 3.

Quality analysis: summary of AGREE-II scores and rankings for guidance documents.

Guideline, year	D1: Scope and purpose		D2: Stakeholder involvement		D3: Rigor of development		D4: Clarity of presentation		D5: Applicability		D6: Editorial independence		Overall
	Score	Rank	Score	Rank	Score	Rank	Score	Rank	Score	Rank	Score	Rank	Score	Rank
AAN, 2000	83.3	9	64.6	7	59.5	3	80.5	7	8.33	11	91.7	3	64.7	7
AAP, 2001	75.0	11	52.1	9	13.1	11	77.8	9	39.6	7	8.3	11	44.3	11
BC, 2003	94.4	4	50.0	10	36.9	9	83.3	5	62.5	3	41.7	9	61.5	8
ASHA, 2006	94.4	4	62.5	8	38.1	8	80.6	6	29.2	8	100.0	1	67.5	5
SIGN 2007	97.2	1	89.6	2	58.3	4	80.5	7	75	2	16.7	10	69.6	4
AAP, 2007	86.1	8	50.0	10	33.3	10	75.0	10	50.0	4	45.8	8	56.7	10
NZ 2008	94.4	4	87.5	3	58.3	4	86.1	4	45.8	5	91.7	3	77.3	2
Miriam, 2008	97.2	1	85.4	4	73.8	2	94.4	2	45.8	5	50.0	6	74.4	3
AOTA, 2009	88.9	7	77.1	5	50.0	7	63.9	11	20.8	9	50.0	6	58.4	9
NICE, 2011	97.2	1	91.7	1	95.2	1	97.2	1	83.3	1	100.0	1	94.1	1
AACAP, 2014	83.3	9	68.7	6	55.9	6	91.7	3	16.7	10	83.3	5	66.6	6
Overall guidelines	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
	90.1	7.4	70.8	16.2	52.0	21.9	82.8	9.4	43.3	23.8	61.7	33.1	66.8	12.8

Analysis of the domain scores from the AGREE-II showed that in general, the documents were high quality in terms of Scope and Purpose (mean score: 90.7, SD: 7.4) and Clarity of Presentation (mean score: 82.8, SD: 9.4); these domains also had the lowest variation in scores. Scores were lowest in Applicability (mean score: 43.3, SD: 23.8) and Rigor of Development (mean score: 52, SD: 21.9). There was considerably more variability in scores in these domains, as well as in the domain of Editorial Independence, with an SD of 33.1 (Table 3).

Seven of the 11 documents provided a search strategy; both AAP documents and the BC document did not. The New Zealand (NZ) guideline’s diagnostic section adopted the recommendations of a 2003 guideline developed by the National Autistic Society of the UK (which has been replaced by the NICE guideline and is no longer available; National Initiative for Autism: Screening and Assessment, 2003). Four of the documents noted the inclusion of non-empirical materials such as other guidelines, noted in the Miriam Foundation and BC guidelines, and book chapters, noted in the American Academy of Neurology (AAN) and American Academy of Child and Adolescent Psychiatry (AACAP) practice parameters. Only four of the documents provided graded recommendations, the Scottish Intercollegiate Guidelines Network (SIGN) and NZ guidelines and the AACAP and AAN practice parameters. Of these, the AAN excluded diagnostic assessment from graded recommendations and the AACAP permitted clinical consensus to influence the strength of the recommendations.

Documents were compared based on the type of organization/institution that produced them. The mean total AGREE-II score for the six guidelines produced by professional associations was 58 (SD: 9.3). The mean of the five guidelines that were not produced by professional associations (four from governments, one from a non-governmental organization) was higher at 75 (SD: 10.8).

Content analysis of guidance documents

The diagnostic assessment recommendations for each of the guidelines are summarized in Table 4.

Table 4.

Content analysis: diagnostic recommendations in guidance documents.

Document	Year	Target audience	Age target	Wait time target	Clinicians who can diagnose	MDT recommended	Recommended assessments	Optional assessments	Tools recommended
Professional associations
AAN	2000	NSa	NS	NS	NS	Yes	Cognitive SLP if child fails language screening	OT Neuropsych Behavioral Academic	Yes, at least oneb
AAP	2001 (2007)	Pediatricians	NS	NS	Physician comfortable conducting a comprehensive evaluation	Ideally	Physical examination Audiology SLP “Some measure of overall cognitive functioning and adaptive skills”	NS	Ideally
ASHA	2006 (2015)	SLPs	NS	NS	Specifies experienced SLPs can diagnose	Ideally	Audiology “Appropriate individual and collective expertise”	“Appropriate referrals to assess needs and comorbidities”	No
AAP	2007	Pediatricians	NS	NS	Physician Psychologist SLP	Ideally	NSc	NS	Ideally
AOTA	2009	OTs OT assistants	NS	NS	NS	Yes	NS	NS	NS
AACAP	2014	Child and adolescent psychiatrists	NS	NS	NS	Yes	Medical Cognitive SLP	OT Physical therapy	No
Governments
BC	2003	Professionals involved in screening, identification, assessment, and diagnosis of young children with ASD	NS (applies to children under age 6)	6 wks	Pediatrician Clinical psychologist Child psychiatrist	Yes	Clinical diagnostic assessment Psychological assessment SLP Medical	OT “Family assessment” Psychiatry Additional specialty assessments	ADI-R; ADOS or CARS
Scotland (SIGN)	2007	Healthcare professionals and others involved in the care of children with ASD	Up to age 18	NS	Healthcare professionals	Yes	SLP “Specialist” healthcare professional	Intellectual Neuropsych Adaptive functioning	Consideration of history-taking and observation tools
New Zealand (NZ)	2008	Primary care practitioners, education professionals, policy makers, funders, parents, carers, specialists, and other ASD stakeholders	Lifetime	6 mos	Healthcare practitioner	Ideally	Pediatrics Audiology	Psychiatry Psychology SLP OT	Standardized interviews and schedules should be used
UK (NICE)	2011	Professionals involved in screening, identification, assessment, and diagnosis of young children with ASD	NS	3 mos	Diagnosis conferred by MDT consisting of core team members	Yes	Core team members: Physician (pediatrician/psychiatrist) Psychologist SLP	Optional team members: Pediatrician Psychiatrist Neurologist Educational psychologist Clinical psychologist OT Nurse Specialist teacher Social worker	No
Non-profit associations
Miriam	2008	Clinicians who screen for or diagnose ASD	NS	5 mos	Physician Psychologist	Yes	Medical Cognitive	Audiology Behavioral Dietician Education Neurological NP OT Pediatrician Psychiatrist Psychologist Social worker SLP	ADI-R and ADOS

Guidelines are grouped by type and displayed in chronological order.

AACAP: American Association of Child and Adolescent Psychiatrists; AAN: American Academy of Neurology; ADI-R: Autism Diagnostic Interview–Revised; ADOS: Autism Diagnostic Observation Schedule; AOTA: American Occupational Therapy Association; ASD: autism spectrum disorder; ASHA: American Speech-Language-Hearing Association; SIGN: Scottish Intercollegiate Guidelines Network; CARS: Childhood Autism Rating Scale; MDT: multidisciplinary team; mos: months; Neuropsych: neuropsychological assessment; NP: nurse practitioner; NS: not specified; OT: occupational therapy; SLP: speech language pathology; wks: weeks; ( ) indicates year the guideline was rescinded, if applicable.

^aEndorsed by other professional associations including AAP, AOTA, the American Psychological Association, ASHA, and the Society for Developmental and Behavioral Pediatrics.

^bRecommended instruments: the Gilliam Autism Rating Scale; the Parent Interview for Autism; the Pervasive Developmental Disorders Screening Test, Stage 3; the ADI-R; CARS; the Screening Tool for Autism in Two-Year-Olds; ADOS.

^cRecommended elements: (1) health, developmental, and behavioral histories; (2) physical examination; (3) developmental and/or psychometric evaluation; (4) categorical DSM-IV-TR diagnosis; (5) assessment of parents’ knowledge of ASD; and (6) laboratory investigation to search for a known etiology or coexisting condition.

Recommendations for MDT assessment

The guidelines were generally very supportive of diagnosis using MDT. Seven guidelines recommended MDT, while the four remaining state that MDT should ideally be used. Among the non-professional association guidelines, the Miriam Foundation, SIGN, and NICE guideline all recommend MDT, the NZ guideline states it is preferable, and the BC guideline requires MDT for diagnosis to be eligible for provincially funded intervention.

The most commonly cited reason for MDT assessment for ASD is the need to develop the neurodevelopmental profile of the child’s strengths and weaknesses (suggested in the NICE, Miriam Foundation, and BC guidelines)—an element of the assessment that does not necessarily influence the diagnostic determination of ASD. An additional reason in the Miriam Foundation document for the need for MDT is the consideration of possible alternative diagnoses. A related indication presented in the AACAP and Miriam Foundation documents is to evaluate the child for co-occurring conditions in ASD. ASD can be associated with a number of medical conditions, as well as co-occurring intellectual disability, other learning challenges, or psychiatric and behavioral disorders (Anagnostou et al., 2014), and there is an accompanying concern noted in these documents about ASD “overshadowing” other diagnoses.

Despite multiple recommendations for MDT assessment, there is little to no accompanying empiric evidence suggesting that a MDT assessment is more accurate than that of a solo clinician or more capable of developing a comprehensive list of differential diagnoses. For example, the AAN and AACAP documents contained evidence-informed recommendations for screening, surveillance, laboratory investigations, and other investigations, but recommendations specific to the diagnostic assessment were based on clinical consensus. The NICE guideline is the only one to describe and critique the empirical evidence for MDTs. In the study by Mahoney et al. (1998), the diagnosis made by a solo child psychiatrist was compared to a case review by a team of three expert raters who had access to the assessment, but not the diagnostic formulation, that is, the team did not directly assess the study participants. That study occurred during the era of multiple diagnostic subtypes of ASD measured agreement between the solo child psychiatrist and the MDT on their classification of ASD diagnostic subtypes. Of note, the agreement on non-ASD diagnoses was excellent (kappa = 0.81). Although the classification of ASD diagnostic subtypes is no longer relevant, the question of the quality of a solo practitioners assessment compared to a MDT remains an important one.

While there was generally broad clinical consensus to support the recommendations (including the “clinical standard” rating for MDT assessment by the AACAP), some of the guidelines noted cases where MDT assessment may not be necessary. The scientific committee who advised on the Miriam Foundation guideline argued that flexibility is needed in the diagnostic approach in “cases in which the diagnosis is obvious” and also for cases where the practitioner cannot easily access a MDT assessment (The Miriam Foundation, 2008). The AAP and NZ documents note that ideally, the assessment is completed by a MDT, but also leaves room for experienced solo practitioners to perform the assessment (Johnson et al., 2007; New Zealand Ministries of Health and Education, 2008). The now-rescinded ASHA guideline contained the most notable departure from the requirement for MDT. While still stating that MDT assessment is ideal, it also stated that, “the SLP who has been trained in the reliable and valid use of diagnostic and assessment tools as well as in the clinical criteria for ASD may be qualified to diagnose these disorders as an independent professional” (American Speech-Language-Hearing Association, 2006).

Recommended personnel for assessment

The documents varied with respect to their recommendations for personnel participating in the diagnostic assessment, even when a MDT is recommended. The most common recommended personnel are physicians to perform a medical assessment, a speech language pathologist (SLP) for a language assessment, and a psychologist for a cognitive assessment. All three of these were recommended in five of the guidelines. Some of the statements did not clearly define the type of professional that needs to be involved, such as the statement in the 2001 AAP document which recommends “some measure of overall cognitive functioning and adaptive skills” but stops short of stipulating a formal assessment performed by a psychologist (Committee on Children With Disabilities, 2001).

Five of the guidelines listed specific personnel or types of assessments that may be of clinical value in the diagnostic process. Occupational therapists (OTs) were the most commonly mentioned optional assessment and were listed for the purpose of assessing sensory processing, which is often affected in ASD. Of note, the American Occupational Therapy Association guideline does not recommend participation of OTs in all diagnostic assessments for ASD, stating only that the OT should “understand the team structure and his or her role as a member of the team” (Tomchek and Case-Smith, 2009). Beyond OTs, there was a wide range of types of professionals that could be involved, including nurses, teachers, social workers, dieticians, and others. The clinical indications for the other listed optional assessments were not clearly stated in any of the guidelines.

Tools in the assessment

One of the greatest areas of variation in the guidelines was the recommendation for psychometric tools to inform an ASD diagnosis. The AAN guideline recommends at least one tool be used. The BC guideline requires multiple tools, including the Autism Diagnostic Interview–Revised (ADI-R) and the Autism Diagnostic Observation Schedule (ADOS), in order to access provincially funded interventions. The Miriam Foundation guideline recommends that the ADI-R and ADOS be considered the “gold standard for diagnosis of ASD in Canada,” but do allow leeway by stating that “a lack of ADI-R, ADOS data should not prevent a child from receiving much needed services if a diagnostician with sufficient expertise conducts the assessment” (The Miriam Foundation, 2008). The ASHA guideline does not endorse specific tools, but in their recommendation that SLPs can independently diagnose ASD, they limit this ability to SLPs “trained in the reliable and valid use of diagnostic and assessment tools” (American Speech-Language-Hearing Association, 2006). Both AAP documents suggest that diagnostic tools should ideally be used, but stop short of recommending them for all ASD diagnoses. The AACAP document lists available tools, but highlights the overarching importance of clinical judgment: “As a practical matter, all these instruments vary in their usefulness for clinical practice. Some require specific training. The use of such instruments supplements, but does not replace, informed clinical judgment” (Volkmar et al., 2014).

The NICE guideline was the only document to provide a critical analysis of the evidence base for the tools. All of the identified studies evaluating the tools were deemed to be of very low quality. As a result, the NICE guideline authors concluded that the clinical benefits of using these tools remain uncertain although they did acknowledge the assistance these tools could provide in performing a systematic, ASD-specific semi-structured interview and observation during a diagnostic assessment. There was also consideration given to the possibility of harm that could arise from relying solely on the score provided by the tool. In consideration of all of this, NICE recommends the use of a semi-structured interview and observation for ASD but does not recommend a specific tool.

The diagnostic assessment and eligibility for ASD services

Diagnosis is often essential for access to ASD-specific services, and some of the documents addressed the link between assessment and access in their recommendations. The more recent AAP document and the AAN practice parameter endorse referral to early intervention upon a positive screen for ASD. These guidelines all pertain to US jurisdiction, in which federal law requires that the local early intervention agency (for children under 3 years of age) or school district (for children aged 3 years or older) provides a diagnostic evaluation within 45 days of receiving parental consent for children with suspected developmental concerns (US Department of Education, 2004).

The guidelines pertaining to non-US jurisdictions were less uniform in their links between diagnosis of ASD and eligibility for services. This issue is generally not covered in the NICE, NZ, and SIGN guidelines, which may be related to the infrastructure for service delivery for ASD in these jurisdictions. Eligibility for provincial publicly funded intervention is listed in the BC guideline introduction as one of the primary motivations for development of standards for diagnosis. The Miriam Foundation’s recommendations are among the more rigorous with regard to recommended elements of assessment, yet they are careful to note that children should not be denied access to services if the ADI-R and ADOS have not been completed, provided an experienced diagnostician has done the assessment.

Target wait times for assessment

None of the professional association documents provided maximum suggested wait times for assessment. In these documents, the need for early identification (facilitating access to early intervention) is largely discussed in sections dealing with screening and surveillance. Here again, US service delivery structures that do not require a formal diagnosis to access early intervention services place the emphasis on access to screening and referral to intervention, instead of providing a suggested wait time for a definitive ASD assessment. Four of the five documents pertaining to non-US jurisdictions differed in that they do provide suggested maximum wait times for a definitive ASD assessment, albeit with no supporting evidence for these recommendations, or clear general consensus (Table 5).

Table 5.

Wait times suggested by in guidance documents.

Guideline	Suggested wait times
The Miriam Foundation	Maximum of 3 months from referral to start of diagnostic assessment Maximum of 2 months from start of assessment to communication of results
BC Guidelines	1 month to specialized assessment (the intermediate step toward a diagnosis) 6 weeks to diagnostic assessment (from the time of referral from the specialized assessor) 3 months to completion of the assessment (from the time of referral from the specialized assessor)
NICE guideline	3 months from referral to MDT assessment
NZ guideline	6 months from referral to diagnostic assessment

MDT: multidisciplinary team; NZ: New Zealand; NICE: National Institute for Health and Care Excellence.

Discussion

This systematic review is the first to demonstrate inconsistencies in recommendations pertaining to all aspects of the ASD diagnostic assessment, including whether ASD must be diagnosed by a MDT, the composition of the MDT, and the time frame for completion of the assessment. The review also found discrepancies in the recommendations pertaining to screening and diagnostic tools. This corroborates the findings of the CADTH Rapid Response Report (2013), which used four guidelines (all of which were used in this review) to specifically evaluate screening and diagnostic tools for ASD.

The guidance documents demonstrated higher quality in Scope and Purpose and Clarity of Presentation. The documents generally identified their target users, the target population, and the health questions with clarity. Similarly, the recommendations were often clearly written and easily identifiable. These are important domains, making the documents and their recommendations easier for target users to identify.

The quality appraisal demonstrated low quality in Rigor of Development and Applicability. The poor Rigor of Development is concerning given that one of the fundamental principles of clinical guidelines is that they should be based on a systematic review of evidence, and that this evidence should inform the strength of recommendations (Institute of Medicine of the National Academies, 2011). The documents in this study handle the range of empirical evidence supporting recommended diagnostic practices in various ways. For example, the NICE guideline provides a thorough critique of the available evidence and is transparent in the reasons for its decision to recommend MDT assessment in the absence of quality evidence, namely, the development of a comprehensive and holistic profile of the child’s strengths and weaknesses. In comparison, the AAN guideline places its recommendations for MDT assessment in a section outside of the evidence-based recommendations, also excluding this section from the “recommendations for research” attached to the evidence-based recommendations in the guideline (Filipek et al., 2000). The AACAP guideline allows for its strongest recommendations, “clinical standards,” to include “overwhelming clinical consensus” (Volkmar et al., 2014). The variation in recommended practices is likely attributable to the paucity of empirical evidence available to support the various components of the ASD diagnostic assessment.

The low applicability scores across all documents are similarly concerning given their potential influence on systems of care. While most documents discussed some challenges to their implementation, only the NICE and AAP 2007 documents provided tools or reimbursement strategies to facilitate implementation. Few documents, particularly those published by professional associations, dealt with health systems issues related to diagnosis, such as access to services and supply of providers. Those that did provide recommended maximum wait times provided little guidance as to how these could be achieved.

The content analysis demonstrated a high degree of variation in the recommendations made by the included documents. In addition to the lack of empiric evidence to support practices, this variation may also reflect the substantial heterogeneity in clinical presentation and developmental trajectory associated with ASD. It may be that a one-size-fits-all approach to diagnostic assessment does not provide sufficient flexibility to accommodate for this heterogeneous population. Children with a more severe presentation of ASD may not require multiple assessors and diagnostic tools to confer a diagnosis with acceptable accuracy. In such cases, inflexible recommendations and requirements for assessment may add inefficiency to an already strained system. Guidelines that are inflexible, “can harm by leaving insufficient room for clinicians to tailor care to patients’ personal circumstances and medical history” (Woolf et al., 1999). This should be balanced with concerns of allocative efficiency for behavioral interventions, ensuring that these costly services are not delivered to those with an inaccurate diagnosis of ASD. Future empiric evidence must incorporate this clinical heterogeneity, evaluating the tradeoffs between guaranteeing a comprehensive assessment for all children with suspected ASD and providing a more streamlined approach that is tailored to the child’s presentation. Until such empirical evidence is available, guideline developers must be transparent about the lack of evidence to support various aspects of diagnostic assessment. The use of formal approaches to achieve consensus, such as the Delphi method, represents the best alternative in the absence of evidence, and should be employed and described in future guidance documents for ASD (Graham et al., 2003).

Clinicians performing diagnostic assessment are left with the difficult task of determining how to proceed in the face of limited empirical evidence and disparate clinical guidance. In light of this, clinicians should be mindful of local resources and wait times, eligibility requirements for ASD services (which may include results from specified standardized tests), and the wishes of families when deciding on how best to assess for ASD.

This review had some limitations. The review examined key elements of diagnostic elements such as personnel, assessment tools, wait times, and eligibility for funding, but did not include all testing involved with ASD diagnosis. Additional testing which does not determine the clinical diagnosis of ASD, such as genetic testing, was not included as this information does not yet influence systems of eligibility for ASD services. Second, the systematic review of guidance documents was limited to professional associations and English-speaking countries with single-payer, publicly funded health care systems. As a result, some guidance documents (particularly those from other countries or from US states) were not included. This was done to facilitate comparison within systems that employ a single-payer healthcare structure, which would be responsible for ASD diagnosis in pre-school aged children. Due to substantial differences between these systems and current health systems in the United States, these comparisons were not undertaken.

In conclusion, clinical guidelines on ASD diagnosis vary considerably in their quality and in the content of their recommendations. Many guidance documents had questionable rigor of development and limited applicability. Further empiric evidence is needed to support diagnostic decision making in ASD and should include analysis of systems impacts, such as wait times.