Deconstructing Diagnosis: Four Commentaries on a Diagnostic Tool to Assess Individuals for Autism Spectrum Disorders

Abstract

Diagnostic assessment tools are widely used instruments in research and clinical practice to assess and evaluate autism symptoms for both children and adults. These tools typically involve observing the child or adult under assessment, and rating their behaviour for signs or so-called symptoms of autism.

In order to examine how autism diagnosis is constructed, how diagnostic tools are positioned, and how their trainings are delivered, we paid for four places on a training course for a diagnostic tool. We asked the attendees (the first four authors) to each produce a critical commentary about their impressions of the training and the diagnostic tool itself. Their commentaries are published here in full. They have various disciplinary backgrounds: one is a social scientist, one an ethicist, one a psychiatrist, and one a developmental psychologist.

The commentaries are followed by a concluding section that summarises the themes, commonalities, and differences between their accounts of the training course. Authors differed as to whether the diagnostic tool is a useful and necessary endeavour. Nevertheless, all critiqued of the tool’s lack of transparency, recognizing context, emotion, and differences in interpretation and power imbalances as playing an unidentified role in the assessment process. Based on this project, we recommend that training for raters for such tools should be accessible to a wider group of people, and incorporate more explicit recognition of its own limitations and commercialisation.

Introduction

Diagnostic assessment tools such as the Autism Diagnostic Interview – Revised (ADI-R) and the Autism Diagnostic Observation Schedule Second Edition (ADOS-2) (Lord et al., 2012, 1994) are considered to be gold standard tools in research and clinical practice to assess and evaluate autism symptoms for both children and adults. These tools typically involve observing the child or adult under assessment, and rating their behaviour for signs or ‘symptoms’ of autism. If the rater considers that the child or adult in question displays frequent or severe symptoms of autism, they are given a high score, and if this score is beyond a certain cut-off point, they are classified as having autism. Such tools are routinely used in diagnostic assessment for both children and adults in clinics, in combination with developmental history, and other sources of information (Hayes et al. 2018). Researchers also use diagnostic tools to identify cases in a given population who qualify as having autism. However, non-clinicians, such as parents and autistic adults, rarely have access to such tools and lack knowledge of how diagnostic decisions are determined because raters require specialist training. In order to sit for a training course in the UK for ADOS-2, for example, trainees must show evidence they are a qualified clinician or practicing researcher. As such ‘who decides’ who has autism is limited to a highly qualified strata of society, perhaps necessarily: a small group of highly trained clinicians and researchers.

Our Wellcome Trust-funded project, called Exploring Diagnosis (www.ex.ac.uk/exdx), and inspired by calls for a ‘sociology of diagnosis’ (Jutel, 2011), studies how diagnostic decisions about autism are made, by whom and for what purpose. In order to examine how autism diagnosis is constructed, how diagnostic tools are positioned and how training is delivered, in autumn 2017 we paid for four places on a training course for ADOS-2, inviting four academics from different disciplinary perspectives to attend. We asked them to produce a critical commentary about their impressions of the training and the diagnostic tool itself (ADOS is considered the gold standard for diagnostic observational assessment but we could have chosen any similar diagnostic tool). The project was influenced by other work that looks at how narratives are constructed by clinicians when they use diagnostic tools to diagnose autism. For example, observing clinical decision making following administration of a psychoeducational measure has shown that clinicians use narrative to separate out autism as a diagnosis (Turowetz, 2015). We were also influenced by research showing how diagnostic tests operate to shape the environments both in terms of the interaction between test ‘subject’ and clinician, and in terms of the setting and atmosphere in which the test is conducted (Maynard and Turowetz, 2017). Such work suggests the nature of the diagnostic test itself may influence the diagnostic outcome. In addition, Hayes and colleagues (2018) have shown that diagnostic guidelines encourage clinicians making diagnostic decisions to consider many social and contextual factors unrelated to whether a child or adult displays symptoms of autism as measured by such tools. We wondered whether the diagnostic tools themselves might promote the construction of diagnostic narratives (and if so, how). This is important because when diagnostic decisions are made by multidisciplinary teams, the ‘symptoms of autism’ that a child or adults displays (generally measured with ADOS or another similar diagnostic tool) are often read as an objective measure of autism.

A second reason for embarking on this small project was to counter the lack of transparency about how diagnostic ratings are obtained in clinical practice. Having said that, of the four commentators who sat the course, only one was a practicing clinician. The four were selected because each has, for different reasons and in different ways and contexts, expressed a critical interest in the medicalisation of autism. These four academic authors are also members of the autism community (defined as autistic people, their families, and professionals who work with them by Kenny et al., 2016). Two of the commentators are autistic activists, two were parents of children identified as being on the autism spectrum, and one a clinician noted for his critical perspective. All have been vocal campaigners or commentators, engaging with social and scientific institutions to both promote or resist the framing of autism as pathological, so they have all been actively involved in processes of medicalisation and/or de-medicalisation (Jenkins and Short, 2017). Given that the commentators were invited because of their distinctive history in relation to autism, we do not claim the views they recorded are representative of any particular group (all parents, for example). As diagnostic training is limited to a select band, we were keen to invite people with an alternative perspective who were likely to question whether the diagnostic tool really can be considered an objective measure of autism. That being said, it is important to note, as one of the reviewers put it, these commentaries are ‘no more objective than the instrument they are examining’. The authors are all known for positions allowing that autism is both a social construct and an actual neurodevelopmental difference, and we acknowledge that their views and the knowledges about autism are situated in their considerable combined experience (Haraway, 1988).

The following accounts, then, are personal reflections of the commentators on the training process for ADOS and the tool itself, written by these authors soon after they sat in on the ADOS training course. The critical commentaries are published here in full. The authors were asked to observe and make notes, especially in light of the language of the course and power relations diagnostic assessment entails. The training took place over several days, was intensive by design, and covered an introduction to the course and four separate modules of the diagnostic tool differentiated by children and (young) adults’ age and level of expressive language. The training incorporated watching filmed clips and grading children according to the set criteria, and finally a review of how to apply the tool in practice. These four authors independently wrote their own reflective commentaries without reference to, nor any consultation with, each other. This was so that we captured their individual and disciplinary perspectives, to express only their own diverse views. The accounts that follow show how diagnostic assessment tools are used to quantify individuals’ behaviour and include arguments about how this may serve to both hinder and help their development. The accounts are followed by a concluding section that summarises the themes, commonalities and differences between these accounts.

1. A window on the Autism Industry

Sami Timimi is a consultant child and Adolescent Psychiatrist.

In May 2017 I attended, along with 13 other participants, training in the use of a diagnostic scale for autism. The cost per delegate for this course is over £500. The full extended training takes 4-5 days and costs over £1000. The training is advertised as being an approved course after successful completion of which you are ‘licenced’ to administer on patients the assessment. However, in order to administer the assessment you must also purchase an administration pack, which contains 50 assessment booklets (as well as other materials you might need) and costs nearly £2000. At the start of the course each participant also received a booklet advertising numerous other courses and assessment packs available for purchase. Any doubt that this is not a commercial ‘product’, and part of a wider money making industry were lessened when the course facilitator, in response to a question on what module you could use with adults who have little language or use typing for communication, answered that such a module had not been developed as it was “not commercially viable”.

I attended this course in the peculiar position of being both student and researcher; there to observe, document, and potentially deconstruct the assessment. As a consultant child and adolescent psychiatrist I frequently encounter young people who have been diagnosed with an Autism Spectrum Disorder (ASD) and the tool we were taught remains the most frequently used assessment ‘tool’ for establishing a diagnosis. I am aware that as an author who has written sceptically about the validity of ASD as a diagnostic concept, I would not be able to view the content of the course through a value neutral prism. Therefore, as the rest of this article represents my views of the assumptions, beliefs, and practices, related to the course and the tool, it will reflect my particular bias.

The tool is advertised as being a “semi-structured standardised measure of communication and social deficits and play associated with ASD“ (note the language of ‘standardised’, ‘measure’, ‘deficits’ depicting an empirically valid and quantifiable approach to identifying a medical problem). It is described as incorporating a series of “standard activities, providing opportunities to observe behaviours directly relevant to diagnosis of ASD at different developmental levels and chronological ages”.

Before presenting my reflections, it’s worth remembering the overall context, which reflects several levels of assumptions. This includes that ASD exists as a discrete, natural ‘thing’, that this autism ‘thing’ can be identified and quantified, that identification and measurement can be done validly and reliable through specific assessment tools, that the training was in use of one such tool, that the items in it encompass and identify the ‘symptoms’ that make up ASD, that it has good psychometrics to enable sensitivity and specificity, and that it is reliable and can be ‘standardised’. Each assumptive level is open to question. If any of these assumptions are unwarranted, then the validity of the diagnostic tool is also open to question.

The training

The language used betrays deep assumptions that were implicit during the course. There was no acknowledgement that these assessments take place in a particular context (e.g. a medical clinic following concerns expressed by someone about a child) or that the behaviour of the examiner could have an impact on how the patient subsequently behaved. The assumption was that the context and examiner side of the relational dynamic in the assessment room are non-significant, so that what emerges during the assessment is purely the result of the interiority of the patient. Throughout the two days the language used betrayed this assumptive framework. For example, the tool can be ‘standardised’ and made ‘objective’, features that achieve a rating are ‘symptoms’, that the job of a ‘good assessor’ will be to ‘look for symptoms’. The language is one of objectification, words such as ‘objective’, ‘abnormal’, ‘standardised’, ‘measurement’, ‘symptoms’, ‘severe’ and so on, regularly appeared.

The quasi-autistic rigidity of seeing only ‘real’ internal qualities in the patients was a recurrent theme. Most of the questions I asked during the course stemmed from genuine puzzlement as to how certain patient behaviours in the videos, the course facilitator ascribed as solely the consequence of symptoms in the patient. The reaction of the facilitator to my questions were defensive, insisting that what was being observed could only be understood as being the manifestations of ASD spilling out into the session. The context and assessment process, they argued, were standardised, allowing for objective ratings to be made that were shaped by the impulses of the patient. Thus, as the facilitator guided us through the scoring, they kept referring to the ‘fact’ that this or that symptom occurred. Non-pathologising interpretations were not tolerated. We were all being trained to become examiners with a keen eye for noticing every minutiae of the not ‘normal’ about the patient.

There was a mechanistic view of how social interaction occurs and can be manipulated. The scenarios/tasks are set up as if the examiner, their actions, and the environment, exist as controllable variables with what emerges as irrefutably demonstrating the patients’ social abnormalities. In one clinical example, the course facilitator, discussing a previous patient’s interactions with his mother, demonstrated this one sided view. In this example the mother told this child, “Why do you never look at me?” The child then started to look at her. The mother now complained, “Why do you stare at me?” The confused boy now decided maybe he should learn to look at her and then away from her. His mother now complained, “Why do you move your eyes from one place to another?” According to this story, the boy eventually developed a complex formula for how long to look at her and away from her. In recounting this tale the facilitator made no comment about this mother’s role in this developing relational discomfort – it was all the result of this kid’s (at the time undiagnosed) ASD.

The symptoms and ratings

No allowance is made for the gender or cultural relevance of the activities/questions. The illusion of objectivity starts dissolving when you see the wording of what you are being asked to rate. For example, for rating “Stereotyped/idiosyncratic use of words or phrases” a mark of 2 (indicating high degree of abnormality) is given if “often uses stereotyped utterances or odd words or phrases, with some other language”. A mark of 1 (indicating some level of abnormality) is “Use of words or phrases tends to be more repetitive than that of most individuals at the same level of expressive language, but not obviously odd”. For, “Quality of social overtures”, 1 is “Slightly unusual quality of some social overtures. Overtures may be restricted to personal demands or related to the child’s own interests, but with some attempt to involve the assessor”, 2 is “Significant minority (or more) of inappropriate overtures; many overtures lack integration into context and/or social quality”. Note that words like ‘often’, ‘unusual’, ‘quality’, ‘some’, ‘odd’ and so on, all require an examiner to interpret – they do not lend themselves to establishing objective facts.

Reliance on the interpretive bias of the examiner was regularly revealed. For example in one video assessment we observed I saw the child regularly smiling, but the facilitator said this was not smiling but ‘smirking’. Furthermore, it is difficult to understand why that should be considered a medical ‘symptom’. Other behaviours rated included: ‘unusual’ use of words; quality of child’s attempt to initiate interaction; whether patient requests things from the examiner; not spontaneously giving toys or other objects to examiner; not showing toys or other objects (e.g. by holding them up) to the examiner; lack of flexible, creative use of objects (e.g. a doll) in a representational manner; unusual sensory interests; and so on, are all open to interpretive variation. All the ratings are of this nature. They bring up questions of where notions of appropriate/inappropriate, normal/abnormal, healthy/symptomatic etc. are derived from and that lead to the categorical diagnostic decisions.

My genuine interpretations using the scoring systems were quite different to that of what the facilitator explained were the ‘objective’ scores, particularly for the two younger patients in the videos for Modules 1 and 2. In fact when Module 1 video was shown, I was convinced that this was being shown to illustrate an assessment of a ‘normal’ child to show us the contrast.

Watching the videos of assessments

I found the videos of Module 1 and 2 assessments painful to watch. The examiner moves quickly from one activity to another, giving each activity a few minutes. In these modules the children were around 3 and 5 years old. The ‘objectification’ of their behaviours spoke to me more about power and a privileged construction of ‘truth’ than the discovery of anything intrinsic to the child. Both children seemed to me to become uncomfortable and in different ways uncooperative (yet still forced to comply), due to, at least in part, the context and unusual behaviour of the examiner. We could only comment on and then code the patient’s behaviour, but were not allowed to interpret the patient’s possible affect or relational/contextual nature of interaction, which looked quite distressing for the young patients at times. It seemed to me that this ‘test’ of social communication was done through setting up a deliberately provocative environment and expecting these young patients to acquiesce unconditionally to the examiner’s demands.

Thus, to me, these looked more like tests of social conformity to a bossy adult’s constantly shifting demands. In one video, early in the session, after the examiner removes the toys the patient was playing with, the patient stands with his back to the examiner and says “You’re not my friend”. The rest of the session plays out a complex interaction, part at times hilarious rebellion by the young patient, part engagement, part distressing to watch pressurising and detachment by the examiner. To me the peculiarly detached, at times repetitive questioning, at times exaggerated unnatural smiling and high-pitched squeaky childish voice of the examiner, appeared more unusual than that of the reaction of the child. However, using the tool this child had ASD and was apparently the ‘abnormal’ of the two actors.

Conclusion

This tool is a diagnostic trap. An invented (socially constructed if you prefer that language) assessment, for an invented set of symptoms that is subjective and lacks insight into role of context and the inter-subjective nature of relationships. It attempts to identify relational ‘deficit’ whilst demonstrating the instrument’s own lack of awareness about the nature of relationships. It relentlessly seeks to uncover evidence of ‘abnormalities’ and creates a context where the examiner can readily find it. It is a system that catches many in its net, from young kids who won’t do as the examiner instructs and in the way the examiner believes they should, to older ones who have an interesting turn of phrase. It is infected with the Western colonial arrogance where the creators, sellers, and now the many examiners who carry out these assessments, believe that they ‘know’ how the universal, culture, gender, sexuality neutral person should and shouldn’t function. They shamelessly promote and sell this around the world subjecting ever more children and adults to its perverse normalization/pathologisation agenda.

2. The tool that could not be used (by autistic people)

Damian Milton is a social scientist, autistic and the parent of an autistic child.

I approached this project with a strong sense of trepidation, and for many reasons I was indeed right to feel such apprehension. Not only was the training that I attended an intense programme with limited breaks, but the language utilised (particularly in the materials) was difficult to engage with without having an emotional reaction to it. This I found particularly problematic for a number of items utilised for assessing adolescents. Having said this, I would like to thank the individual that delivered the training for their efforts, and for what was no doubt a difficult group to work with.

Throughout the training a pathologised deficit model of autism was utilised, framed as a developmental disorder that one could establish through the accurate recording of atypical behaviours across a number of specified domains. Potential strengths of being autistic were absent from the discourse. This ontology was presented as factual, evidence-based and validated. The discourse produced regarding autistic personhood is therefore that of a disordered ‘biopsychosocial other’. This included for example recording autistic behaviour as “mechanical” and thus reifying the much critiqued machine-like metaphor often used to describe autistic people (Milton, 2014). Such a framing is reminiscent of Foucault’s (Foucault, 1973) concept of the ‘medical gaze’, but rather than suggesting the separation of mind and body in a dualistic fashion, the mind is reduced to body and behavioural contingencies deemed either normal or abnormal, irrespective of cultural influences.

“What we want to see is requesting behaviour, eye contact and other communicative behaviour” (trainer).

This quote indicates that the more abnormal, idiosyncratic, absent, overactive etcetera, a behaviour is seen to be, the more likely it will record a score. Of course, in some measurements, these are well defined, in others however, they are far less so, and are far more dependent upon the tacit intuitions of the professional administrating the test.

In order to be deemed competent enough to administer the test, a professional would need to score over 80% reliability with other practitioners. In this sense, reliability is built on a consensus of how to administer and interpret the behaviours of others. Upon completing the practice examples of coding on the training course, it became apparent that I was perhaps ‘top of the class’ in spotting anything to do with ‘sensory’ related activity, but consistently rated the people in the film clips as having more social insight and reciprocal interaction than I was ‘meant to’. It would be a very interesting test to see if autistic people reliably perceived such social interactions in this way. Of course, a pathologised account of such interpretations might suggest that an autistic person lacks the social insight and social skills to see them lacking in others, or one would have a lower personal perceptual threshold due to one’s own difficulties in this area. Or, one might see these instances as a mismatch of salience within social contexts, differences in expressive language, or disposition, more akin to the ‘double empathy problem’ that I have previously discussed (Milton, 2014; Wittemeyer, et al., 2012), referring to the mutual breakdown in reciprocity often found in interactions between autistic and non-autistic people, and the wider power dynamics this is situated within.

There was great emphasis throughout the training placed on it being seen as standardised, reliable, and objective. The materials were presented as the ‘gold standard’ of diagnostic tools, based on “international consensus”. However, a number of aspects of the programme could bring this into question. The trainer even suggested at one point that two of the items within the schedule were not objective but subjective and intuitive. One is meant to make brief notes whilst administering the test items, film recording was seen as optional, and the order of the tasks non-compulsory. It was said at one point that parents should not be present, yet this was contradicted by the practice experience of one of the trainees who attended the course. Taking such aspects of data collection into account, one could question both the reliability and standardised nature of the tool.

The algorithm used to calculate the overall score an individual receives is clear, but it is not exactly clear as to how each item and weighting was arrived at, nor thresholds for indicating what was described by the trainer as the ‘severity of symptomology’. The overall number of items used to define the threshold for autism contained nearly twice as many items for ‘social interaction’ compared to either ‘language and comprehension’ or ‘restricted interests and behaviours’. Such a framing of autism suggests a deficit model based on the model of a ‘triad of impairments’, with the strongest weighting being on deficits in perceived ‘theory of mind’. Such a framing is contested however by a number of autistic scholars (Chown, 2014; Lawson, 2010; Milton, 2012; Murray et al., 2005; Wittemeyer, et al., 2012) as well as research studies (Gernsbacher et al., 2017; Sasson et al., 2017; Sheppard et al., 2016) that suggest ‘theory of mind’ is dispositional and contextual in its performance.

Other than a brief mention regarding their being available an array of “treatment packages”, the importance of diagnosis and intervention did not enter into discussions to a great extent. The focus instead was directed toward accurate assessment and the tool’s reliability. The importance of diagnosis and what resultant ‘treatment’ might look like was somewhat implied however, particularly when one takes into account the theoretical and discursive framing of how autism was being accounted for in the training documents. This brief mention did however frame autism within a medical model and also suggested how such treatment was commercialised into ‘packages’.

“Anything the child does, we are so happy!” (trainer).

The more an individual performed in a typical fashion, the ‘better’ it was deemed for a person, perhaps by implication suggesting that non-autistic people are ‘better’ people? Such a discursive framing helps to fuel the use of remedial interventions (whether evidence-based or not), rather than looking at the multiple nuanced reasons why these observed behaviours may be occurring. None of the items rated in the scale on their own would indicate a diagnosis of autism, yet an overall high score on a number of factors would. It would be possible to meet the threshold showing very different presentations of autism, yet this is always defined in relation to the typically developing child, becoming a collective umbrella term for developmental trajectories of ‘othered’ humans.

Such othering led to phenomena such as echolalia being described as not meaningful. Autistic ways of being are described as “compensatory strategies”. Interests are described as “obsessions”, “fixated” and abnormal for not being able to “shift flexibly around the interests and behaviour of the administrator”, whilst the reverse is seen as unproblematic. The use of eye contact in conjunction with other behaviours described as “appropriate”, not to mention the discourse of “temper tantrums” or “empathy” in relation to cartoon characters. Behaviours recorded are done so from the perspective of a non-autistic pathological model, and therefore when “little sense of reciprocity” is recorded by the professional, the breakdown in mutual comprehension is seen as the fault of only the autistic person’s behavioural ‘deficits’. Girls on the spectrum were presented by the trainer, as to often have more ‘typical interests’, or interests in social science subjects “sparked by their underlying difficulties”, seemingly missing that I had announced that I was an autistic social scientist myself at the beginning of the training, and if I am not mistaken present as male.

Throughout the process, it was patently obvious that the amount of autistic input into the design of the tool was next to zero. Why else would “empathising comments from the administrator” be encouraged one might ask (such fakery if seen as such may not be taken kindly), or test administrators being required to be “larger than life” according to the training materials? Given that autistic people (as defined by the scale criteria) are more likely to have sensory sensitivities, perhaps reducing such mannerisms might be beneficial? Also, being clear and honest may be better than acting ‘empathetic’ to a person one has just met and is putting covertly through a medicalised behavioural assessment schedule? An important part of applying the various activities contained in the schedule highlighted in the training was the use of what were called ‘social presses’. This term applied to prompting social activities through providing cues which were thought to determine in advance a likely behavioural response in terms of a child’s profile in relation to stages of development. This would involve ‘scaffolding’ social interaction to see if a child / young person reacts as expected and with what level of support. The positionality of the professional administrating the tool is firmly framed however, as that of a non-autistic person. Perceptions of interaction, sensory differences, interests and behaviours all need to be coded in terms of deviation from the ‘typical’ norm. Therefore, the power relations presented by the tool places the clinician firmly at the centre.

In the definition of autism presented, autistic people would lack the social insight and intuitions needed to adequately assess it in others. It is certainly true, that my honest assessments would probably not reach the reliability standards required for someone to be seen as a competent user of the test materials (despite perhaps a more accurate assessment of cognitive and sensory differences). Within such a discursive framing, it would be impossible for an autistic person to be trained to be fully ‘reliable’ in administrating the test, leaving clinicians with the power not only to define “autism” to some extent, but being the gatekeepers of who can access the diagnostic category. Therefore, the positionality and theoretical framing of the assessment helps to create an intensely wide power divide with all the social and personal consequences that dynamic could lead to.

The training suggested that those administrating the test should always be “keeping the parent in mind” and their emotional reactions to the processes being undertaken. Little emphasis however was placed on the wellbeing of the ‘child’ under observation. An adult under such observation would likely feel patronised, if not dehumanised. One has to question the ethics of informed consent for the individuals and families who undergo such assessments. Professionals utilising the tool are advised not to give out the quantitative data to those under examination, thus controlling the systems that identify people as autistic or not. I would argue that such tools creates a contrived social situation embroiled in (yet often disguised) unequal power relations and reify a conceptualisation of autism which directly leads to the disempowerment of autistic voices.

Lastly, I would just like to mention the more than present ‘elephant in the room’ which stalked the training throughout, and that was the idealisation and reification of developmental ‘normalcy’. Perhaps my colleague on the training Sami Timimi would consider working with me on a follow-up to his previous text on the ‘myth of autism’ (Timimi et al., 2011): ‘The myth of normal’?

3. Picture the scene

Virginia Bovell is an ethicist and parent activist

I am one of a group people who are being trained to use a diagnostic tool for autism. We are watching a filmed interaction between a diagnostician and a small boy.

The boy is doing things with a laminated picture that you wouldn’t expect of a typically-developing child. He is placing it on his head, then he lifts it and waves it around. He seems to be experimenting with how it feels and how it looks at different angles as it moves through light and air. And he seems to be fascinated and extremely pleased with what he is doing.

My neighbour, a fellow trainee, and I start to laugh. We laugh with recognition and delight in the fun the boy is having. He is not reliant on approval from the diagnostician, and we appreciate his exuberance and his ability to see things differently.

It takes a while for me to notice the atmosphere in the room. It seems no one else is laughing.

Later, I reflect on what happened.

The first thing to point out is that the two of us who laughed both have autistic children. Was it this that helped us to celebrate some charming and commendable qualities in this boy’s behaviour, rather than rue them and pathologise them? Possibly. Yet our response doesn’t just say something about us as parents - it says something about the nature of autism itself, which is described by many as integral to someone’s whole person, rather than being just a part (Grandin, 2006; Sinclair, 2012).

For example, if we had been watching most scenes in which a child was being assessed to identify a potential disability – visual impairment say – it would be cruel to laugh. If she was bumping into objects that she couldn’t see, then laughter would denote the most sadistic kind of slapstick humour. Indeed, I wonder if the others were thinking something similar about us, as my colleague and I laughed with this little boy, whose diagnosis of autism would shortly be confirmed. Did they see in us some kind of callous indifference at best, or taunting insensitivity at worst?

“I was laughing with empathy”, my colleague says to me later. In the classic phrase, we weren’t laughing at him, we were laughing with him. We were delighting in his single-mindedness, lack of conformity, freedom of spirit.

So what does this say about autism?

I am reminded of the phrase “the duality of autism and giftedness” (Walsh, 2010). This is the suggestion that the advantages and disadvantages in autism are two sides of the same coin, that there are real strengths alongside the deficits upon which diagnosis fixates. Yes, our boy may not be displaying much ability for social interaction, but on the other hand he may well be displaying enhanced perceptual awareness of the sight and sound and feel of the object he is manipulating (Mottron et al., 2006).

I am also reminded of Michelle Dawson’s account of a similar scenario. In a presentation in 2012 (Dawson, 2012), she spoke about how she, and she alone among a large audience, took pleasure in observing a child so fascinated and absorbed in his object of interest that he didn’t pause to interact with the human being beside him. Dawson likened the audience’s emotional response to a paragraph of emojis: row upon row of sad faces, interrupted with a lone smiley face. The smiley face was Dawson’s. Resolutely at odds with the others, she was alone in seeing something positive and natural in the child’s behaviour, in contrast to the others who were hung up on the absence of social interaction.

But if we only did things that other humans required of us, what kind of society would we be living in? There is evidence that autistic people may be less susceptible to social pressure, and therefore able to make more stable moral and perceptual judgments, (Cage et al., 2013; Izuma et al., 2011). Others (for example Wendy Lawson (Lawson, 2010, 2008)) cite the creative potential that comes with independent perception.

So at this point my instinct is to regard the diagnostic process as one that sidelines qualities of real ability and strength. By perseverating on reduced levels of social interaction, we seem only to be looking at what he’s not doing, rather than where he excels.

A couple of us try to point this out:

“Surely” I say to the trainer “ he simply wasn’t as interested in the person as he was in what he was doing”. And this, I suggest, chimes with the monotropism theory of autism (Murray et al., 2005) whereby the distinct cognitive style of autistic people relates chiefly to a paramount focus on interests, and this should not be devalued or ignored.

But here comes the reality check. The trainer tells us about the boy’s history, in the context of his social and institutional surroundings.

Imagine the world this boy inhabits, with its social pressures and rules, the constraints of school and the demands of the wider environment. Imagine the future that lies ahead of him. If this young chap is so single-mindedly averse to prompts and guidance from others - if he remains impervious to the social world of the NT majority and unaware of how to make his way in it - how will he stay safe, have his needs met, thrive? If he struggles to express himself and be understood, to the point of exclusion at best, and abuse and victimization at worst, what will become of him?

I understand the trainer’s point of view. But still it feels unfair to hone in on this boy’s deficits, given that his contribution is only part of what might lead to such potential difficulties. As Milton (Milton, 2012) has described it, there is a double empathy problem in operation, a mutual incomprehension. Or, as another colleague has put it, “Robinson Crusoe couldn’t have been autistic until he found Man Friday” (Starr, 2009). This speaks to the primary emphasis on social interaction that the diagnosis of autism requires. In this sense, autism is not only – or perhaps at all - an individual problem. It is all about the individual within a wider social environment. As many autistic writers and their NT sympathisers have written, people around the autistic person should also learn to adapt. We must – as it were - meet an autistic person halfway, since the problem is not so much within him or her, as it is within the mismatch between him/her and the wider world. “Empathy is not a one-way street” (Chown, 2013).

Given this focus on autism as a phenomenon requiring two or more people, is the individual-focused nature of the current diagnostic process inherently misconceived? How do you get a fixed measure of something that is by definition inter-relational?

Paradoxically perhaps, I begin to recognise that it is precisely because autism is about the relationship between an individual and their (social) environment that obtaining a diagnosis might help the boy. Without it, will there be any attempt to recognize a need for others to adjust their approach to him? Will he be viewed merely as willfully uncooperative and hence condemned to exclusion without support? What if, in a world dominated by NTs, his contrasting interests and the need for both him and others to adjust how they try to interact, goes by un-signposted and hence un-recognised??

And what about this boy’s future sense of his own self? What if the opportunity to find other people with similar dispositions is missed? This could happen, without the label that helps people find like-minded others. I am thinking here of the many first-person accounts from people who have acquired a diagnosis after years in the wilderness; the relief in finding they are not alone but are in fact part of a community with a shared sense of identity (Chapman, 2016; Dekker, 2016).

This search for others with shared experiences applies not just for the individuals themselves, but for their families. When a diagnosis comes, parents can look for and then find each other, to get help and information about how to build a case for the reasonable adjustments their child may need. With knowledge of the specific domains of challenge facing their children, parents can push for improved services, until such a time – if it ever does come - that their children are able to self-advocate. In other words, diagnosis can build ‘solidarity’ (Prainsack and Buyx, 2011).

Some might argue against this idea of solidarity, pointing instead to the divisions in the autism community. With its several dimensions, and its sometimes startlingly varying manifestations, we may of course be talking about autisms plural, rather than the singular monolithic entity that continues to elude natural and social scientists alike (Happé and Ronald, 2008; Müller and Amaral, 2017; Verhoeff, 2012). This had led (Chapman, 2017) to offer an analogy between autism and the feminist movement with regard to intersectional theory and differences: the priorities of learning disabled autistics and those with co-occurring conditions may not always be congruent with those of able, articulate autistics, just as the experiences of black women and white women within the feminist movement will not always be the same.

I believe there will be much more written about this in the coming years, but in the meantime, I still feel there is an important utility in the umbrella category of autism spectrum disorder. It crucially signposts the potential need for supports – even if the nature of these supports will vary according to individuals and contexts. If you start to chip away at diagnosis, you take away services - a point made very clearly by the furore surrounding a London service proposal to change the criteria (see (Dreaper, 2017).

Acquiring a passport to entitlements needs to done on a fair, transparent and non-arbitrary basis, and criteria that meet some semblance of objectivity are necessary also in the area of intervention studies. We all want to be assured that society is investing in effective support systems, not wasteful and harmful ones (see for example (Fleming et al., 2016). So, notwithstanding the many challenges, I have supported the call for more intervention research. We need to be able to trust the measures that have been used in such studies, such as the PACT research, which was widely commended for its adherence to good scientific standards (Mottron, 2017). But as a lay person reading about PACT, I was none the wiser about the precise nature of the tool used to measure the outcomes. And that is precisely why I welcomed the opportunity to learn about the diagnostic tool….

… which brings me back to the opening scenario in this article. Here I am, on this course, amongst a group of people who are learning what goes on in the diagnostic process and how to administer the tool. I like and trust my fellow trainees’ concern for their client group, and the trainer is clearly steeped not just in diagnostic protocols but in the real-life challenges facing her subjects, for whom she feels great affection. Everyone here is approaching the task in good faith. Setting aside the controversy over whether people can self-diagnose, it seems obvious that self-identification of autism is never going to be a viable or realistic way forward for many - young children for example, or adults with learning disabilities. So I do see the merit in our shared endeavour.

Still, I feel ambivalent. I’m uneasy about the medical dominance in the use of such tools with regard to autism. My involvement as a ‘lay’ person testifies to the fact that you don’t need to be employed in professions allied to medicine, to learn how to use the tools. And it is disconcerting that the diagnostic materials are copyrighted. People get a diagnosis that can change their lives, but accessing the criteria by which they are diagnosed is hidden behind a paywall. Permission to diagnose and assess becomes an adjunct of a wider medical/industrial system in which only certain people have power, and in which some people profit, while the people who go off and live their lives post-diagnosis seem relatively passive and powerless in the process.

And I am concerned about the extent to which diagnosis is imprecise, despite the scoring systems and algorithms and use of identical testing materials. During our training it becomes apparent that there is room for considerable disagreement and several interpretations of the same phenomena. This worries me, given that the implications are huge – where a divergence of one or two points might make the difference between diagnosis and non-diagnosis. Talking to my fellow trainees, it becomes clear that they share this concern. How reliable are they going to be during the months/years in which they are still relative novices, and does this concern apply also to the testers in the intervention trials?

So much remains elusive, but perhaps this is inevitable. If there is still no shared understanding of a definitive autistic essence, how can there be a definitive diagnostic tool? It will no doubt continue to change with successive incarnations of the international classification systems, and with greater identification of subgroups.

To meet their critics, the committees who decide on the criteria and on the resultant diagnostic tools will need to seek out the contribution of autistic people and those who live with them. They will need to give greater recognition of the positive features of autism and the positive qualities that autistic people have, and the whole process should come out from behind the wall of professional and commercial exclusivity.

I believe we need to keep discussing these issues, across ‘lay’ and ‘professional’ stakeholder groups, and I hope this series of articles might contribute helpfully to the discourse.

4. Diagnosing the diagnosis of autism: systematic pathologisation or a key to understanding?

Steven K. Kapp is a developmental psychologist & autistic activist.

A clinical training of an interaction-based assessment tool illuminated autism’s construction as a disorder mostly of social communication diagnosed mainly according to the observations of non-autistic people. My experience with the training highlighted that while standardised behavioural assessments may pathologise autism, overlook the individual’s perspective, and occur in an artificial context, instruction on how to interpret observable actions may provide at least some consistency that might further access to services. It also might serve as a basis for research on which supports help which individuals, as we know little about who benefits from an autism diagnosis. While the facilitator shared predetermined scores for behaviours under the guise of a ‘discussion’ to achieve ‘consensus’, a training for research reliability would have had even less room for interpretation, which at least provides a stable foundation upon which to investigate the validity.

One could readily focus a critique on the commercialisation of the autism field into an industry. Indeed, the training began by passing around leaflets or catalogues to other courses and assessment materials by the psychometric product corporation that provided our meeting space and objects from pens to graph paper marked with their logo. It ended with an e-mail from the company to receive a 5 percent discount on any products for the assessment or another related training course upon completion of a post-training coding assignment. This completion would also provide a certification for participants to use (any module of) the tool in clinical practice despite only attending two days of a group training that struggled to make time for the four age- and language-based modules across the autism spectrum and even less for individual attention (but plenty of opportunity to observe sanctioned behaviours like pressing knuckles against the side of the face while looking away, which a trainee might mark down as socially inappropriate listening if observed in an autistic client).

Moreover, the scale suffers from an administration in an artificial context that may not have ecological validity. Unfamiliar adults administer it, which may disadvantage autistic people uncomfortable with strangers but who may display relatively typical markers of sociality among loved ones such as parents (Feldman et al., 2014; Parma et al., 2013), and yet the number of caregivers who attend the testing sessions does not affect the coding.

Alternatively, this format may advantage autistic children and adolescents who may relate relatively well to adults – or even perceive higher quality of life in their relationships with authority figures like parents and teachers than typically developing youth (Cottenceau et al., 2012). The experience of testing under explicit instructions may disadvantage autistic people who have difficulty with executing actions on command due to movement (dyspraxia) and executive functioning challenges (Mostofsky and Ewen, 2011). While atypically high automatic imitation (in behaviour – echopraxia – or words – echolalia) associates with higher degrees of autism (Bird et al., 2007; Sowden et al., 2016; Spengler et al., 2010), the training may encourage codes that dismiss the mimicry as not ‘meaningful’ or ‘spontaneous’. Participation may not reflect full consent, and even attempts to build rapport by enthusiastic and high-pitched clinicians (which the facilitator praised) may backfire, as ‘babytalk’ (child-directed speech) often does not benefit the communication of autistic children as it may for their typically developing counterparts (Solomon, 2011) – possibly because it may exacerbate to focus on the phonology (sound) of words at the expense of semantics (meaning) (Norbury et al., 2010). Thus the tool suffers from the limitations of interaction and perception by clinicians who may struggle to interpret autistic people’s behaviour, yet the training is to interpret particular behaviours in particular ways (especially taught and enforced for research reliability), which at least promotes consistency.

Eyeing Autism: Optical Illusion or a Matter of Perspective?

The facilitator explained that coders too often disagree on subtle or slight observed atypicalities in eye contact, so they now receive training to only mark them when quite clear, reflecting difficulties in measurement despite its emphasis in various autism assessments. Coders also must penalise an individual even when they seem merely ‘shy’ and disregard any eye contact made with a caregiver or loved one in the room. Culturally competent coders may recognise that children in non-Western cultures, as well as atypically developing children in the U.K. (such as autistic and blind children), may not rely on gaze for reciprocal social interaction (Akhtar and Gernsbacher, 2008). Even crèche (day care) staff in South Africa who self-perceived little knowledge of developmental milestones noted that they do not expect direct eye contact and that past early childhood it becomes judged as disrespectful with peers and adults (Grinker et al., 2012). Perhaps the emphasis on eye contact stems from Westerners’ (mis)perception of co-interactants or onlookers, as they may feel rejected or devalued by someone who averts their eye gaze (Wirth et al., 2010), contributing to the alienation of some autistic people.

Furthermore, raters may observe that autistic people’s reduced or atypical eye contact may not indicate social disinterest in or disengagement. It may reflect a coping mechanism for feeling overwhelmed from eye-to-eye contact due to difficulties with visual overload (Gernsbacher et al., 2008), listening and watching simultaneously (Doherty-Sneddon et al., 2012; Falck-Ytter, 2015), or processing complex movement (Elsabbagh et al., 2012; Hanley et al., 2013; Weisberg et al., 2012). Similarly, autistic individuals may avert eye contact when perceiving others’ expressions as threatening (without necessarily pervasive social anxiety: (Gernsbacher and Frymiare, 2005; Tottenham et al., 2014). Autistic people’s eye contact may lack typical social flexibility for reasons including domain-general (i.e., not limited to social contexts) poor oculomotor (eye movement) control (Kirchgessner et al., 2015) or atypically sustained (or ‘sticky’) visual attention (Keehn et al., 2013; Sacrey et al., 2014), and ‘freezing’ gaze toward facial expressions perceived as threatening in a manner atypical even for non-autistic socially anxious people (White et al., 2015).

In contrast, the restricted and repetitive behaviours and interests domain of autism measures may provide more meaningful information than interactive behaviours that inherently vary by social context. Sensory and motor behaviours may underlie the autism ‘symptoms’ and items of repetitive motor movements and sensory interests, while hyperfocused attention may contribute to perseverative interests or behaviours. Despite the heterogeneity of autism, sensory, motor, and general attentional differences (as measured by ADOS and other instruments) consistently emerge as its earliest manifestations in empirical research such as longitudinal studies of infant siblings of autistic children (Gallagher and Varga, 2015; Gliga et al., 2014; Rogers, 2009; Sacrey et al., 2015). Similarly, studies using various autism instruments report that repetitive behaviours show relative stability while atypical social behaviours reduced more quickly across different developmental trajectories in autistic toddlers through adults with various intellectual and language abilities (Fecteau et al., 2003; Fountain et al., 2012; Pellicano, 2012; Piven et al., 1996; Seltzer et al., 2003; Soke et al., 2011). Additional evidence for these general (not socially specific) mechanisms as driving autism stems from the only (to my knowledge) longitudinal study from toddlerhood to adulthood to claim some individuals outgrow autism according to standardised instruments, which reported that a reduction in repetitive behaviours between ages two and three predicted no longer meeting criteria on the ADOS by age 19 (Anderson et al., 2014).

The apparent importance of these general features of autism despite their small proportion of autism’s diagnostic criteria and affiliated instruments suggest meaningful patterns. Nevertheless, I tended to ‘under-score’ sensory and motor symptoms and ‘over-score’ communication symptoms relative to the group. This dichotomy may both highlight the difficulty of evaluating domain-general behaviours in social contexts and perhaps my indoctrination into the socially and medically biased autism field.

Reliability: Looking through the Kaleidoscope of Autism through a Forced Lens

Although one could endlessly debate the validity of the particular diagnostic tools or indeed the autism construct itself, the assessment tool arguably enables the achievement of reliability, perhaps because of its basis on elicited behaviours in particular contexts. Despite differences in criteria used to determine and allocation of Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) diagnoses within the autism spectrum by North American university-based clinical sites, the centres shared similar distributions of scores on standardised assessment measures, especially the (social communication scores of) ADOS (Lord et al., 2012). The only study to my knowledge to test the interrater reliability of the ADOS more naturalistically via routine clinical practice across multidisciplinary sites found agreement on scores comparable to research-reliable samples (Zander et al., 2016). Another independent study reported that only the ADOS (Module 4) could reliably identify autism among adults in the community in England (Brugha et al., 2012). In contrast, separate research teams have reported inconsistent results from alternative questionnaire measures (Brugha et al., 2012; Sizoo et al., 2015).

Making a contrast with psychopathy further suggests the strengths in reliability of a behaviourally based standardised diagnostic instrument for ASD relative to more subjective measures. Psychopathy differs from autism as a personality disorder with a callous manipulativeness that enables some people to exploit others through predatory social skills, and diagnostic assessment is made using the Psychopathy Checklist – Revised (PCL-R; (Hare, 2003). Although another interview-based tool that requires certified training and reputedly the diagnostic ‘gold standard’ of its field, the PCL-R fails to demonstrate inter-rater reliability outside controlled research settings, especially for its core psychosocial personality traits as opposed to items of impulsive behaviour or criminal history (Edens et al., 2010). PCL-R diagnosed offenders can often generate an impression of empathy rather than meanness within the first few seconds of an interaction (Fowler et al., 2009; Robinson and Rogers, 2015), and those with high ability to infer others’ thoughts or feelings may more often malinger (report false medical symptoms or illness) to avoid harsh punishment (Nentjes et al., 2015). Agreeable (trusting, caring, etc.) people may more often feel convinced by these manipulative acts or give the benefit of the doubt, therefore raters’ own personality traits influence their scoring on the PCL-R (Miller et al., 2011). Thus while both autism and psychopathy inherently affect social functioning, autism’s behavioural definition and the deception practiced by psychopathic people may explain the higher (moderate) correlations achieved between autistic youth, their parents, and teachers in the social domain (Stratis and Lecavalier, 2015), compared with the only modest such inter-rater agreement for adolescents with ‘fledgling’ psychopathy (Docherty et al., 2016). The autism field’s relative agreement on definition and greater availability of supports also lend more utility to the diagnosis.

Conclusion

Diagnostic tools may oversimplify and pathologise autism, but as the facilitator said, no type of assessment can be used alone and they hardly serve as a measure of treatment outcomes. The DSM-5 Neurodevelopmental Disorders Workgroup that revised autism’s diagnosis said criteria work best for 5-to-8-year-old white boys (Swedo et al., 2012), and similarly (as acknowledged by the facilitator in planned presentation or answers to questions) the instrument struggles in application to some groups (women and girls, adults especially without fluent speech or with severe intellectual disability; typists who do not speak; non-native speakers of English; people with selective mutism, blindness or deafness; or physical impairments). It may not apply to all autistic people, but perhaps it can help to identify patterns for particular subsets. Assessments may benefit from incorporating strengths (as scientifically supported) in the items, as these may help to suggest support tailored to individual profiles of strengths and weaknesses. Such a change would require a revolution in the classification of autism, one long overdue to view individuals holistically and not as a collection of pathology.

Discussion- A summary of themes

Ginny Russell is a social scientist and epidemiologist

Here, I attempt to summarise the common themes raised by the authors. In order to do so, the commentaries were coded and points where all had raised similar topics as well as disagreements were noted. I identified three recurring themes in the four commentaries about the diagnostic tool: one theme centred on epistemic authority and power, whilst another raised questions over the tool’s objectivity. All four authors discussed the commercial nature of the course. A final theme covers differences between the commentaries. Overall the authors described how the tool played an active role in shaping the context of the assessment and our understanding of how we read behaviour and ultimately what we know autism to be. For example, the fourth author notes the “interaction-based assessment tool illuminated autism’s construction as a disorder mostly of social communication”. The second author writes how the tool has a role “recording autistic behaviour as ‘mechanical’ and thus reifying the much critiqued machine-like metaphor often used to describe autistic people”. All authors also passed judgement on the utility of the tool and diagnosis more generally, but disagreements arose here.

1. Power relations

All four authors commented on the power relations inherent in the training process and the autistic authors noted especially the power gap between autistic and non-autistic people inherent in the tool’s development and implementation. One author described how the tool was presented as ‘factual based’, masking the power relations between the passive subjects of the test, and clinicians or researchers, who do the testing, and have authority in in the diagnostic decision making process, acting as ‘gatekeepers’ of diagnosis. Those in power were able to judge who qualified as normal or abnormal, such individuals are bestowed the epistemic authority to diagnose and/or deny diagnosis. Commentaries described how language on the training course was used to position autistic behaviours as always problematic, describing ‘symptoms’ of autism. Whereas the commentators (who were themselves part of the autism community) sometimes viewed behaviour in a different, more positive light:

I saw the child regularly smiling, but the facilitator said this was not smiling but ‘smirking’

We laugh with recognition and delight in the fun the boy is having. He is not reliant on approval from the diagnostician, and we appreciate his exuberance and his ability to see things differently.

The process involved learning to record children’s behaviour in standardised, replicable, objective, quantifiable, mechanistic and measurable ways, as ‘a collection of pathology’. Thus the course trained students to interpret children’s behaviour through a lens of autism, reminiscent of the Foucauldian medical gaze (1973). In this way, diagnostic tools themselves might promote the construction of autism through a deficit-based diagnostic narrative (through training participants to recognise certain ways of interpreting autistic behaviour):

Autistic ways of being are described as “compensatory strategies”. Interests are described as “obsessions”, “fixated” and abnormal for not being able to “shift flexibly around the interests and behaviour of the administrator”, whilst the reverse is seen as unproblematic.

Our authors commented on the influence of context in assessment and the power of the training to promote reading of children’s behaviour as autistic symptoms. Such reading in the light of a diagnosis is suggestive of writings of labelling theorists of the 1970s, for example Rosenhan’s ‘On being sane in insane places’ (1973): once incarcerated for schizophrenia, Rosenhan and his (sane) colleagues had their behaviours routinely interpreted as manifestations of schizophrenia by health professionals.

The emotional responses of the commentators who found the film clips of children ‘distressing’ and affecting were not brought to the fore during the course. It is not clear if the emotional effect of the training on practitioners and subjects is discussed in detail as a routine part of training to use the diagnostic tool. There was an overall impression of unease in all the commentaries concerning informed consent and around whether children were benefitting from and enjoying the assessment process, contributing to a further sense of imbalance of power.

2. Objectivity, subjectivity and context

Although the tool was presented as a way to standardise, quantify and measure behaviours reliably and objectively, questions over a lack of objectivity emerged as a recurring theme. Although one author commented on the extensive research evidence that shows reliability of this tool, all authors discussed instances where contextual factors might undermine the replicability of children’s behaviour. These included examiners’ behaviour and how this might impact on the child, the unfamiliar setting and person, age, gender and wider cultural context that might establish which behaviours are deemed especially problematic. An example was direct eye gaze which in South Africa is ‘judged as disrespectful with peers and adults’ (whereas on the course lack of gaze was taught to be interpreted as deficient). This was seen as an instance of culturally determined values being represented as children’s deficits.

The language of the course also served to present the rating process as ‘standardised’, ‘valid’, ‘reliable’ and ‘objective’. Thus the visceral and emotional nature of a child’s behaviour and interaction with the rater was (perhaps necessarily) reduced to a set of data. A final score was calculated: ultimately this was reducible again to a yes/no binary – does the child have autism or not?

The course trained its participants to interpret social and communicative behaviours as a facet of the individual person: an inherent quality or trait of the individual. All the authors made the point that social skills only exist in relation to others, e.g. “Robinson Crusoe couldn’t have been autistic until he found Man Friday”. The commentaries called for transparency over the constructed relationship between the examiner and the child, and the setting and nature of the test, which might provoke certain types of behaviour.

These looked more like tests of social conformity to a bossy adult’s constantly shifting demands.

Thus, in a similar way to Turowetz and Maynard (2015, 2017) the commentaries drew out how the nature of the test itself (“a contrived social situation”) could influence the diagnostic outcome. Two authors in particular took umbrage with the tool’s assessing and identifying autism primarily through deficits in social skills and communication. One placed more emphasis instead on sensory differences: suggesting the tool should be reformed with more emphasis on sensory sensitivity, rather than social behaviour. Such arguments drew attention to how the tool itself plays an active role: in this case constructing what we understand autism to be: primarily a condition of social problems. This was a point of critique ‘it attempts to identify relational ‘deficit’ whilst demonstrating the instrument’s own lack of awareness about the nature of relationships’.

3. Commercialisation and lack of transparency

A final aspect of the tool that all authors drew attention to was the status of the tool as “a commercial product”. The training and tool employed a business model involving upselling of various other related products. This raised the question of in whose interests was it to promote the tool, and promote diagnosis and identification of autism more generally. The commercialisation of autism into the ‘autism industry’, an enterprise designed to engender commercial gain, was not explicitly described as objectionable. The authors gave the impression that what was objectionable was actually the lack of transparency about this commercial aim of the tool, and its pretension of objectivity without much succour to its own limitations. One author noted the “whole process should come out from behind the wall of professional and commercial exclusivity”. However, this raises the question of whether the commercial nature of the tool could conflict with the need for clarity about limitations of assessment. In other words, if the training was more open and involved more reflection regarding of the role of context, constructed relationships and the training itself as a mechanism to read behaviour through an autism-as-deficit diagnostic lens, would this undermine raters’ faith in this particular tool? And could that lead to less use (and sales)? Would more transparency in training about limitations of the tool destabilize the commercial enterprise?

I would argue this is not necessarily the case as most clinicians and researchers are able to understand that tools they use are imperfect and take a pragmatic view. The ethnographers and medical sociologists have shown how doctors are taught to rely on their own knowledge as a source of certainty (Atkinson 1995). Latimer (2013) shows how clinicians will ignore a diagnostic test if it conflicts with their own medical judgement. Rather, Latimer shows, test results for genetic conditions are woven in with other sources of evidence such as family history, patient testimony and impact on functioning to create a diagnostic story or narrative about the patient’s predicament. Social factors (like local access to specialist schooling) may further influence diagnostic decision-making in autism clinics.

Hedgecoe (2004) has written about how medical authority rests on the claim that clinicians are ‘certain’ about a diagnosis. Diagnostic tools such as ADOS are presented as factual, reliable and objective. But what of the commercialisation of the training and use of the diagnostic tool? Hedgecoe talks about how acknowledging uncertainty in assessment might undermine the medical and commercial endeavours of establishing authority. What, he asks, does it do to professional status of the rater if their tools are shot through with uncertainty?

4. Where authors did not agree

There was divergence about the overall utility of the tool, and validity of autism as a concept. One author who was autistic himself, so had invested in autism as a concept, argued for reform and modification of the tool. Another author took a more revolutionary stance dismissing the entire tool as a “diagnostic trap”, with a wholesale rejection of the concept of autism. Along similar lines, the authors had differing assessments of the utility of the tool. Whilst all provided a critical commentary, two authors ultimately made arguments for the utility and necessity of assessment: ‘obtaining a diagnosis might help the boy’. This was because of the crucial gatekeeping function of diagnosis in releasing resources, allowing others to adjust their expectations, aligning researchers to a common understanding, and the way an autism diagnosis could allow access to a supportive community of autistic others. Without the diagnosis, the parent activist wondered “will he be viewed merely as wilfully uncooperative and hence condemned to exclusion without support?” According to this account, although the tool was flawed, identification and diagnosis was necessary: somebody has to judge, somebody has to have authority: as “self-identification will never be a viable way forward”, and “I still feel there is an important utility in the umbrella category of autism spectrum disorder”. In contrast the first author questioned the basic assumption that “ASD exists as a discrete, natural ‘thing’, and ‘that this autism ‘thing’ can be identified and quantified”.

Although all were critically appraising the tool (as requested), none of the authors gave much thought to an alternative system that might replace the functions of diagnosis. But this was not in their brief. The current diagnostic system has been heavily critiqued from both psychiatry and outside but diagnosis serves many valuable functions in our society (Rose, 2013). Nevertheless, attempts have been made to forge alternative systems of classification. For example, the Power Threat Meaning Framework (Johnstone et al, 2018) is a UK-based attempt led by clinical psychologists and service users to drop the diagnosis of mental health conditions, although this has come under fire from social scientists and psychiatry. The International Classification of Functioning (ICF) presents an alternative taxonomy to the Diagnostic and Statistical Manual of Mental Disorders (DSM) and International Classification of Diseases (ICD), using ability as well as disability to classify individuals into types.

Medical sociologists (Jutel and Nettleton 2011) have discussed how medical nosology, through application as diagnosis, promotes and maintains a certain medical reality. Authors differed as to whether using the diagnostic tool was ultimately a worthwhile and necessary endeavour. If we accept we need a category of autism (and not all the authors did) then surely it becomes necessary to standardise and measure autism in ways researchers and clinicians (tool users) recognise and agree on. A repeated critique from the autistic authors, though was the lack of autistic input in the tool’s development. The question of who decides how autism is defined is clearly political as well as evidence-based. Diagnostic tools may be the best way of identifying autism at the present time, but authors drew attention to their active role in constructing what autism is understood to be. In this sense the understanding of autism we currently work with is whatever it is the gold standard tool measures.

Despite some fairly fundamental disagreements, all critiqued the tool’s lack of transparency, recognizing context, emotion, and differences in interpretation and power imbalances as playing an unidentified role in the assessment process. Based on these analyses, we recommend that training for raters for such tools should be accessible to a wider group of people, and incorporate more explicit recognition of its own limitations.