Commentary: The construct validity of ‘camouflaging’ in autism: Psychometric considerations and recommendations for future research

Abstract

Research on the construct of “camouflaging” in autism and its sociodemographic/clinical correlates has far outpaced the work being done to establish the construct validity of camouflaging and its distinction from other similar constructs. The imprecision with which camouflaging is defined and measured has serious implications for future research on this topic, and unless additional effort is made to produce reliable and valid measurements of this construct, researchers will not be able to meaningfully assess important questions such as whether the effort of camouflaging one’s behavior contributes to increased mental health difficulties. By reviewing the psychometric strengths and weaknesses of various operationalizations of camouflaging, this commentary highlights a pressing need for further measure validation in this area. Specific methodological guidance is provided for researchers interested in rigorously testing the validity of putative camouflaging measures.

The concept of “camouflaging” has become a topic of substantial interest in the field of autism research, although controversy abounds regarding the definition, operationalization, and clinical relevance of this construct (Cook et al., 2021; Fombonne, 2020; Lai et al., 2020). Generally, camouflaging refers to the process whereby an autistic person attempts to appear “less autistic” in social interactions through the use of conscious or unconscious behavioral strategies. Anecdotally, camouflaging behaviors have been linked to adverse consequences such as physical/emotional exhaustion, role confusion, and worsening mental health in autistic individuals (Cook et al., 2021), and a small body of research has begun to investigate these relationships empirically. However, relatively little attention has been paid to the methods used to quantify camouflaging in these studies, and the psychometric properties of various camouflaging measures remain largely unexplored. Moreover, despite calls for additional research on the construct validity of camouflaging measures (Fombonne, 2020; Lai et al., 2020), recent commentaries stop short of offering concrete recommendations for future validation studies. To further this discussion, the current commentary evaluates the psychometric strengths and weaknesses of the ways camouflaging has been operationalized to date, providing specific guidance for researchers seeking to establish the construct validity of these measures.

Self-report Measures of Camouflaging

To date, the most frequently-used measures of camouflaging behavior have been self-report questionnaires, particularly the Camouflaging Autistic Traits Questionnaire (CAT-Q; Hull et al., 2019). The CAT-Q contains 25 Likert-type items organized into three subscales of “compensation,” i.e., making up for social-communicative difficulties in a non-standard way (sample item: I have developed a script to follow in social situations (for example, a list of questions or topics of conversation)), “masking,” i.e., hiding aspects of one’s autistic presentation from others (sample item: I adjust my body language or facial expressions so that I appear interested by the person I am interacting with), and “assimilation,” i.e., working to fit in with others in social situations, (sample item: I have to force myself to interact with people when I am in social situations). Self-report measures of camouflaging have many advantages, including sampling experiences from a wide range of situations (as opposed to a single laboratory-based assessment), adequate face-validity (i.e., variance in self-reported camouflaging behavior likely reflects variance in “true” camouflaging behavior), good reproducibility, low cost, and potential for remote administration. In addition, there exists a substantial body of literature on the process of self-report questionnaire development and validation, including widely-accepted guidelines for best-practices at all stages of the measure development process (PROMIS Health Organization & PROMIS Cooperative Group, 2013). Although self-report measures are subject to biases such as acquiescence, response style artifacts, and social desirability effects, this modality remains a promising option for the measurement of camouflaging and its various facets.

While existing psychometric evidence is lacking for most measures of self-reported camouflaging, the CAT-Q has been subjected to preliminary psychometric testing (Hull et al., 2019). In its initial validation study, the CAT-Q demonstrated acceptable internal consistency reliability, a three correlated-factor structure with adequate fit, mixed evidence for measurement invariance across four subgroups (autistic men, autistic women, non-autistic men, and non-autistic women), and positive correlations with self-report measures of autistic traits and psychopathology. CAT-Q scores also showed a fair degree of temporal stability over a three-month retest period, providing preliminary evidence that the self-reported tendency for one to camouflage may be a stable behavioral trait. However, validation is an ongoing process, and additional psychometric information is needed to ensure that individual or group differences in CAT-Q scores truly reflect differences in camouflaging. One particularly salient psychometric question is whether CAT-Q scores should be analyzed at the level of total or subscale scores. Although CAT-Q total scores have been reported in a number of studies, camouflaging is a theoretically multidimensional construct that is bound together only by the intended goal of its constituent behaviors (i.e., to appear less overtly autistic). Moreover, correlations between various dimensions of camouflaging in empirical studies have been modest (Cook et al., 2021; Hull et al., 2019), potentially indicating that the CAT-Q total score does not represent a coherent “general camouflaging” factor. Nevertheless, future studies utilizing appropriate psychometric methods (e.g., Stochl et al., 2020) are needed to clarify whether the CAT-Q and other self-reported camouflaging measures truly do measure a single superordinate construct.

Another important area of psychometric inquiry is whether some CAT-Q items are confounded by social anxiety or other psychopathology (e.g., I always think about the impression I make on other people; Fombonne, 2020). As much research on camouflaging has been interested in the relationships between these behaviors and mental health outcomes, it is crucial that any putative camouflaging questionnaire be distinguishable from social anxiety or anxiety-related safety behaviors (Piccirillo et al., 2016). Otherwise, cross-sectional relationships between these measures and mental health outcomes will remain uninterpretable due to the potential for reverse causality. To better address this issue, the CAT-Q and other putative camouflaging scales should be subjected to quantitative semantic analysis (Rosenbusch et al., 2020) to determine if they are indeed substantially similar to existing measures of social anxiety. In addition, future studies of discriminant validity should be conducted using multi-measure latent variable models, which can test hypotheses about whether two sets of items seem to measure one single overarching construct (e.g., testing the distinctiveness of burnout and depression; Verkuilen et al., 2020).

A final measurement property of the CAT-Q that requires more extensive study is differential item functioning (DIF, also known as measurement invariance), the degree to which the scale’s measurement model varies between subgroups of the population. DIF is important to consider when comparing test scores between groups that are hypothesized to differ in their true level of camouflage (e.g., autistic men and autistic women), as group differences in effect sizes may be substantially biased by the presence of DIF. The degree of DIF between autistic and non-autistic adults can also inform the ongoing discussion of whether camouflaging is a meaningful construct when applied to non-autistic populations (Fombonne, 2020; Lai et al., 2020), as this analysis can determine whether a given questionnaire assesses qualitatively different traits in the two groups. While the initial CAT-Q validation study did include a limited amount of DIF testing, this analysis may have had inadequate power to detect meaningful group differences in measurement models according to either gender or diagnostic status, and DIF effect sizes were not reported to quantify the degree of invariance observed in the study. Additional research on this topic should employ larger samples and more in-depth statistical analyses to examine the DIF of the CAT-Q and other camouflaging measures, determining the degree to which self-reported camouflaging items function differentially according to age, sex/gender, diagnosis, education level, cognitive ability, and other key sociodemographic and clinical variables.

Discrepancy Measures of Camouflaging

Another popular way to measure camouflaging is the so-called internal-external discrepancy approach (Cook et al., 2021; Lai et al., 2020), in which the observed behavior of an individual (e.g., autism diagnostic observation schedule [ADOS] scores) is compared to that individual’s social-cognitive performance (e.g., scores on a theory-of-mind [ToM] task or in some cases, a self-report measure of autistic traits). Within this framework, camouflaging is operationalized as the degree to which external (observed) social behavior is “less autistic” than would be predicted by scores on the internal measure (e.g., by subtracting normalized ADOS scores from normalized ToM scores). Although discrepancy measures are less portable and more resource-intensive than self-report questionnaires, these measures arguably assess different facets of the camouflaging construct (Cook et al., 2021). Whereas self-report questionnaires measure the (perceived) frequency of camouflaging behavior, discrepancy measures ostensibly capture the outcome of these behaviors (i.e., the degree to which one successfully compensates for social-cognitive difficulties). Thus, although researchers may be tempted to conflate these two operationalizations of “camouflaging,” self-report and discrepancy measures should not be viewed as equivalent or interchangeable. Much like self-report and performance measures of executive functioning, these two assessment modalities provide nonredundant and complementary information, both contributing to a richer understanding of a given individual’s clinical presentation.

Despite the growing popularity of discrepancy measures in the camouflaging literature, there is substantial controversy regarding their validity and interpretation as measures of camouflaging (Fombonne, 2020; Lai et al., 2020). One major issue with this method, first raised by Fombonne (2020), is that the component measures used in existing studies may not be ideal for assessing camouflaging. For instance, scores on the module 4 ADOS include ratings of “insight into typical social situations and relationships” and “communication of own affect,” items that tap internal states and social cognition rather than observable behavior. Furthermore, given the predominant use of the ADOS as a diagnostic tool, there are pronounced floor effects in the non-autistic population, substantially limiting the degree to which “camouflaging” can be meaningfully assessed in other groups of interest, such as individuals with social anxiety (Lai et al., 2020). Rather than using the ADOS, future studies on camouflaging should consider behavioral measures such as the Conversation Probe role-play test (Morrison et al., 2020), in which trained raters code the social behavior of the participant during a 5-minute unstructured “get-to-know-you” interaction with a confederate. The most appropriate measures of “internal” autism status have also been debated, although proponents of the discrepancy approach tend to favor the use of laboratory-based social cognition assays (Lai et al., 2020). However, recent studies have found very modest relationships between these tasks and real-world social interaction outcomes in both autistic and non-autistic participants (Morrison et al., 2020), casting some doubt upon the key assumption that social-cognitive ability is a major predictor of observable social behavior (Lai et al., 2020). If this assumption does not hold in general population individuals, who presumably camouflage substantially less than their autistic peers, the residual variance in social cognition-predicted behavioral outcomes is much more likely to reflect construct-irrelevant variance than the effects of camouflaging itself.

One way to overcome many of the limitations in this internal-external discrepancy approach is to abandon the measurement of “internal” autistic status entirely and focus solely on observable behaviors under different experimental conditions (i.e., the “experimental discrepancy” approach). For example, consider an experiment in which participants engage in an unstructured “get-to-know-you” conversation under two conditions: a “naturalistic” condition that mirrors the typical Conversation Probe role-play task (Morrison et al., 2020) and an “authentic” condition where the participant is instructed to “not worry about social conventions and show your partner your true or real self.” Assuming that the manipulation is successful (e.g., when viewing videos of their interactions, participants identify a larger number of their own camouflaging behaviors in the “naturalistic” condition compared to the “authentic” condition; Cook et al., 2020), differences in observer ratings of social behavior between the two conditions would likely reflect the degree to which an individual (successfully) camouflaged their authentic self during the “naturalistic” condition. Additional manipulations to increase camouflaging (e.g., instructing autistic participants to conceal their autism diagnoses from their conversation partners) could also be used, and outcomes such as self-reports of how much a participant attempted to camouflage during a given conversation may also be used to provide complementary information to observer ratings. Although experimental manipulations have not yet been utilized in the camouflaging literature, such methods are common in social psychology, and established statistical methods such as latent change scores and response surface analysis can be used to analyze two-condition experimental data while avoiding the psychometric limitations of observed difference scores (Gollwitzer et al., 2014; Humberg et al., 2019). Thus, the experimental discrepancy approach represents a promising measure for use in future camouflaging studies, although foundational research is still needed to determine which experimental manipulations and social outcomes are able to produce the most robust and replicable indicators of camouflaging.

Conclusion and Summary of Recommendations

In sum, both self-report and discrepancy measures show promise as putative measures of different facets of camouflaging, but studies to date have minimally addressed the construct validity of these measures. As hypotheses about camouflaging and its consequences rely on valid measures of the construct(s) of interest, additional psychometric work to address issues of construct validity in camouflaging are paramount. To further examine the validity of self-report measures, questionnaires such as the CAT-Q should be assessed in terms of (a) the suitability of sum scores, (b) distinctness from social anxiety and other psychopathology, and (c) differential item functioning across subgroups of the population. Additional study of discrepancy measures is also warranted, although experimental manipulations with associated validity checks are recommended to provide built-in tests of construct validity and ensure that score variance primarily reflects the construct of interest. Lastly, in the absence of research demonstrating the convergent validity of self-report and discrepancy methods, these two operationalizations of “camouflaging” should be considered separate but related constructs and referred to as such (i.e., the CAT-Q measures self-perceived engagement in camouflaging [“camouflaging intent”], whereas a discrepancy in behavior ratings between experimental conditions measures observed camouflaging behavior [“camouflaging efficacy”]; Cook et al., 2021). Although a sizable amount of research is needed before hypotheses about camouflaging can be rigorously tested, such work will help to ensure the rigor and replicability of studies on this topic for years to come.