Developing the Multidimensional Visual Scale Assessing Affect, Anxiety, Pride, and Energy Through a Research Partnership with Autistic Scholars

Abstract

Autism research studies have traditionally failed to represent the full diversity of the autism spectrum due to the lack of measures available for use with participants who prefer to express themselves visually. A multidimensional measure of emotions, which can include both picture- and text-based prompts, may improve accessibility of emotion rating measures and broaden participation in research and educational evaluations to include those who communicate in diverse ways. Picture-based measures designed to assess participants' emotions may be useful for research concerning autistic identity and service evaluation, two areas where representation of diverse perspectives is needed. Our participatory group of autistic and nonautistic researchers developed a Multidimensional Visual Scale Assessing Affect, Anxiety, Pride, and Energy (AAPE) by adapting and expanding upon an existing emotion rating scale. When testing the AAPE with autistic college students (n = 72), their open-ended responses indicated that the AAPE's dimensions of affect (97.2% correct), anxiety (79.2% correct), and energy (84.7% correct) were well comprehended without text-based labels with potential for improvement in how pride (52.8% correct) was represented. When provided with the labels that each dimension was intended to represent, participants generally agreed that each emotional dimension was well represented. When tested in an informal educational summer camp with autistic children and adolescents (n = 50), the AAPE was well received and revealed insights about the students' emotional responses to different instructional strategies that can guide curricular improvements. The AAPE has utility as a tool to help diverse autistic individuals self-advocate and improve research and services.

Lay summary

Why was this study done?

There are very few tools that autistic people can use to share how they feel. We wanted to develop a tool to help autistic people express their emotions using pictures. Pictures can help autistic people share how they feel.

What was the purpose of this study?

We wanted to make an easy-to-understand tool that autistic people can use to share how they feel.

What did the researchers do?

Our research group is participatory, meaning that autistic and nonautistic researchers worked together to make our tool. An autistic artist drew the tool. We called it the AAPE, which stands for the emotions it assesses: Affect, Anxiety, Pride, and Energy. We worked together to see how well the AAPE worked. We used a survey to see if autistic high school and college students understood our first try at the AAPE and we learned how to make the AAPE better from these students. We worked together to make the AAPE better. Then, we did another survey with autistic college students to see if our second try at the AAPE worked better. Then, we asked autistic kids and teenagers to use the AAPE to share how they felt about different ways of teaching.

What were the results of the study?

In our final test, we asked 72 autistic college students to tell us what emotions they thought the AAPE was showing. College students thought that affect (97.2%), anxiety (79.2%), and energy (84.7%) showed the emotions we aimed to show with room for improvement in how pride (52.8%) was shown. After we told participants which emotion each scale was showing, they agreed that affect (average score 4.28 of 5) and anxiety (4.29 of 5) showed the emotions best, followed by energy (4.08 of 5) and pride (3.5 of 5) on a scale from 1 to 5 (strongly agree).

Students preferred using the AAPE compared with text-based surveys we used in the past. Results showed that the AAPE does a good job at measuring emotions. Edits may still be needed to better show “pride.” Work is needed with nonspeaking people to make sure our measure works well for people who communicate in different ways.

What are potential weaknesses in the study?

The autistic adolescents and adults we have tested the AAPE with so far have not been very diverse. We did not include participants who communicate without speaking in these first tests of the AAPE. We plan to use the AAPE with more diverse groups in future studies.

How will these findings help autistic adults now or in the future?

In a future study, we will use the AAPE to rate pictures that show experiences of being autistic, like feeling outside a social group, having sensory difficulties, and making patterns. We would like to do this because some studies talk about negative experiences of autism, like feeling the need to hide autistic traits, but other studies describe strengths of autism, like strong memories and advanced knowledge in particular subjects. However, these studies do not talk about the emotions that come with these experiences and if these experiences are shared with autistic people who do not use speech to communicate. The AAPE is a tool that might help us understand how diverse autistic people feel about autism.

Introduction

Studies assessing attitudes toward autism typically require participants to communicate through spoken or written language, greatly limiting representation of the full diversity of the autism constellation.^1,2–5 Literature investigating topics such as autistic experiences, services, and quality of life often rely on caregivers to speak for those individuals deemed “lacking capacity to self-report.”^6,7 Despite calls for increasing representation of people who have traditionally been underrepresented within autism research (e.g., people with co-occurring intellectual disabilities and people who do not speak),^8,9 most existing research assessing the perspectives of autistic individuals has relied on purely language-based assessments.^10,11

Assessments that are accessible to individuals across the autism constellation are needed so that a more representative group of autistics may share their experiences using communication methods designed around their abilities and challenges. Given that autism is often associated with visual strengths,¹² picture-based options may increase engagement with and/or comprehension of survey items. Indeed, assessments that include picture-based options have been found to be more engaging and accessible than purely text-based measures for young children and individuals with intellectual disabilities.^13–16 Therefore, picture-based measures may also allow diverse autistic individuals, including people with co-occurring intellectual disabilities and people who do not speak or read, to participate in research and educational evaluations. Picture-based strategies such as photovoice and Talking Mats are increasingly (although still rarely) used to assess the first-person perspectives of autistic people with diverse communicative capacities.^10,11

The current study presents the participatory process of developing an emotion rating scale, the Multidimensional Visual Scale Assessing Affect, Anxiety, Pride, and Energy (AAPE), which is designed to be accessible to people who communicate and comprehend in diverse ways. This is the first stage of a larger ongoing project conducted by our participatory research group, which focuses on measuring aspects of autistic identity using a picture-based scale, with accompanying text-based item descriptions, to allow multiple routes of access.

We focused on developing an emotion rating scale because emotions are central to the development of identity and relationships yet it can be difficult for autistic people to recognize and express emotions.^17–21 Although researchers have long speculated that autistic people have specific difficulties forming affective relationships and understanding emotions,^22,23 more recent research and accounts from autistic people suggest that emotion recognition difficulties observed among some autistic people are attributable to language difficulties and/or co-occurring alexithymia (difficulty in identifying and describing one's own emotions) rather than autism.^24–27

An emotion rating scale that includes both pictures and words may provide more opportunities for autistic people with language difficulties and/or alexithymia to express their emotions than purely text-based rating scales. Indeed, during the review process for this study, an autistic co-author (the artist who drew the AAPE) reported that he experiences alexithymia. Despite having highly advanced verbal skills, he finds it easier to identify emotions in pictures than words. Expressing himself through drawing has helped him learn to understand emotions more deeply. By developing a multimodal emotion rating scale, we hoped to provide autistic people with diverse communicative, emotion recognition, and cognitive skills with a tool they can use to express how they feel about their experiences so they can self-advocate and help improve available supports.

Learning from an established picture-based measure of emotion

The AAPE is inspired by one of the most widely used picture-based rating scales of emotions, the Self-Assessment Manikin (SAM; Fig. 1).²⁸ A fixture in emotion-related literature cited over 6000 times, the SAM has been used by researchers to address numerous research questions, most commonly the neural correlates of emotions. Described as a “nonverbal pictorial assessment technique,” the SAM provides participants with a 5-point scale for rating emotional valence, arousal, and dominance using a gender- and context-neutral avatar. Contrasting with the viewpoint that people experience discrete and universal emotions (e.g., fear), the SAM frames emotions as responses to stimuli that vary along key dimensions (e.g., valence, arousal, and control).²⁹ The SAM has proven useful as a method of rating participants' feelings about various contexts and prompts.^30,31 Our team became aware of the SAM after viewing a poster by a colleague who was adapting the SAM in collaboration with Deaf individuals during counseling sessions.³²

FIG. 1.

The Self-Assessment Manikin, an existing picture-based scale to measure emotional responses to stimuli in three areas: valence (A), arousal (B), and dominance/control (C).²⁸

Although the SAM effectively captures multidimensional aspects of emotions, the illustrations used for the SAM may not be entirely intuitive. The widely cited article establishing the validity of the SAM as a “nonverbal” assessment did not provide much detail about how the SAM was administered, instead noting that the SAM ratings used in the article had been obtained using the International Affective Picture System (IAPS).²⁸ The IAPS is a popular research tool containing emotionally evocative images that were developed by the creators of the SAM. The IAPS Technical Administration Manual instructs researchers to describe each of the SAM scales using a plethora of emotional vocabulary so that participants are able to use the SAM as intended by its creators.³³ For example, the manual describes the excited terminus of the SAM's arousal scale using the words “stimulated, excited, frenzied, jittery, wide-awake, aroused” (p. 4) and the calm terminus using the words “relaxed, calm, sluggish, dull, sleepy, unaroused” (p. 4). The instructions that are read to participants before they use the SAM to rate the IAPS are one and a half pages long. In a recent study where the instructions for the SAM were adapted for use with 10 autistic adults, the instructions to help participants learn how to use the SAM were ∼900 words long.³¹ While the designation of the SAM as a “nonverbal” assessment suggests that it may not require language to rate one's emotions, the lengthy verbal instructions that accompany the SAM suggest that the SAM requires substantial language-based clarification before it can meaningfully be used.

Existing rating scales developed using participatory processes

To the best of our knowledge, only two other research teams have documented a participatory process wherein people with and without disabilities collaboratively adapted established measures to increase their accessibility for people with developmental disabilities.^34,35 One team of autistic people, people with intellectual, physical, or sensory disabilities, and people without disabilities wished to examine associations between violence, disability, and health among people with developmental disabilities.³⁴ Nicolaidis et al. selected commonly used measures of depression, stress, post-traumatic stress disorder symptoms, somatic symptoms, and social support to include in a survey. The original measures' text-based Likert scales were flagged by some members of their team as confusing. Therefore, they changed the wording of response options to be more precise and added graphic representations of response items indicating frequency and/or valence. For example, a 5-point Likert scale with response items ranging from “none of the time” to “all of the time” was represented by five cylinders of incrementally increasing fullness. Items evaluating satisfaction were adapted to include graphic smiley faces with “very much satisfied” represented by a simple full smile and “not at all satisfied” with a simple frown face. When the adapted measures were used with a large sample of people with developmental disabilities, the internal inconsistencies of all measures but one were good to excellent.

A second research group collaborated with eight youth with developmental disabilities to develop the Pediatric Disability Inventory–Patient-Reported Outcome.³⁵ Researchers and youth group members examined and defined the construct of “functional performance” to reflect the needs and experiences of people with developmental disabilities, identified tasks to include within the assessment, and refined items. This process appeared to be substantially less participatory than the process developed by Nicolaidis et al.³⁴ as the youth did not play a role in developing the research question, selecting measures, or dissemination. However, pictures emerged as a useful tool to help the youth engage, aligning with Nicolaidis et al.'s findings.

Inspired by prior participatory research and associated calls to prioritize autistic voices and intellect in autism research to increase its social validity,^36–38 our research team has committed to conducting research using participatory methods, which include autistic and nonautistic researchers and community members in research conceptualization, design, and dissemination. As discussed above, adapting purely text-based response options to include pictures is a key strategy for including more diverse autistic individuals in dialogue about their experiences, supports, and research. In the current study, we develop a multimodal (composed of pictures and words) emotion rating measure that begins to capture the multidimensionality of emotions. Our work expands upon research using unidimensional picture-based response options assessing valence, such as the smiley faces used by Nicolaidis et al.,³⁴ by providing opportunities for people with diverse communicative needs to express a greater range of emotions.

Study Aims and Overview

The overarching aim of the current study was to develop an intuitive and humanistic emotion rating scale, inspired by the original SAM, which can be used for research purposes without (or with) written or verbal instructions for autistic and nonautistic participants. To achieve this overarching aim, we had three sub-aims:

• 1.Develop a new multidimensional emotion rating measure (the AAPE)
• 2.Revise the AAPE guided by initial evaluations
• 3.Use the AAPE in an applied setting

Methods and findings will be presented in four sections: The Participatory Approach section will orient readers to the values underlying this research and the strategies we used to align our work with these values. Next, we provide the methods and results for each aim. Each aim builds on the prior aim; we present methods and results for each aim in the order they were carried out to mirror our iterative research process. This project was approved by the institutional review board at the City University of New York.

Methods

Participatory approach

The research activities described in this article were conducted by a participatory group of autistic and nonautistic researchers that included the authors of this article and a larger group that collaborated more distally in this and other research initiatives. Authors of this work include a nonautistic doctoral candidate (A.R.), two autistic undergraduates (J.D.S., the artist who drew the AAPE, and A.J., who was involved with conceptualizing the energy dimension and revising the artist's renderings), an autistic academic (S.K.K.), and two nonautistic academics (K.G.L. and D.D.). Group members could join meetings in-person or virtually (depending on their location and/or preference) held once every 6 weeks. People who joined virtually did so via Skype using whichever method of communication they preferred (audiovisual video, voice-only, text-only, or some combination). For all qualitative data described in this article, coding was conducted by A.R. and J.D.S. Coders achieved greater than 80% agreement on at least 20% of participant responses in each code category.

The aims of this study were initially developed by A.R. and K.G.L. (neither of whom is autistic), after viewing the aforementioned poster using the SAM with deaf clients, as a direction for A.R.'s doctoral dissertation. The overarching idea for this study built on discussions between S.K.K. and K.G.L about the need for methods to reach autistic people who do not communicate through spoken and/or written language after they developed a study about autistic identity when they were both graduate students and found that it lacked representation by nonspeaking people.²

The idea to assess and adapt the SAM for use with autistic participants was then shared with the participatory research group. Group members expressed strong support for the idea, indicating interest in developing new visuals to represent each emotion. J.D.S., an autistic then-college student who is an artist and worked on this project as part of his successful honors thesis, agreed to draw the scale adaptations. Iterations of each dimension were shared at participatory research meetings where all group members could review and critique elements of each dimension (such as face shape, facial expression, and body position) and suggest changes. In an effort to be transparent about our methods and experiences, Table 1 includes comments from participatory group members about the research process and our communication as a research group.

Table 1.

Comments from All Authors About the Participatory Research Process

Process/Research area	Comment
Structure and development of large participatory group initiative	Overall, I would categorize the level of autistic involvement as “authentic engagement/collaboration” (Nicolaidis et al., 2019, p. 2010).38 The original decision to adapt the SAM was not itself participatory, but after hearing a presentation on it to the participatory group, I felt intrigued. When the study began I had amassed many years of experience with community-based participatory research with the Academic Autism Spectrum Partnership in Research and Education (AASPIRE), but other researchers lacked the knowledge to carry out that depth of community engagement … We tried to incorporate elements of the AASPIRE working process such as the keep/change self-reflective exercise (Nicolaidis et al., 2011), but we found ourselves running tight for time in our one-hour meetings every month or so (as opposed to two-hour meetings as often as every two weeks for AASPIRE). I recurrently had difficulty speaking that forced me to type in several meetings, but that occurred because of technological problems rather than due to my autism, yet we are considering moving to text-based meetings (Nicolaidis et al., 2011).39 The ongoing growth in the level of participation has been a pleasure to experience.—S.K.K.
	I would like to mention that both A.R. and K.G.L. were very kind about taking on executive functioning tasks related to my participation in the research. They were also very accommodating and flexible in their procedural and planning styles in general. If I had been left to coordinate the logistics of my inclusion, I would not have been able to make a meaningful contribution. On the whole, I thought that A.R. and K.G.L. were skillful and compassionate collaborators and allies.—A.J.
	Working in this participatory group was my first time engaging in authentic participatory research. Earlier projects focusing on autism had stemmed from my involvement as a Coordinator for Project REACH in which autistic voices were included, but in the form of participants and/or consultants providing input without involvement in the various steps of the research process. My involvement in this larger participatory group has been extremely beneficial to me as a researcher. Rather than input from autistic individuals that is then interpreted and/or presented to the larger scientific community by a neurotypical group of researchers, our projects are shaped by autistic voices at various stages of the research process (from planning to dissemination). My autistic colleagues have helped bring the initial ideas of A.R. and K.G.L. to fruition in a way that would not have been possible without authentic collaboration. As our participatory group grows, we have become even more neurodivergent and our projects reflect this. Along the way, the group has had to adapt to the divergent strengths and needs of its members—we have undergone regular reassessing and adjusting of our roles, responsibilities, and strategies, which I consider a key strength of a truly participatory group.—D.D.
	In 2013, when we started the mentorship program from which the participatory group that developed this study grew, we asked potential mentees to lead program development. Our stated intention from program inception was for the program to be participatory. However, the process of developing a truly participatory program took years. In retrospect, our initial descriptions of our program as participatory were premature. However, as autistic students and students with other disabilities stepped tentatively into leadership roles as mentors, public speakers, and researchers, they served as guides for other students who wanted to become leaders while sparking structural changes that better supported autistic leadership (e.g., collaborative guidelines). Mentorship, both formal and informal, is the core strength of our research group. As J.D.S.'s formal mentor, A.R. helped him blossom as a scholar and activist. Reciprocally, J.D.S. guided A.R. and the broader group by sharing his artistic skills and insights derived from his experiences at ASAN's Autism Campus Inclusion Program. As a more experienced participatory researcher, S.K.K. informally mentored A.R. while using insights derived from his experiences as a member of AASPIRE to improve our participatory processes. D.D. shared her insights as a prior Project REACH coordinator to guide A.R. in becoming the new coordinator and to inform participatory processes. As Project REACH's first Neurodiversity Coordinator, A.J. helped make the mentorship program and research group increasingly supportive of neurodivergence. We hope our work will encourage others interested in developing participatory initiatives by showing that you can and should learn as you go. Our participatory initiative emerged slowly and organically as the group collaboratively learned how to support future autistic and non-autistic leaders. We continue to learn how to improve the process through critical reflection and dialogue.—K.G.L.
	I feel that the overwhelming strength of this study and the other research projects our group is working on is the expertise shared openly and consistently by our autistic group members. Forming and strengthening relationships between autistic and non-autistic undergraduates, graduate students, academics, community members, and distinguished researchers is a unique research environment and one of mutual respect. Learning from each other and working within this group has been the most important part of my doctoral training.—A.R.
Communications and meetings	As the primary artist for the AAPE and an aspiring scientist myself who participated in data collection and analysis as a secondary coder, I felt I was important to the study … I worked with the other researchers, joining them on Skype calls and occasionally in person, to discuss methodology whenever possible.—J.D.S.
	On email chains about my scales, all authors—autistic and non-autistic—were invited to provide critique. This afforded me valuable insight not only in how non-autistic researchers interpret my scales, but how other autistic individuals might. In addition, I was accommodated in my difficulties with task management through personal reminders to look at an email chain or complete some task (as emails and Skype meetings with multiple participants can be overwhelming), something for which I was endlessly grateful … it was a pleasure to work with Project REACH and the other participants from different universities, and I look forward to collaborating with them more in the future.—J.D.S.
	Input from autistic and non-autistic researchers was invited and considered at various stages of the research process. Meetings have been held via Skype and/or in-person. Skype was a familiar platform for me (having used it throughout graduate school) and required no adjustment or accommodations on my end; however, there were technological issues that oftentimes limited the participation of some of the other members. As a result, we discussed moving to other platforms, such as text-based ones, but did not do so for this project. A.R. and K.G.L. did an excellent job of eliciting participation from all members via various modes of communication (e.g., Skype voice and text, emails, and Google Docs/Sheets). At times the Skype calls and lengthy email chains were challenging and limited my full participation (as well as the participation of other autistic and non-autistic members), however, there was always the ability to communicate ideas through the Google Doc used to plan/record meetings before, during, and after. Through these various modes, I believe that the voices of all participants were able to be heard and reflected upon utilizing multiple modalities.—D.D.
	We are still very much learning and growing as a group. I feel that our biggest areas for improvement are communication outside of structured meetings, consistent documentation and sharing of progress, and clear communication methods during meetings. Recent discussions have highlighted areas where we can evolve by trying non-video-based meetings and asking group members to write a summary of their contributions and thoughts within a shared document. Collaboration between researchers with numerous other commitments is always a delicate process and we experience these challenges within our group as well.—A.R.
Study design, idea generation	A.R., K.G.L., and I workshopped the survey on Qualtrics, trying to figure out the best ways to format the questions around the scales such that the students could guess at what the scale was portraying while also evaluating their responses accurately.—J.D.S.
	Autistic co-authors and I had the opportunity to substantially critique and influence all the major decisions, through at least video-based meetings, Google Docs, and e-mails; I feel satisfied our iterative process addressed my concerns. As a decade-long collaborator of K.G.L. and an independent autistic researcher, who had come to know A.R., I knew the team shared my values for neurodiversity and participatory research. K.G.L. and I recognized that the online surveys we had co-constructed had failed to recruit a sample with the developmental diversity of the autism spectrum, and long wished to develop a way for autistic people with more limited language skills to participate. This study represented an opportunity to try to enable more participation of autistic people in autism research as both co-researchers and as research participants.—S.K.K.
	While it may seem as though the paper outlines a linear process of iteration and revision driven solely by the priorities and expertise of the non-autistic first and last authors, that was not the case. Both K.G.L. and A.R. embedded multiple access points and modes of collaboration into each stage of the development process and privileged the impressions and input of their autistic co-authors while refining the emotion rating scale. For example, I proposed and advocated for the inclusion of the energy dimension based on my own navigation of my embodied experience of autism. Further information about my reasoning for doing so can be found within Table 2, Section H. Both K.G.L. and A.R. were supportive of its inclusion.—A.J.

AAPE, Multidimensional Visual Scale Assessing Affect, Anxiety, Pride, and Energy; SAM, Self-Assessment Manikin.

Methods: Aim 1—developing a new multidimensional emotion rating measure

We speculated that more intuitive (e.g., recognizably human) drawings than the SAM would be necessary to increase accessibility. We created two separate dimensions of arousal to measure both positive arousal (neutral to joyful) and negative arousal (neutral to anxious). An adapted scale for dominance, interpreted by this group as social dominance/confidence, was also constructed. Discussions within our research group resulted in multiple iterations of the picture-based adaptations to the SAM over the course of 1 year. Decisions around which emotions to include and how best to portray them using a humanistic, yet gender- and racially neutral, character emerged through extensive dialogue. Figure 2 depicts the process of iteratively adapting the AAPE. Table 2 depicts key insights of our group members throughout this process.

FIG. 2.

Diagram showing the iterative process of scale development, as inspired by the Self-Assessment Manikin, resulting in a 4-item visual rating scale of emotions. Affect (from negative to positive), negative arousal/anxiety (from calm to anxious), pride (from shameful to prideful), and energy (from depleted to energetic) comprise the completed scales presented in this study. A full description of the items and comments from participatory research group members on the design and iteration process are included in Table 2.

Table 2.

Reflections on Scale Development from Autistic Members of Our Participatory Research Group

	Description of scale item	Reflections on iterative process
A	First pilot version of an affect scale, adapted from the original SAM. • This scale fared uniquely well with interpretability and required little change beyond making it more humanistic.
B	First pilot version of a scale for positive arousal, adapted from the original SAM.	• For dimensions B and C, we initially thought about using descriptive line drawings beneath the faces as an additional prompt for participants. Group members decided that this was too much input and the lines were removed. • “I distinctly remember wanting to use hands for the B/C/F scales because of how expressive hands can get in autism … hands are a bit of a language unto themselves when it comes to the autism spectrum.”—J.D.S.
C	First pilot version of a scale for negative arousal, adapted from the original SAM. • Representing only negative arousal, this original version ranged from neutral to anxious.	• “Those used to have lines underneath them, which I thought made it clearer what it was about, but surprisingly others thought it was too much information … hmm, but this ended up being one of the most fun ones to draw I think”—J.D.S.
D	First pilot version of a scale for dominance, adapted from the original SAM. • This scale was removed after the first round of testing due to the group's shift toward developing scales which could be used in future research relating to autistic identity.	• “… a very hard thing to portray, got stuck on that for ages because the scales ideally depicted only one person from a standardized angle as well. How do you depict dominance without other people or changing the way the figure is perceived? As an artist, I'm still not sure.”—J.D.S.
E	Final test version of an affect scale. • Our research group decided to merge the pilot scales for affect and positive arousal to make this scale range from sad to joyful.	• “[It was] better to show a full range, which makes the construct clearer too”—J.D.S.
F	Final test version of a negative arousal scale. • A change from C, this dimension now ranges from calm to anxious rather than neutral to anxious.
G	Final test version of a scale for pride ranging from shameful to prideful. • This scale was added to align with other research conducted by our participatory research group related to autistic identity.	• “The hardest part of the whole process from my perspective—getting shame to look ashamed enough instead of grieving or otherwise tortured. The second panel where I was advised to move the hand/arm placement juuuust enough to convey the emotion (because covering the eyes a few degrees north or south apparently gives an entirely different feeling!) was a tug-o’-war of the subtlest kind.”—J.D.S.
H	Final test version of a scale for energy ranging from depleted to energetic. • This scale was added to align with other research conducted by our participatory research group related to autistic identity.	• “When explaining my lived experience of autism, I often describe it as a psychosocial metabolic disorder. There are some settings that deplete my stores of energy (due to overstimulation, having to ‘mask’ or appear neurotypical, etc.) and there are others that replenish it. The experiences that produce one effect or another can seem counterintuitive on the surface, but have the potential to lend greater insight into the innermost processes of autistic life and functioning. That's why I thought it would be an important dimension to include.”—A.J. • “I erased the faces as a secondary option because subtle changes in expression had previously been brought up as being ‘too much stimuli’ in other scales. I think the faceless ones are a little creepy, actually.”—J.D.S. • “I liked the energy bars; reminds me of the disability spoon theory.” [that represents the energy people with disabilities expend for daily tasks through spoons that become depleted until recharged through rest]—S.K.K.

Letters A–H refer to emotion dimensions labeled in Figure 2.

We used a computer-based survey to evaluate interpretability with minimal instructions of an unlabeled version of the SAM (Fig. 1) and an unlabeled initial version of the AAPE (Fig. 2; initial AAPE dimensions highlighted in gold). Participants were asked, “What do you think these pictures are measuring?” about each of the three SAM scales (valence, arousal, and dominance) and four initial AAPE scales (affect, positive arousal, negative arousal, and social confidence) with no further instructions. At this stage, we presented the images without accompanying text to evaluate their interpretability without reliance on text-based scaffolding. Our aim was not to evaluate the validity of the SAM, as we did not use the extensive verbal instructions that are supposed to accompany the SAM to clarify its meaning. Instead, our aim was to establish the degree to which the SAM and the initial AAPE were accessible as nonverbal emotion rating measures.

Autistic high school students (n = 15, M_age = 15.8 years, 93.3% male, 46.7% White/Caucasian, 20% Hispanic/Latinx, 20% Black, 20% Asian/Pacific Islander) were recruited from an informal educational program, and autistic college students (n = 16, M_age = 21.7 years, 87.5% male, 37.5% White/Caucasian, 12.5% Hispanic/Latinx, 12.5% Black, 18.8% Asian/Pacific Islander, 18.8% not reported) were recruited from a mentorship program at a large, public urban college. Autism classifications were ascertained by parent-report for high school participants and self-report and Individualized Education Plan documentation for college participants in this wave of data collection. Nonautistic students (n = 197) were recruited from the Psychology subject pool at the same college.

Results: Aim 1—developing a new multidimensional emotion rating measure

Among the combined 31 autistic high school and college students who completed the pilot survey, the unlabeled SAM dimension of valence (Fig. 1A) was most likely to be understood, with 81% of students correctly reporting that this scale measures moods, feelings, or level of happiness. Only 19% of students described a construct relating to “arousal” (Fig. 1B) when presented with the scale; many participants reported that the man was hungry or hurt. Only 6% of students correctly described the SAM scale for “dominance” (Fig. 1C) as the authors intended. Instead, students believed the dominance dimension represented body size or physical changes. Nonautistic college students interpreted the SAM with slightly improved accuracy: 93% demonstrated understanding of valence and 28% understood arousal. Yet only 16% understood the dominance dimension. The dominance dimension of the SAM also exhibited questionable convergent validity in the original article establishing the measure, even though text-based supports to guide interpretation were provided in that study.²⁸ These data indicated that the SAM is not accessible without accompanying verbal or text-based clarifications.

Data suggested that our adapted multidimensional emotion rating measure (the initial AAPE) was more accessible than the unlabeled SAM. Most of the 31 autistic participants (96.8%) correctly described the affect scale of the AAPE as depicting emotions, feelings, or moods. The scales measuring positive and negative arousal were similarly well recognized with 90.3% and 96.8% of the autistic participants correctly describing the emotions depicted in these scales, respectively. Participants used words such as “stressed” and “anxiety” to describe negative arousal and “joy” or “excitement” to describe the positive arousal dimension. However, only 58.1% of the participants correctly described the dominance/social confidence scale using terms such as “shyness” and “confidence.” This is a significant improvement when compared with the 6% of autistic participants who were able to correctly describe the “dominance” dimension of the SAM, but still below optimal for use in future research as intended.

Methods: Aim 2—data-informed iteration of multidimensional emotion rating measure

Given the suboptimal results for the dominance/social confidence dimension of the AAPE, we eliminated the dominance dimension at this stage. We decided instead to include dimensions depicting emotions that autistic members of our team felt would be most relevant to our long-term goal of using our emotion rating scale to evaluate autistic identity (to be described in a future publication). Based on previous research concerning camouflaging of autistic behaviors and the large amounts of energy autistic individuals report dedicating to camouflaging,⁵ we decided that dimensions of shame to pride and depleted to high energy would be useful. Camouflaging in this case means to engage in behavior that aims to mask or hide parts of oneself from others in an attempt to “pass” when interacting socially. The decision to focus on energy was sparked by A.J.'s personal experiences with camouflaging contributing to low energy. Sketches of these new dimensions are depicted in Figure 2 with accompanying examples of the decision-making process in Table 2.

We evaluated the final iteration of our Multidimensional Visual Scale Assessing Affect, Anxiety, Pride, and Energy, the AAPE, as an aspect of a broader online survey about the experiences and identities of autistic postsecondary students. A total of 72 students completed this survey with an average age of 24.03 ± 6.9 years. Students were 40.3% male, 43.1% female, 13.9% nonbinary, with 2.8% reported “agender” or “mostly male.” Students in this sample were 76.4% White/Caucasian, 6.9% Hispanic/Latinx, 9.7% Asian, and less than 5% Indigenous (n = 3), Black (n = 2), Pacific Islander (n = 2), Middle Eastern/North African (n = 1), or preferred not to answer (n = 3). Participants were eligible if currently enrolled as a university student (certificate through doctoral level, full-time or part-time) and if they had an autism diagnosis (verified by self-report). Participants represented nine different countries (n = 47 from the United States, n = 15 from the United Kingdom, n = 10 from other countries).

Participants were presented with the final AAPE emotion scales and asked, “What do you think these pictures are showing?” They were then told what the emotion for each particular scale was intended to portray and asked to “Please rate how much you agree that these pictures show (intended emotion)” on a 5-point scale from strongly disagree (1) to strongly agree (5). Affect was described using the labels “sad to joyful,” anxiety using “calm to anxious,” pride using “ashamed to proud,” and energy using “low energy to high energy.”

Results: Aim 2—data-informed iteration of multidimensional emotion rating measure

Coding of open-ended responses to the unlabeled dimensions indicated that our four emotion scales were generally well understood by autistic college students in our sample. Consistent with previous results, the scale for emotional valence performed best with 97.2% of the 72 participants interpreting the scale correctly. Our scale of negative arousal was also generally interpretable with 79.2% of participants correctly interpreting the scale. The new scale for energy level performed well; 84.7% of participants correctly interpreted it. Similar to the limitations of the original SAM dominance dimension and our revised dominance/social confidence dimension, our new shameful to prideful dimension performed the worst with 52.8% of participants correctly describing the intended construct. Participants who misinterpreted the scale for pride described the dimension as anxious or worried to calm (n = 10) or afraid (n = 4), and nine participants responded with “I don't know” when asked to describe the dimension.

When participants were informed what the scales were intended to depict, their quantitative ratings were better than average for all dimensions. Participants generally agreed that affect (M = 4.28, SD = 0.92) and negative arousal (M = 4.29 ± 0.86) depicted the intended emotional dimension, followed by energy level (M = 4.08 ± 1.00) and pride (M = 3.50 ± 1.20). It appears that simple text-based instructions indicating the poles of each dimension may help participants understand them. Findings suggest that the pride dimension requires further revisions before being used without accompanying explanatory language.

Initial application of AAPE in a technology camp for neurodivergent youth

In summer 2019, we used the final AAPE to provide youth in an informal technology education program with opportunities to share how they felt about different instructional strategies. The summer program is delivered through a 30-hour week, with separate programs for children (7–12 years) and adolescents (13–20 years).

Fifty students whose parents confirmed they had an autism diagnosis (25 children, M_age = 9.9, SD = 1.6, and 25 adolescents, M_age = 15.9, SD = 2.1) completed brief online surveys at the end of each week. There were asked, “How did you feel when your teacher asked you to [teaching strategy] this week?” The teaching strategies assessed included: “play games,” “group activities,” “activities on your own,” “taught the class by talking to everyone,” and “showed you how to do something and then let you do it yourself.” After each prompt, the four unlabeled dimensions of the final AAPE were presented (affect, anxiety, pride, and energy). Based on observation and informal conversations, students appeared much more willing to complete the survey with the AAPE than they had been over the prior three summers when we had used purely text-based surveys.

The internal consistency of the AAPE, or degree to which positive affect, low anxiety, pride, and energy aligned with one another, was generally acceptable for all activities in the full sample. However, internal consistencies tended to be higher for children than adolescents: games, α = 0.70 (child α = 0.83; adolescent α = 0.53); group activities, α = 0.75 (child α = 0.87; adolescent α = 0.48); individual activities, α = 0.85 (child α = 0.92; adolescent α = 0.71); teacher talking, α = 0.82 (child α = 0.79; adolescent α = 0.84); and doing it yourself, α = 0.81 (child α = 0.89; adolescent α = 0.64). The difference in internal consistency between children and adolescents may reflect heightened distractibility among adolescents, which, anecdotally, has been observed by our research team and/or the more complicated (and sometimes contradictory) emotions that adolescents experience relative to children.⁴⁰ The adolescent classroom at this organization also serves students across a wider age range when compared with the child-age classroom, and this variability may also impact the AAPEs internal consistency.

A repeated-measures analysis with age (adolescent vs. child) as a between subjects factor revealed differences in the affective dimension across activities, F(1, 48) = 3.15, p = 0.016, and no interaction with age (p = 0.15). Follow-up analyses revealed that games elicited more positive affect than group activities and the teacher talking (ps < 0.01). An analysis with anxiety as the outcome measure revealed differences in anxiety, F(1, 48) = 5.15, p = 0.03, and no interaction (p = 0.54). Students rated the teacher talking as more anxiety-provoking than games and solitary activities (ps < 0.04). A similar analysis revealed differences in energy, F(1, 48) = 2.68, p = 0.03, and no interaction (p = 0.51). Students found games more energizing than the teacher talking and group activities (ps < 0.04). Doing it yourself was also more energizing than the teacher talking (p = 0.05). Consistent with the limitations in the unlabeled pride dimension observed when addressing Aim 2, no contextual differences were observed for pride (p = 0.16).

Students were also asked if they would like to return to the camp in the future. Heightened positive affect, pride, and energy during games and group activities, energy during solo activities, and positive affect in response to the teacher talking were all associated with heightened desire to return (ps < 0.04).

Discussion

This article presents the development, iteration, and initial evaluation of a novel, picture-based, multidimensional measure of emotions developed by a participatory group of autistic and non-autistic scholars. Three of the dimensions (i.e., affect, anxiety, and energy level) received acceptable ratings for future research at this time, whereas the fourth dimension may require further adaptations to more clearly capture the construct of “pride.” This dimension depicts the most complex emotion chosen by our group and required the most extensive editing during development. We experienced the most disagreements among group members on how to best portray this range of emotions and spent significant time editing character elements, such as hand, eye, head, shoulder position, and facial expressions.

While there may be room for improvement in how pride is portrayed within the AAPE, difficulties in recognizing complex emotions, specifically those that index social norms, may also be a reason for decreased comprehension of this dimension. Previous research on emotion recognition in autistic people suggests that understanding complex emotions may be more difficult than understanding the more straightforward emotions portrayed by our other dimensions of the AAPE that elicited better comprehension.⁴¹ Indeed, prior research suggests that autistic children may experience specific difficulty in recognizing embarrassment and shame relative to nonautistic children.⁴¹ The authors attributed this difficulty to reduced sensitivity to violations of social norms. They did not, however, assess for potential alexithymia. As seen in some open-ended responses from participants who cited other emotions such as anxiety or fear when asked to describe the pride dimension and as noted by an autistic reviewer of this article, the experiences of shame and pride may differ between autistic and nonautistic people. This dimension may also be difficult to identify without a specific situation to react to.

A follow-up study comparing interpretability of the AAPE among autistic and nonautistic students may help to disambiguate the reasons why the pride dimension proved hardest to capture. We recommend that researchers include the pride dimension in their future research but that they also ask participants to share what each unlabeled dimension means to them and then provide text-based labels to ensure that the dimension is interpreted as intended (Appendix Fig. A1). Providing open-ended opportunities for participants to interpret and critique measures is a valuable extension upon participatory research partnerships as it allows participants to also contribute to measure refinement and evaluation of the validity of research methods.

Overall, the AAPE makes a significant contribution to the current emotion literature and has potential for applications beyond autism research. Our initial application of the AAPE with autistic children and adolescents in an educational program suggests that it is engaging, generally has acceptable internal consistency (particularly for children), and is informative. As expected, autistic youth's ratings of instructional practices using the AAPE largely aligned with the central premise of Universal Design, which learners prefer multimodal activities rather than just being talked to.⁴²

We also believe that documentation of our participatory research process throughout this study, including a discussion of all authors' contributions from idea generation to data analysis and reporting, and the inclusion of autistic authors' voices when reflecting on the overall research processes, represents a move toward a fully transparent participatory model that we hope will be replicated in future publications. We are also learning as a group how to better accommodate diverse communication styles and executive functioning challenges (which are not specific to autistic collaborators). Difficulties with technology during virtual meetings, more advance planning, and clearer documentation of the decision-making process are areas our group has been working to improve. While it is promising that research groups are increasingly moving toward participatory models guided by helpful recommendations from more established groups,³⁸ articles often do not yet include this level of detail when describing participatory models.⁴³

Limitations

While results are promising and indicate that the AAPE has utility, there are limitations in sampling and methodology. We conducted initial assessments of construct validity but did not systematically assess convergent and divergent validity. The use of both contextually situated and broader online samples is a strength of this study. However, the racial and ethnic diversity of participants was limited. The AAPE should be assessed with a more diverse sample and in relation to other measures to confirm its validity. Autism diagnoses for participants were based on either parent-report (for children and adolescents) or self-report (for the majority of college students who participated in this study); our inability to independently verify diagnoses is a limitation of this study.

An original goal of this study was to develop a scale that could be used without any written or verbal prompts to assess emotions and autistic identity with nonspeaking autistic people. After evaluating the degree to which the dimensions were interpretable without text, we recommend incorporating text-based prompts, so as not to alienate people by assuming a false equivalence between nonspeaking and low literacy. Unfortunately, we have not yet been successful in our attempts to recruit nonspeaking participants, so do not yet know if the AAPE will be interpretable or reliable for nonspeaking people. It is possible that the AAPE may require additional iterations or supports to be suitable for a population with more diverse communicative strengths and challenges. We hope to expand our participatory process to include nonspeaking individuals in future iterations to ensure that our measure will function as intended. We believe that it is essential to take these steps in our future research to ameliorate the gap in the literature surrounding autistic identity in nonspeaking people.

Conclusions and future directions

The AAPE was well received by participants and has utility for future research studies. Additional editing may be required to represent more complex emotions, namely pride, before using this particular dimension in future research, especially without accompanying text. Additional testing is needed to assess feasibility, validity, and reliability of the AAPE with people with more diverse communication skills.

In reflecting upon our overarching aim, to develop a multimodal emotion rating scale to provide autistic people with diverse needs and skills with a tool they can use to express emotions, we have thus far developed a scale with a participatory group of autistic and nonautistic researchers and tested our scale with both autistic college students and school-aged participants. Internal consistencies were promising for children but show room for improvement when used with adolescents.

As discussed above, we intend to report on the use of the AAPE to rate picture-based depictions of autistic identity in a future publication. Work with autistic people has characterized the experience of being autistic and the ways autistic people make sense of the autism label.^4,44,45 Some studies have reported negative experiences associated with autism including feeling the need to camouflage or mask traits to be perceived as normal by society, social groups, and employers,^5,46 whereas others describe strengths associated with autism such as strong memories, advanced knowledge, and systematic thinking.⁴⁷ These studies have provided a glimpse into the lived experiences of autistic people. However, specific emotions associated with these experiences and the degree to which these experiences are shared with autistic people who do not communicate through the language-based methodologies used in these studies remain unclear. The AAPE is a promising tool to begin to address this lacuna in the literature. The AAPE, available open access from any author of this report (Appendix Fig. A1), is accessible and engaging, and has utility for people with diverse communicative preferences.

Appendix

APPENDIX FIG. A1.

Images used in the Multidimensional Visual Scale Assessing Affect, Anxiety, Pride, and Energy (AAPE). Scoring guidelines: Our research team currently scores the AAPE numerically by assigning a value from 1 to 5 for each emotion response option from left to right, with the exception of Anxiety, which is reverse scored. For example, joyful, calm, proud, and high energy each receive a score of 5, whereas sad, anxious, ashamed, and low energy each receive a score of 1.

Authorship Confirmation Statement

A.R. helped design this study, processed and analyzed the data, conducted the literature review, and wrote many drafts of this article. This work served as one of four studies included in her doctoral dissertation. J.D.S. was the undergraduate artist responsible for creating and editing all versions of each emotion dimension and served as the secondary qualitative data coder for all qualitative participant data. S.K.K. (in collaboration with K.G.L.) helped conceptualize the study design in discussions about the need for picture-based research measures while a doctoral student himself and contributed to the literature review, scale development, and writing as a postdoctoral Research Fellow at the University of Exeter, supported by the Wellcome Trust, grant number 108676/Z/15/Z. A.J. was involved in the development and iteration of scale dimensions and assisted in editing this articlet. D.D. was involved in the design and iteration of scale dimensions and assisted in the editing of this article. K.G.L., who is A.R.'s doctoral advisor, developed the idea for this study with A.R. and S.K.K., played a leading role in designing the study, and contributed very substantially to the literature review and writing of the article. All authors contributed significantly to participatory research group meetings in-person or online and contributed their ideas and time to the development of this article. All co-authors have reviewed and approved the submission of this article. This article has been submitted solely to this journal and is not published, in press, or submitted elsewhere.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

Funding for this project was provided by the FAR Fund and the Graduate Center of the City University of New York.