“We have been magnified for years - Now you are under the microscope!”: Co-researchers with Learning Disabilities Created an Online Survey to Challenge Public Understanding of Learning Disabilities

Abstract

Public attitudes towards learning disabilities (LDs) are generally reported as positive, inclusive and empathetic. However, these findings do not reflect the lived experiences of people with LDs. To shed light on this disparity, a team of co-researchers with LDs created the first online survey to challenge public understanding of LDs, asking questions in ways that are important to them and represent how they see themselves. Here, we describe and evaluate the process of creating an accessible survey platform and an online survey in a research team consisting of academic and non-academic professionals with and without LDs or autism. Through this inclusive research process, the co-designed survey met the expectations of the co-researchers and was well-received by the initial survey respondents. We reflect on the co-researchers’ perspectives following the study completion, and consider the difficulties and advantages we encountered deploying such approaches and their potential implications on future survey data analysis.

Introduction

The authors are a UK-based group consisting of academic researchers embedded alongside members and staff from Heart n Soul, a creative arts organization which believes in the power and talents of people with learning disabilities (LDs). All members of Heart n Soul identify themselves as having LD and/or autism (for a holistic picture of what LD entails in the context of Heart n Soul, see https://www.mencap.org.uk/) but they are not required to specify a type of diagnosis or severity (as per the “deficit model”). Others describe themselves as neurodiverse (ND which often incorporates LDs and/or autism). As a diverse team involving individuals with LDs and/or autism (including ND), together we performed a type of collaborative and participatory ‘co-research’ called ‘inclusive’ research [132] whereby people with LDs and/or autism have ownership of research and their concerns are at the forefront [13, 26, 57, 79, 86, 121, 132, 140].

In ‘inclusive research’, there are several ways to highlight the contributions of non-academic researchers with disabilities to the research process [134]. Here, we use the term ‘co-researcher’ to recognize the potential differences in perspectives and skills of academically trained researchers and those with lived experiences of LD and/or autism but without formal research training [134]. Eight members of Heart n Soul were co-researchers [Pino, Lizzie, Donald, David Mc, Michaela, Mark S, Robyn, and Kelly (a pseudonym)]. In line with a fully inclusive research process [127], co-researchers are also co-authors, except for one who chose not to be named and not to be an author. Throughout co-research, the academic researchers were receptive and responsive to the needs, ideas, and concerns of Heart n Soul.

In the UK, conditions for learning disabled people are declining. In England, 1.5 million people have LDs, and 50% report feeling lonely [22, 41]. Further, only 6% of adults with LDs were in paid/self-employment in 2015 with substantial geographical variations across councils [96]. There is continuing evidence of poor practice in specialist hospitals for people with autism and LDs, and people with LDs are three times more likely than the general population to die from preventable causes [49, 96]. UK local authorities have cut care services and removed benefits from all except those with the most critical needs (12% of all people with LDs in England, 2011) and reduced community provision. This significantly impacts quality of life for people with LDs, particularly for those with mild or moderate LDs. In contrast, multiple studies indicate that attitudes towards LD amongst the general public are mostly positive, inclusive and empathetic [21, 28, 75, 88, 94, 113], even if this does not match the experiences of family members [2, 69, 72] or members of communities like Heart n Soul.

Working with Heart n Soul to address some of the disparities above, we aim to utilize the potential malleability of social norms through online communications. To move towards this aim, we established a team of co-researchers to launch the first online survey led by people with LDs to challenge public understanding of LD. The aim was for co-researchers to derive and ask questions in ways that are important and meaningful to them, and representative of their lived experiences. As summarised by one of the co-researchers at the start of this project, we aimed to turn the microscope and the magnifying glass away from us: “We have been magnified for years – now you are under the microscope!”.

Researchers with LDs have the power to self-represent and self-advocate and, therefore, methods and outcomes must be accessible [28, 82-93]. Our co-researchers actively made decisions on Which questions should be asked in the survey, How they should be asked; Who should ask each question and When i.e. in what order. To enable meaningful participation [81], we first worked with co-researchers to co-design an accessible online platform for deploying accessible surveys. Co-design happened as a consequence of co-research and a subset of co-researchers had dual roles as co-designers.

In these efforts, we were guided by the principles of ‘inclusive research conducted with, by and for people with LDs’ [80]. Co-researchers in the research team argued strongly that the survey should help any potential respondent “understand how they are and how they would like to be”, and “how people with LD and autism are.” This way, our co-researchers perform research not only on the lives of people with autism and LD as reported in previous studies [31, 84]. Therefore our research strategy goes far beyond performing inclusive research which is relevant to people who are subject to it [122].

We describe and evaluate how the accessible online platform was developed, and how co-researchers created an online survey using it, whilst reflecting on how we utilised inclusive research elements and practices in achieving the study goals. We present the survey as a creative research output, therefore not a traditional survey, but still with potential to capture information in a way that meets the expectations of our co-researchers. We describe our experiences, discuss key challenges in work of this kind, particularly how perspectives of co-researchers with LDs drove the research process, and how such outputs have the potential to produce meaningful outcomes. The survey results are not discussed in favour of the methods used to create it, with the focus on the diversity of skills and abilities rather than individual disability labels and types of medical diagnosis.

Related Work

Attitudes Towards People With LDs

Many studies have taken place in healthcare or social care contexts to evaluate or improve services for people with LDs [12, 15, 30, 37, 62, 67, 87, 135]. Family members of people with LDs and education workers have also been the subjects of attitudinal studies [18, 44, 45, 47, 99, 115, 118, 119]. This research is key in understanding systemic and personal viewpoints. However, many participants have a high level of knowledge of LD, either through specialist professional training or through frequent caregiving contact [30, 45, 47, 61, 63, 87, 115, 118, 119, 128]. Therefore, the findings do not represent the attitudes of the general population [4, 51, 93, 113].

Research aiming to achieve an overview of public attitudes vary in their success at recruiting representative participant populations. Due to the ease of recruiting university students within academic contexts, participants often have a higher than average level of education [1, 42, 55, 90, 94]. This is crucial, as education has been linked to more accepting attitudes towards LD [1, 74, 88, 90, 113, 139]. Younger people also display more positive attitudes [74, 88, 90, 113, 139] and many participant populations are fairly young [111, 113]. Participants’ gender is also problematic in surveys of the public, with many studies relying on heavily female response rates [30, 37, 45, 62, 115]. Women show more inclusive attitudes than men, indicating that they feel more similar to people with LDs and are less likely to exclude them [36, 73, 87, 90, 110, 130].

Measures assessing attitudes often center around social distance and the willingness of participants to associate themselves with people with LDs in different contexts and at different levels [14, 55, 88, 94, 105, 108, 111]. Commonly, the Community Living Attitudes Scale - Mental Retardation [48] does this [9, 74, 87, 92, 106, 108, 116, 124]. Assessing the knowledge of and beliefs about LD among the public has been a relatively new research area, with The Intellectual Disability Literacy Scale being specifically designed to evaluate these components in lay people and across cultures [110].

Inclusive Research and Co-Design Practices

Although individuals with impairments are often included in accessible technology design, particularly in the development of assistive technology, there is a focus on those with motor, hearing or visual impairments [76]. There are a limited number of methodological guidelines and frameworks to guide design for ND adults, although several models exist to aid design with children [10, 11]. Thus, the adult ND community is often overlooked despite cognitive impairments being the second most prevalent form of disability in the US [85]. Therefore, there is a need for novel co-design methodologies to enable the development of innovative applications that carefully consider and address the specific needs of the neurodiverse.

Inclusive research tenuously linked to LD attitudes does exist [16, 53, 68], but there is a paucity of research stemming fully from the ideas and wishes of people with LDs [83] and in which these co-researchers are acknowledged as authors [127]. As a consequence, people with LDs may continue to be excluded from the conversation when it comes to public attitudes towards them [6, 32, 102, 121]. Additionally, without the input of people with LDs, it is difficult to optimally implement education strategies stemming from knowledge of public attitudes [136]. To improve the lived experiences of people with LDs, they need to be involved in participatory research and application design from the start [71, 78, 133].

Several participatory frameworks have been proposed to support design with the ND [35, 39]. Whilst most of these frameworks were created with children in mind they may have utility for adults. In particular, the Diversity for Design framework [10] has focused upon creating a structured environment by sensitively understanding participants’ culture with additional tailored supports to individuals. Crucially, this framework also seeks to reposition individuals with LD and Autism by focusing upon their strengths to better understand the complexities of their interactions and experiences.

Changing Attitudes to LD

Some research has intended to measure attitudes towards LD so that an educational resource could then be created to improve these attitudes [30, 37, 62, 67, 74, 94, 101, 123, 130] or to pilot education interventions without assessing existing attitudes [23, 52, 97, 114, 131]. However, no studies exist whereby the survey itself is intended to educate the public and to reduce stigma towards the learning disabled community [107, 112].

Whilst online forums/social media have been used to recruit participants [14, 90, 101, 109, 111] and to administer questionnaires [14, 101, 109, 111], there is a gap regarding the use of creative technologies to present an accessible, engaging and educational survey, despite the benefits of similar interventions in reducing mental health stigma [25]. Well-designed digital questionnaires have the advantage of reaching a population that is more likely to demonstrate honesty [64], comprising a wider demographic spread [59, 66, 82, 112, 129], more so than other methods such as telephone surveys [21, 73, 74, 89]. Through video questions asked directly by people with LDs, it is possible to avoid ambiguous vignettes or lengthy descriptions that can exclude or confuse participants when it comes to defining ‘LD’ [109, 111, 136]. People with LDs are motivated and able to complete tasks using technology if the interface is accessible and the task is engaging [24, 27, 50, 56, 65, 77, 91, 95, 100, 117, 137]]. As people with LDs also constitute members of the general population, we need our survey to be navigable to everybody. The use of creative technology should ensure that this is the case, as it can be used by, and designed by [3, 117], people with LDs.

Methodology

We engaged in an inclusive co-research process that led us to deploy a number of methods to meet our goal of producing a survey with co-researchers with LDs and Autism. These methods arose directly as a result of discussion with co-researchers. They included i) Design Thinking / co-design to facilitate design and development of the platform [19]; ii) elements common to Team Ethnography for the most inclusive and holistic understanding of the survey development process.

Design Thinking and co-design were used within co-research to better incorporate the needs of the ND community including LD & Autism [34]. For this study, co-design is understood as an extension to participatory design where stakeholders are involved in the design process to better understand their aspirations, needs and experiences [54, 126].

The elements of Team Ethnography [33] included observation, interviewing, and inductive analysis of emerging data using both digital and non-digital tools. Team members coordinated, shared, and discussed their observations and reflections. This way, we have obtained multiple accounts of the same event adding to the validity of the qualitative research process [33]. For example, the Project Coordinator recorded her reflections about our journey on the Padlet platform, which we used as a mechanism for recording many forms of research data¹.

First, we developed a protype survey platform through co-design with a subset of co-researchers. We then expanded this work to incorporate a larger team when designing the survey. These processes led to iterations of platform and survey development described below (Figure 1).

Figure 1. Research Activities Timeline.

Survey Platform Development

In order to support the design of accessible web-based surveys as part of the study, we designed and developed a new accessible survey platform to better enable our co-researchers in the process of survey design.

Goegen, Radhill and Daniels (2018) [43] suggest universal design guidelines for survey elements such as multiple-choice questions and Likert scales to enable surveys to be accessible. However, these guidelines were developed without direct input from the LD community. As such there is a significant paucity of research concerning survey design for and from the LD community to guide design and development. In order for the survey platform to be digitally inclusive, a structured design workshop informed by Benton, Vasalou, Khaled, Johnson & Gooch (2014) [11] using the Design for Diversity (D4D) framework was run with co-designers from the LD & Autism community to elicit requirements. D4D enabled the research assistant to support the culture of the LD & Autism co-researchers by utilizing co-design methods aligned with their interests and communication abilities (e.g. sketching and drawing elements of a survey user interface). The framework suggested consideration be given to tailor the co-design workshop to accommodate individual differences (e.g. personality and attention span) which was particularly important as there was significant diversity in terms of impairment across the co-designers. The workshop was set in a quiet and calm setting to minimize sensory overload.

Six people who had previously used online surveys participated in the co-design workshop. Three co-designers were in the co-research team. The remaining three were members of Heart n Soul. One co-designer had previously participated in a design workshop which helped give reassurance to the others. Two additional assistants were also present to help encourage engagement and participation. To support a structured environment, the workshop employed a series of co-design user experience methods which also served as probes to elicit requirements. Two proto-personas - one neurotypical (NT) and one ND - were created [20]. The aim was to create a design that was highly accessible. The workshop began with a discussion of the ND persona and how it could be refined to be more representative. The co-designers were very engaged and keen to be heard. The positive and negative elements of surveys previously completed were explored. These experiences were added to the ND persona so that it reflected experiences and co-designers were encouraged to draw potential paper prototype solutions to the negative aspects of online surveys (See Supplement No.1 for examples of paper prototyping).

The outputs from the workshop were synthesized by the academic researchers with recommendations from the Web Content Accessibility Guidelines (WCAG 2.1) and the AASPIRE (Academic Autism Spectrum Partnership in Research and Education) Guidelines for Autistic Web Users [98] and used by the research assistant to create an initial interactive web-based survey prototype.

Below, we describe our six-month journey from when the idea of creating the Team of Co-researchers was initiated through survey development and production to its testing and improvement. Throughout the paper, for content brought about by the Team of Co-researchers that is more sensitive in nature, we do not refer to a specific co-researcher in order to protect them from unnecessary judgements and misconceptions. During the process of survey design, workshops and meetings relating to this activity were described as ‘Tea Parties’ by co-researchers.

Survey Design Workshops: Tea Parties

Pino, a musician with lived experience of LD, organized the first Tea Party in The Hub “to know how and understand how people [with LDs] feel” as the first step to derive questions to be asked to the public in the survey platform. Subsequently, the idea was to identify those individuals with lived experience of LD who would like to form a Team of Co-researchers to work very closely together on the survey development. Therefore, a series of further Tea Parties were organized to generate initial ideas and explain to the Team of Co-researchers what research is, to create a story board with meaningful questions to inform the survey, to reflect on the usability test results and to decide on the final set of questions and how they should be asked. A standard meeting agenda outlining the expected outcomes, an accessible meeting agenda, as well as an accessible invitation were prepared prior to each Tea Party. At every stage of the research process, we had multiple conversations with co-researchers reflecting on their involvement and hopes for the project.

Survey Testing

We engaged in two rounds of iterative survey testing. The first round tested the survey platform only. The second tested both the survey platform and the survey that had been designed by the Team of Co-researchers. The second round of testing featured a greater number of co-researchers and iterations, which we describe in detail. To evaluate the survey platform prototype two usability tests were conducted with both ND and NT participants. A convenience sampling strategy was utilized to recruit a subset of co-researchers. The focus of the informal tests was to gain design insights to improve the prototype. Usability testing within the Design Thinking methodology can be utilized iteratively to glean key design insights which can be implemented quickly using an agile methodology for software development [46]. Typical measures used to evaluate usability include success rates, time on task, errors and subjective responses regarding satisfaction [7]. Elements of the D4D framework were incorporated into the usability test protocol. Small modifications were made to the to ensure the test was accessible. For example, use of the Think Aloud protocol [38] was rejected as it may cognitively overwhelm ND participants. Instead, after each task was performed the moderator asked how the user felt about the task and what they would like to change about the functionality.

The first usability test utilized a MacBook Pro laptop, an iPad and iPhone XR from 5 ND and 5 NT participants. A mock survey was created using five demographic questions and three survey questions, one of which was a video question. In addition to the moderator who facilitated the test, a data logger and observer were also present to record participant behavior and comments and support the users during the test; these individuals were neurotypical. They also supported the participants in understanding what a usability test was and its purpose. Three ND participants were concerned about their computer skills. However, the moderator made it clear that it was the survey platform that was being evaluated. Participants were then asked to complete a short paper-based demographic survey about their disability and how they rated their computer skills.

One key aims of the test was to examine the usability, accessibility and comprehension of survey questions that utilized different mediums. A scenario was created where users were invited to take part in a survey via email or mobile device. All participants were given the scenario and asked to complete the mock survey using a device they would typically use to answer a survey. All participants decided to use the iPhone. Completion of the survey was split into several tasks. Task 1 was to complete the demographic questions successfully, the remaining three tasks comprised of answering each of the survey questions. A short post-test paper-based questionnaire was completed by participants to ascertain their satisfaction regarding the prototype. The moderator completed the questionnaire as a mini interview for those who felt they could not respond in writing.

Results

Survey Design Workshop Findings

A number of negative comments stemmed from demographic questions posed in surveys. Questions regarding gender and disability were cited as particularly frustrating. One co-designer commented “I don’t think of myself as having a disability so I wouldn’t tick that box…no.” Another commented, “I don’t fit into any of those boxes they write.” Similarly, regarding questions relating to gender: “I don’t like it when you have to choose only male or female. I can’t fit in what they want.” Using paper prototyping co-designers sketched open text responses for questions regarding gender and disability. These were considered important to the co-designers to reduce marginalization by restrictive predefined responses e.g. male/female.

Most commented on the lack of accessibility of online surveys. They found completing a wholly text-based survey difficult. This may be due to the overwhelming cognitive demands of literacy [17]. In particular the language used in questions was confusing to them. An example given “I don’t like it when they ask you a question and then they ask you again but in a different way… it’s like they are trying to trick you. “ Thus, to minimise confusion and facilitate engagement and trust, co-designers suggested a key requirement was the ability to ask and answer survey questions using multiple mediums (e.g. video and audio) and where text was used the language was clear and simple.

There was no difference in time on task, completion rates or error rates between the NT and ND. Most participants utilized the multimedia alternatives to questions (5 NT and 4 ND) confirming the use of co-designer outputs was universally useful and accessible. More than half of the ND participants used text-based responses with the remaining choosing to respond with audio supporting multimodal question functionality. Those that chose audio commented they found it easier than writing. This finding is similar to those from Williams & Hennig (2015) [138]. However, ND participants found the language relating to GDPR difficult to understand and suggested having videos of the consent questions. This suggestion was immediately incorporated in to the design.

First Tea Party

In preparations to the first Tea Party, Pino and Michaela had a conversation about the study aims¹. Based on the transcript of this conversation, we colour-coded the text manually and derived emerging themes as connected clusters of ideas, which were then discussed, simplified and subsequently used by Pino for further discussion at the Tea Party.

Overall, thirteen individuals who “can talk and push it!” (as described by Pino) attended the first Tea Party. All participants signed an informed consent (compliant with UK research ethics requirements). Pino started the discussion by first asking all “What does it feel like working with people with LDs?”. The founder of Heart n Soul summarised it as “a joy and a privilege”, after which he reminded us of the project’s big picture: “And you know, here we are sitting in The Hub […] which is the world’s most important research space […] so we’re making progress”. Thereafter, a very passionate and emotional discussion emerged around the following themes prioritised by the academic researchers based on the participation and the post-meeting transcript review: Panic and the ‘WOW’ Effect (David Mc: “and they [people without LD] are like, you can see the panic in their faces the first time they walk in, but by the end of it they’re like, oh my god I can’t believe it [the talents of people with LDs]”); The value of Heart n Soul (Lizzie: “Without Heart n Soul there would be no me”); Love (Kelly: If the world was such a beautiful and loving place! I want to continue making it happier for people. People like us - we are left alone…”).

Further Exploration of Emerging Themes

Based on conversations at a Tea Party, themes were further explored and refined as follows¹:

• People’s perceptions (i.e. How people understand things): co-researchers are curious how the public perceives them at first (first impressions) (David Mc, Lizzie, Michaela) and then in a situation when they can showcase their talents (David Mc, Lizzie, Kelly).

• Pride: co-researchers are proud of their skills and disability (Mark S, Kelly) and are proud to be here with us at The Hub (David Mc, Donald).

• Asking questions (to and from the public): Pino and Kelly would like to take this opportunity and ask the public whether they have any questions to them. As a team, we framed this idea as “a conversation with the public”.

What is Research?

Michaela and a freelancer experienced in running creative sessions for people with LDs, facilitated a Tea Party titled “What is research?”. As part of the research activity involving a discussion and a role play, we wanted to explore the co-researchers’ understanding and views of the following: that they own the ideas and perform co-research with the rest of the team; that their questions are typed and stored by the academic researchers as part of the research process; these questions are going to be shared with the public in the Survey Platform; subsequently, the information collected will be analysed and then we will write ‘A book of what we said’ as suggested by Pino; they will decide whether they want their name to remain private.

Creating a Story Board to Inform the Survey

At this point, the eight individuals who were most committed and enjoyed coming to The Hub following the Tea Parties officially formed the Team of Co-researchers. We were hoping for the Team of Co-researchers to create at least one story board in the space of three meetings of three hours in duration each to be used as a template for the first survey. To support co-researchers in this task, we involved an artist specializing in collaboration and visual communication (co-author) to help us with visual recording and facilitation. Also, we hired a professional film-maker to proceed with filming of the initial ideas generation as part of practice and creative research process to give an opportunity for the Team of Co-researchers to express themselves in a variety of ways. Prior to each meeting, we prepared a clear agenda outlining a plan and expected outcomes. We remained flexible and adapted to the needs and ideas of the Team of Co-researchers at every stage. Each meeting was co-facilitated by several members of our team depending on their availability and the meeting objectives. Each meeting begun with an Introductory Session consisting of: relaxation; significance of the project and the role and importance of the Team of Co-researchers; agreement on how we will work together in a meeting; recap of the previous meeting. Prior to the first meeting, all co-researchers received an accessible information package¹.

The Story Board

The Team of Co-researchers came up with three story boards¹ of which only one was fully developed and with the potential to measure baseline public understanding of LD as judged by the academic researchers (Figure 2). The story-board was also in accordance with the Team of Co-researchers’ earlier suggestions to assess how people understand things in the first survey. The story board was developed by Pino, Robyn, and Kelly. Here, we focus on Who asks a question and How; the animations in the background of the questions-videos constitute an integral part of the research process but are outside of the scope of this analysis.

Figure 2. The story board developed by co-researchers.

Filming of the First Version of the Survey

For those co-researchers that wanted to be filmed, we provided each with the visually depicted questions prepared by the artist to be used as prompts in the filming studio. Several co-researchers prepared additional questions and made drawings of their ideas on paper which were subsequently used as prompts for filming¹. The founder of Heart n Soul reminded co-researchers to “think about questions that as many people as possible could answer”.

For example, a question prepared by Pino “Do you respect us the way we are” is more appropriate than “Why don’t you respect us?” as suggested by another co-researcher. During the meeting, the Team of Co-researchers also discussed the survey development process. For example, one of the co-researchers posed a question to the rest of the team: “Do you think we are doing the wrong thing speaking who we are?”.

After the meeting, the professional filmmaker processed the videos of the questions according to the selected story-board, consulted with the Team of Co-researchers, and a version of the survey was created with the following script:

First Version of the Survey²:

1. Question No.1 (Lizzie):

   1. We have been underneath the microscope for years. What do you see when you see me? (video)
2. Question No.2 (Pino + people):

   1. Do you trust us? (photo)
   2. Can you tell me why you chose that answer? (text)
3. Question No.3 (Pino):

   1. Are you frightened of people with learning disabilities? (video)
   2. Can you tell me why you chose that answer? (text)
4. Question No.4 (Lizzie):

   1. Would you cross over to the other side of the road if you saw me coming? (video)
   2. Can you tell us why you chose that answer? (text)
5. Question No.5 (Mark S with voice-over from Pino):

   1. How would you feel if you were in our shoes? (video)
6. Question No.6 (Donald):

   1. Do you have any questions for me? (video)

Survey Testing

The second usability test evaluated both the survey and the platform. The focus was to seek participants’ qualitative responses regarding the usability and accessibility of the platform, and the survey’s content. Seven participants took part in the test (Table 1). The same range of devices were available, and participants chose the device they would use typically. Mobile devices e.g. iPad and iPhone were primarily chosen; mobile devices are shown to enable multitasking with little impact on the quality of answers in comparison to surveys completed on a PC [5]. Mobile devices also aided accessibility: two ND participants expressed a preference for the iPad because of its inbuilt accessibility features and larger screen when compared to an iPhone.

Table 1. Range of devices used by participants.

Profile	Age Range	Employed (Yes/No)	Education Level	Device
ND/ Male	45-54	No	Literacy & Computing Certificates	iPad
ND/ Male	35-44	No	NVQ	iPad
ND/ Male	25-34	Yes	Bachelor	iPhone
ND/ Female	Preferred not to say	Yes	Masters	iPhone
NT/ Male	25-34	Yes	Masters	iPad
NT/ Female	18-24	Yes	Bachelor	iPhone
NT/ Female	25-34	Yes	Masters	iPhone

All participants appreciated the video / audio questions and the interactivity and accessibility they brought. However, ND participants requested all text (e.g. consent and initial information/ instructions) also be available in an alternative format e.g. audio and/or video. They also commented that without additional accessible formats the platform made less sense and was more difficult to use. Participants also cited that the video questions provoked a thoughtful response. NT participants asked about the possibility of saving their answers and returning to the survey at a later point to complete. These responses fed directly to technical development.

Question No.2 and Question No.3 had the discriminatory power of differentiating those identifying themselves as NT from those identifying themselves as ND. To Question No.2a, all NT participants responded “Yes” with two responding to Question No.2b “I have no reason not to”. Two ND participants responded “I don’t know” with one further commenting “I would not trust an entire group of people, I would not trust a group of neurotypicals nor people with learning disabilities, it’s individual”. Most participants were confused with what the pronoun “us” refers to in the question i.e. people with LD or without (Issue No.1). To Question No.3a, all NT participants responded “No” while all ND participants responded “I don’t know”. Responses to Question No.3b revealed that NT participants have “no reason to be afraid” while all ND participants indicated that it “depends on a person”. Overall, the commonalities found across the responses to Question No.2b and Question No.3b as open-ended questions are significant. The results of the usability test informed our preliminary hypothesis that the lived experience of a LD makes one see a person with a LD as an individual, “as a human being” (Co-researcher No.8).

NT participants did not feel comfortable answering Question No.1 with several participants pointing out that they were uncomfortable judging Lizzie in the video (Issue No.2). Two participants responded that “Lizzie looks sad” and one ND indicated that in Question No.1 “it still feels like she is under the microscope” (Issue No.3). One participant asked whether Lizzie derived her own questions (Issue No.4). Most participants expressed their preference for questions with a story and context i.e. Question No.4 and Question No.5 (Issue No.5). NT participants appreciated Question No.6 but were not sure whether to direct the question to Donald or a wider group of people (Issue No.6). Overall, we found that all participants provided valuable commentaries into how they feel while answering the questions, especially the ones that included the element of judgement (Issue No.7) (See Supplement No.2 for detailed outcomes).

Evaluating the Results of the Second Usability Test

Co-researchers were presented with Issues No.1 – No.6 that were identified as part of the usability test. The meeting was quite emotional with several co-researchers having doubts about their questions and the reasons for asking them. In particular, Lizzie clarified that she would not like people to think that she is sad and she definitely “does not want to be under the microscope any more!” Instead, she would like the public “to have thinking experience, a different mind-set!”

To address these issues the Team of Co-researchers decided on the following improvements to the survey:

• To address Issue No.1: Question No.2 needs to be asked by several co-researchers in the video. Also, the order between Question No.2 and Question No.3 needs to be reversed.

• To address Issue No.2 and Issue No.5: Lizzie needs to provide some context to her first question What do you see when you see me? so that respondents do not feel like they are being asked to judge her.

• To address Issue No.3 and Lizzie’s own concerns that the videos do not fully reflect her personality: Lizzie needs to be more natural in the video (i.e. “bright and confident”), although still authentic to her story.

• To address Issue No.4: All co-researchers need to introduce themselves before of each question so that the questions are owned by co-researchers and respondents have no doubt about question authorship.

• To address Issue No. 6: Donald should ask whether respondents have any questions for him or for the rest of co-researchers.

• To address Issue No. 7: Certain questions require a sub-question referring to feelings of respondents i.e. And how do you feel about me asking you that question?

Filming of the Second Version of the Survey

After the meeting, the improved versions of the videos were processed by the professional filmmaker and discussed with co-researchers who decided to ask about how you would feel and what would you do “if you were in our shoes?” (Question No.5). The second version of the survey was created with the following script³:

Second Version of the Survey⁴:

1. Question No.1 (Lizzie):

   1. Hi, my name is Lizzie. I’ve got a question for you… People stare at me all the time. What do you see when you see me? (video)
   2. How do you feel when I ask you that question? (video)
2. Question No.2 (Pino):

   1. My name is Pino, and I would like to ask you a question. Are you frightened of people with learning disabilities? I want you to answer that question. And I want you to tell us what you really think (video)
   2. Can you tell me why you chose that answer? (text)
   3. How does it feel when I ask you that question? (video)
3. Question No.3 (All co-researchers):

   1. Alright then. I’ve got another question for you… Do you trust us? (video)
   2. Can you tell me why you chose that answer? (text)
4. Question No.4 (Lizzie):

   1. Hi, it’s me Lizzie again. I would like to know, what would you do in this situation? … Would you cross over to the other side of the road if you saw me coming? (video)
   2. Can you tell us why you chose that answer? (text)
   3. How do you feel about that question? I know it could be awkward. Do you know what? Just be honest. That’s all I’m asking - to be honest. (video)
5. Question No.5 (Mark S only):

   1. How would you feel, and what would you do, if you were in our shoes? (video)
6. Question No.6 (Donald):

   1. My name is Donald and I’ve got a question for you: Do you have any questions for me or the other people? (video)

Final Survey Revisions

Following the development of the second version of the survey, there were a number of revisions performed to improve representation of the co-researchers and the project overall. We also included and tested demographic variables for future analysis and data clustering. In response to co-researchers’ comments regarding the inclusion of video consent, introductory videos were created in which Robyn explains who the team are, what data is collected and why. For quality assurance checks and to improve our message, we shared the survey with a broader audience, including Heart n Soul collaborators and staff at the Wellcome Trust in London.

Although the analysis of the survey results is out of scope, we have received initial public response. The final survey was launched in December 2019 and can be found at www.heartnsoulasks.com. It has currently received 800 responses at the time of writing and remains open. We have received very positive feedback within the survey, on social media, and as part of public engagement events including the Festival of Minds and Bodies at Wellcome Collection on 30^th November 2019. Many respondents have remarked that the survey is unusual “this [survey] is the most beautiful I’ve ever seen.”, provides space for reflection “I learned a lot about myself by taking the Heart n Soul survey”, and is very accessible “I’ve got a collection of bad surveys I use in teaching but struggle to find good ones for templates […] This one is really hard to fault and if you’re doing an online survey you should definitely use it as a template and give credit. I showed it in class today, very well received”.

Reflections from Co-researchers

Once the final survey was developed, Pino, David Mc and Lizzie offered their extended reflections on the research process. Following publication of the survey, Pino published a first-person article in the Guardian newspaper [40] where he describes his involvement as the lead co-researcher (“My colleagues and co-researchers […] are relying on me to make sure everything goes well”).

Both Pino and Lizzie pointed out the challenges associated with the first stage of the survey design process. Pino mentioned that “coming up with the questions was difficult”. Lizzie recalled similar experiences: “I was really unsure about the questions we were asking. All of us were a little bit unsure of what to ask and what we expected”. Once the survey was completed, the three co-researchers felt proud and were positive about the outcomes.

When discussing the project goals and their involvement as co-researchers, all three reflected on the initial idea of ‘the microscope’. For David Mc, being a co-researcher was like “having the opportunity to be on the other side of the research, because research is usually done on us and we wanna know what people think and learn about other people, the more we find out about other people the more we learn about ourselves”. Lizzie was intrigued by how people might respond to the survey and what impact it might have on people with LDs: “We’re flipping the microscope on them, its amazing, I would still like to know what it felt for them for the microscope to be on them. […] I would like to know what it was like so they would understand how it feels for us. Get an answer like, oh god I didn’t know that. Now that the microscope is on you, how do you feel? […] has this experience made you change your mind about people with disabilities, has it made you open your eyes and see? After all these years, if someone came back and said my eyes have been opened by people with disabilities, then we’ve done it, job done and dusted.”

Finally, all three emphasized that they want the respondents to be honest in the survey. David Mc summarized this as “Putting people on the spot and finding out who’s ganna say what we wanna hear or saying the stuff that we like, don’t wanna hear. I don’t mind hearing all the bad stuff what the questionnaire brings up.”

Discussion

Co-researchers with LDs developed an inclusive online survey to challenge public attitudes towards LDs using a co-designed accessible survey platform. Both the survey platform and the online survey have been well received by the co-researchers throughout the research process. We have also obtained very positive feedback from the survey respondents who appreciated the accessibility, uniqueness, and the self-reflexive elements of our “beautiful survey”. While these generally positive comments and impressions about the survey meet the expectations of our co-researchers, we are uncertain about their impact on the participants’ honesty in responding to the questions, especially when some are predicted as being “negative”. We aim to explore this in detail with co-researchers (see future directions).

When designing the survey, co-researchers were suggesting questions that were important to them and reflective of their experiences. Therefore, for co-researchers, the survey questions are much more than research. Since the questions are personal, co-researchers hope that the responses will reveal more about their own lives and learn more about themselves in line with the idea of “having a conversation with the public”. At times, this led to co-researchers having self-doubts about the purpose of the project and its future impact. Therefore, an important aspect of the survey design process that was challenging was in maintaining and reinforcing the purpose of each Tea Party. In our study each meeting began with a reminder of the project’s significance and the important role that Team of Co-researchers had. This was not simply to remind co-researchers of the purpose of the workshop, but more importantly, to encourage them to feel positively about their ability to impact the process of research.

There were challenges associated with the survey platform development. Although the aim was to inform a design that was accessible and usable by all, co-designers found it difficult to empathize with the NT persona. This may be indicative of the double empathy problem which Milton, Heasman & Sheppard (2018) [70] describe as a “disjuncture in reciprocity between two differently disposed social actors who hold different norms and expectations of each other, such as is common in autistic to non-autistic social interactions” (p.7). As such there were concerns that there was the potential that the double empathy problem could be reflected in the design; recent research indicated designing for accessibility with regard to language impacted the use of a website for NT users [104]. However, the use of multiple mediums in surveys for NT users is not new. Audio and video questions may also improve question comprehension, reduce social desirability bias [60], improve respondent engagement (particularly when the topic of the survey is of a sensitive nature) and reduce attrition rates [8] for NT respondents. Therefore, the design suggestions put forward by the co-designers were applicable for a universal design. Additionally, the core philosophy of D4D framework although devised as a co-design approach with children was helpful in guiding design and usability testing with ND adults. However, further research is required to form the development of a ND design protocol for adults [29].

Consent and Safeguarding

The issues of informed consent and safeguarding procedures were challenging. We have an ethical responsibility to protect those vulnerable team members who are new to research. We moved from on-paper consent, and research training, towards their full involvement. However, we remained careful at all times: one co-researcher decided to cease engagement in the process, and stepped down from the project for some time. As a matter of process, we stayed in touch with all co-researcher’s guardians in order to evaluate if the process was potentially having negative impacts on their health and wellbeing. We did not assume that all co-researchers wanted to be named as co-authors or be fully acknowledged. Most co-researchers had very valid reservations towards recognizing and attributing their contributions towards the study due to e.g. identification potentially leading to harmful judgement or eligibility for benefits which have not been identified previously [122].

Co-researchers preferred video over verbal or written media both for receiving information about usage of personal data and when they provide informed consent. This is a noteworthy finding especially as ensuring informed consent with vulnerable populations can be challenging. Using video as a medium for imparting consent information has been explored within a medical context [103, 125]. However, such studies utilize video for imparting information in a unidirectional communication. However, the survey platform enables participants to also respond using a medium of their choice facilitating a bi-directional consent process aligned to the mediated consent process advocated by Sterling and Rangaswamy (2010) [120]. Future work may also make it a more informed and negotiated process.

Future directions

Our co-research process, despite its many challenges, has generated tangible outcomes - specifically the survey platform and the online survey with data. These outcomes can be measured and assessed, and we aim to do this more fully in the future. Based on public responses to the survey (outcomes), we will be able to evaluate the quality and relevance of our inclusive research process through evaluating responses and other outcomes as part of a robust research process.

For example, initial evaluation of the survey data during the survey testing stage indicates that the questions asked by co-researchers have potential to generate valuable data that appears robust. In particular, the survey has the potential to discriminate ND versus NT individuals, gives an opportunity to open up in a safe but at times thought-provoking environment (e.g. by asking the survey respondents what they think about the questions/how they feel while responding) and also could add to wider conversation on accessible Big Data approaches [58]. Based on the first set of feedback received from the survey respondents, our approach indicates the survey is considered to be high quality. We aim to convince researchers that this type of exploratory research might be worth pursuing for purely scientific reasons as well as outreach to a greater public about the issues faced by individuals with LDs, and therefore greater understanding. In this way, our research could make impact that people with LD will value [81].

The hybrid approach we have taken has led to a functional and accessible survey and platform which in turn can be used to collect survey responses to research questions posed by people with LDs. As the next step, we intend to work with co-researchers to explore, evaluate and analyse research data from the survey through interactive machine learning and visualisation approaches.

Conclusion

Co-researchers with LDs created an online survey to help challenge and shift public understanding of LDs. In this paper, we describe how the survey platform was developed, how co-researchers created an online survey, and we also specify and reflect upon the inclusive research elements and practices in achieving the study goals. Response to the survey has been positive from both co-researchers and survey respondents. We feel this survey and the approaches we have taken are leading the way in terms of inclusive research practices, whilst potentially resulting in the generation of meaningful data. Finally, we reflect on the views and perspectives of co-researchers following the completion of the study, and consider the difficulties and advantages we encountered deploying such approaches. These may be applied in any research setting wherein non-academics with a range of abilities, identities, and backgrounds are placed at the heart of a CHI-relevant research process, but in this case, they have led to outcomes that we hope will be of interest and benefit to others attempting to reach similar goals working with LD communities.

CSS Concepts.

• Human-centered computing~Human computer interaction (HCI).