Abstract
Introduction
Aphasia is an impairment of language that occurs in 30%–40% of stroke survivors. This often chronic condition results in poor outcomes for the individual with aphasia and their family. Long-term aphasia management is limited, with few people receiving sufficient services by 6–12 months postonset. We present a protocol for the development of coproduced aphasia service elements. We will use experience-based codesign (EBCD), an approach that enables service users and providers to collaboratively develop services and care pathways. Drawing on the experiences of people with aphasia, their families and clinicians we will establish priorities for the development of new services and later work together to codesign them.
Methods and analysis
This research will be coproduced with people with aphasia (n=30–60), their families (n=30–60) and speech pathologists (n=30–60) in Queensland, Australia, using EBCD. A consumer advisory committee will provide oversight and advice throughout the research. In phase 1, we will use semistructured interviews and the nominal group technique to explore experiences and unmet needs in aphasia rehabilitation. Data will be analysed using thematic analysis and the resulting themes will be prioritised in multistakeholder focus groups. Outcomes of phase 1 will inform future research (phase 2) to codesign services. Financial costs and participant experiences of EBCD will be measured.
Ethics and dissemination
Human Research Ethics Committee approval for phase 1 has been obtained (HREC/2020/QRBW/61368). Results will be reported in peer-reviewed journal articles, presented at relevant conferences and, following EBCD suggested best practice, fed back to participants and community members at a celebratory event at completion of the project. The inclusion of service users in all stages of research will facilitate an integrated approach to knowledge translation. A summary of research findings will be made available to participating sites.
Keywords: quality in health care, stroke, qualitative research, rehabilitation medicine, stroke medicine
Strengths and limitations of this study.
This research uses experience-based codesign (EBCD) methods with people with aphasia, significant others and clinicians, to ensure that their priorities are central in new service design.
Our research methods are designed to support the inclusion of Aboriginal and/or Torres Strait Islander Peoples of Australia and culturally and linguistically diverse populations, who are often excluded from research.
This study contributes to building knowledge about the use of EBCD to coproduce tailored services in new health service design.
Research findings may be used to guide the development of aphasia services locally, nationally and internationally.
Evaluating the implementation of codesigned services is not within the scope of the current project, however, is a future research direction.
Introduction
Aphasia is an impairment of language that impacts an individual’s ability to communicate.1 Global stroke prevalence is estimated to be more than 80 million, with an incidence of 13.7 million.2 Based on a 33% aphasia incidence rate poststroke,3 aphasia affects approximately 4.6 million stroke survivors worldwide. For most people, aphasia persists as a chronic condition4 and results in poor outcomes for both the person with aphasia and their family, long after discharge from healthcare services.5 Out of 60 major diseases and 15 health conditions, aphasia was identified as having the worst effect on quality of life, even when compared with cancer and dementia.6 This negative impact is compounded by mood disturbances. Kauhanen et al7 found that 70% of stroke survivors with aphasia developed depression between 3 and 12 months poststroke, approximately double the rate experienced by stroke survivors without aphasia. People with aphasia are at increased risk of loneliness and social isolation,8 have longer stays in hospitals, access more rehabilitation services, experience greater mortality rates and are less likely to return to work following a stroke than those without aphasia.9 10 Despite these poor outcomes, no research has comprehensively explored the unmet service needs of this population or identified their priorities for future service development.
Unmet needs in people with aphasia
The unmet needs of people with aphasia have not been comprehensively explored. Current understanding is guided by qualitative exploration of patient experiences in discrete areas of practice and recovery and by research focused on the needs of the general stroke population. Qualitative research provides a foundation for understanding unmet need in the context of specific aspects of rehabilitation (eg, treatment goals11 and management of mood disturbances12) and recovery (eg, experiences of returning home13). Wray and Clarke14 conducted a systematic review of qualitative studies exploring the longer-term needs of stroke survivors with communication impairments. The 32 included studies explored: perceived changes in social roles; experiences within the first 3 months following stroke; experiences of communication disability; the consequences of aphasia; factors inhibiting participation; and experiences of returning home.14 Unmet need was identified to fall into four main areas: (1) managing communication outside of the home, (2) creating a meaningful role, (3) creating or maintaining a support network and (4) taking control and actively moving forward with life.14 While this research provides a basis for understanding the unmet needs of people with aphasia in specific areas of practice and recovery, the relative importance of these needs have not been established. Research within the general stroke population has sought to identify the long-term unmet needs of stroke survivors in multiple countries including Australia,15 Ireland,16 Scotland,17 Sweden,18 the USA19 and the UK.20 Zawawi et al21 conducted a systematic narrative review of 105 studies reporting unmet needs in stroke-suvivors from America, Asia, Australia, Canada, Europe, New Zealand and the UK. Unmet needs were classified into categories of: (1) physical and other stroke-related problems, (2) social participation, (3) information and (4) rehabilitation and care. Communication (including speaking, reading, writing and hearing) was identified as an unmet physical function need in 13 of the included studies. In the category of social participation, unmet community reintegration needs also related to unfulfilled communication needs (eg, difficulty participating in social activities within their community and inability to fulfil communication obligations) in 14 studies.21 Although communication was identified as an unmet need for stroke survivors in these studies, the specific aspects of communication impacted were not well described. For example, Kersten et al20 surveyed 315 younger stroke survivors between 18 and 65 years in the UK. Speech therapy was identified as one of three most pressing needs for young stroke survivors unable to return to work.20 However, the specific nature of the ‘speech therapy’ required was not detailed, limiting the extent to which this information can be used to drive service development. In another study, Andrew et al15 conducted surveys with 765 stroke survivors across Australia, 12 or more months poststroke. ‘Speaking difficulties’ were a frequently reported unmet physical need,15 however, the basis of these difficulties (eg, language vs motor speech impairments) was not described, making it difficult to delineate what support was required. While it is clear that long-term communication needs exist for stroke survivors, there is a need for a comprehensive exploration of unmet need that also provides insight into the relative importance of identified needs, in sufficient detail to drive the development of services or interventions to remedy these issues.
Complexity of need in people with aphasia
Traditional healthcare models focus on aphasia treatment in the first 6 months of recovery, with limited access to services following discharge home.4 22 For the person with aphasia, the impact of social, vocational and family involvement on quality of life, often becomes apparent following their return home and subsequent discharge from services.23 This poses a challenge within current models of care, especially when community follow-up and delivery of community-based services is limited.24 International research exploring aphasia management has found that current models of aphasia care are unable to provide sufficient dose, intensity or duration of aphasia services to achieve optimal outcomes.25 Katz et al25 conducted an international survey of 175 clinicians from Australia, Canada, USA and the UK, and found that provision of community-based services were limited across all participating nations. Clinicians working in both metropolitan and regional facilities have identified competing approaches,4 26 resource constraints4 24 26 and reduced funding4 26 as contributing factors, with deprioritisation of aphasia management in the acute setting further limiting care.26 Research has also shown that community-based aphasia management is limited and therapy provision is less likely to occur by 6–12 months postonset.4 24 For example, one study evaluated the usual care data of 278 participants over 3 months, from 21 National Health Service sites in the UK.24 They found that community-based therapy was less likely to occur for people with aphasia beyond 12 months poststroke. They suggested resource constraints as a possible reason, driving clinicians to provide an average of seven sessions (over 3 months) to more patients—below the level of intervention shown to be effective.24 In another nationwide study, Rose et al4 surveyed 188 speech pathologists across Australia regarding aphasia treatment practices. Clinicians reported that community follow-up for aphasia management was limited and all services had ceased by 6–12 months.4 Given the significance of long-term needs for people with aphasia (majority of those diagnosed with aphasia show symptoms of aphasia 1 year postonset,27 28 it is concerning that the availability of community-based services is limited. With the prevalence of stroke survivors expected to rise,4 29–31 it is unknown how already overburdened systems will be able to meet this increasing demand for aphasia services.
Codesigning new aphasia services
Understanding the healthcare experiences of people with aphasia is essential to developing new services that meet their needs. Research exploring experiences of care has been recognised as a key component to improving quality in healthcare systems and is positively associated with improved healthcare outcomes.32 Doyle et al32 conducted a systematic review of 55 studies to explore the relationship between patient experience and health outcomes and healthcare quality. They found consistent positive associations between patient experiences, patient safety and clinical effectiveness for a range of healthcare settings and diseases, suggesting that positive experiences for patients are clinically relevant. The inclusion of patient experiences in service design processes, highlights the specific strengths and weaknesses in the delivery of services needed to improve healthcare quality. Coproduction and codesign methodologies encourage the interaction of multiple stakeholders to ensure the most effective outcomes for service users are established.33 In healthcare, this involves service users such as patients or their family members as stakeholders in the design, management or evaluation of the services they interact with.33 Central to current work in coproduction and codesign is the importance of exploration of power relations and decision-making processes and frameworks. Experience-based codesign (EBCD), a specific example of a codesign method, provides an approach to explore gaps in services and priorities for service development from the perspectives of patients and families, to ensure tailoring of services during the codesign processes that follow. The EBCD approach can be broadly divided into six stages according to early descriptions that are provided by The Point of Care Foundation in the UK. It is important to recognise that these stages are combined to form an entire codesign process, so each stage should be viewed as interacting with future stages:
Project set up (ethics, project development, establishing stakeholder/advisory groups).
Gathering staff experiences.
Gathering patient and carer experiences (typically 12–15 filmed narrative based interviews).
Initial codesign event that brings together staff, patients and carers to view a filmed representation of patient touch points (key themes related to negative experiences identified in the data) and collaboratively identify priorities for codesign workshops.
Codesign workshops (usually 4–6, where staff and service users collaboratively work together in an iterative process to redesign a service);.
Celebration event.34–36
In recent years, EBCD has been used in healthcare to redesign services for palliative care service delivery in emergency rooms,37 pretreatment pathways in head and neck cancer,38 mental health service experiences,39 40 emergency department experiences,41 medicine management during care transitions,42 integrated cancer care centre services for patients with breast and lung cancer,43 and outpatient cancer care.44 EBCD combines narrative theory, participatory action research and design thinking, to effectively ensure those with the lived experience are embedded within and actively contribute to the design process.34 45 We present a protocol for phase 1 of a project to use EBCD to inform the future coproduction of aphasia service elements with people with aphasia, significant others and speech pathologists. Conceptual designs of aphasia service elements addressing key priorities identified in phase 1 will be developed for use in future codesign processes in phase 2.
Study aims
The overarching aim of this research is to coproduce aphasia service elements. In this protocol, we describe the first stage of this process, gaining consensus on priorities for aphasia service codesign. Specific aims are:
To explore positive and negative experiences of aphasia care, and identify priorities of aphasia service users (including Aboriginal and/or Torres Strait Islander Peoples and culturally and linguistically diverse populations), their significant others and clinicians.
To gain consensus on priorities for new aphasia services from the perspective of people with aphasia, their significant others and clinicians.
To evaluate the experiences of participants engaging in the EBCD process.
Methods and analysis
This qualitative study using EBCD will combine participatory research engagement methods (eg, consensus meetings) with patient and public involvement (eg, consumer advisory committee input) to identify priorities for aphasia service codesign.
Patient and public involvement
A consumer advisory committee has been established to provide expert feedback on key elements of the project such as recruitment strategies, interview or focus group considerations (including codevelopment of the experience of participation feedback questionnaire), codesign training and dissemination of outcomes. This advisory committee is made up of individuals with lived experiences of aphasia services, who represent each stakeholder group and consists of a ratio of at least two people with aphasia and their significant others, to one clinician and the cultural capability officer from a metropolitan hospital in Brisbane, Australia. Research questions and study design (including data analysis processes and assessment measures) have been developed by the research team (LA, VJP, DAC and SJW). Public involvement is intended to guide: conduct of study procedures and recruitment (eg, consumer advisory committee feedback on what to emphasise to facilitate understanding and engagement by stakeholders, expert advice on facilitating engagement for Aboriginal and/or Torres Strait Islander Peoples, codevelopment of a recruitment video) and interpretation and dissemination of results (eg, as coauthors on papers and development of plain English summaries of research papers). To date, advice from the consumer advisory committee has resulted in: (1) codevelopment of a recruitment video to make research information communicatively accessible, (2) cocreation of a plain English summary of this protocol and (3) development of a community engagement plan to build trust with local communities and facilitate engagement with community members. The consumer advisory committee will receive updates throughout the project and opportunities to provide feedback on processes prior to each stage. The research team will establish procedures for codesign training and workshops with input from the consumer advisory committee to facilitate understanding and engagement by all stakeholders. Patient and public involvement will be reported in line with the Guidance for Reporting Involvement of Patients and the Public (GRIPP-2) long-form checklist.46
Study design
This EBCD34 42 47 approach will be conducted in two phases (see figure 1 for research overview). Phase 1 (detailed in the current protocol) aims to establish the priorities for service development. Semistructured interviews and the nominal group technique (NGT)48 49 are used in phase 1 to explore experiences and unmet needs in aphasia rehabilitation. Data will be analysed using thematic analysis and the resulting themes will be prioritised in multistakeholder focus groups. Phase 2 of this research (not detailed in the current protocol) is dependent on the completion of phase 1 and aims to codesign new aphasia rehabilitation services. In phase 2, aphasia centre services will be codesigned in a series of multistakeholder workshops. EBCD methods have been used for healthcare service design41 including with vulnerable populations.39 The flexibility of the EBCD approach allows for modifications that support the involvement of populations with specific needs,37 making it a suitable choice to support people with aphasia driving the service design elements (see figure 2 for modifications to EBCD stages). A steering committee (comprising authors SJW, DAC and VJP) will monitor achievement of milestones and timely completion of the project.
Figure 1.
Overview of the experience-based codesign (EBCD) procedures used for coproducing tailored aphasia service elements in this research project. NGT, nominal group technique. PWA, person with aphasia.
Figure 2.
Overview of the modifications made to our experience-based codesign (EBCD) project and anticipated timeline of activities.
Participants
-
People with aphasia
Inclusion criteria: (1) confirmed presence of poststroke aphasia (severity at time of participation as determined by Aphasia Severity Rating Scale (ASRS)50; (2) six or more weeks (subacute) poststroke (or following discharge from inpatient aphasia rehabilitation services, people with aphasia identify therapy needs following discharge home from hospital and with long-term management23); (3) aged 18 years or older.
Exclusion criteria: (1) Presence of other neurodegenerative or neuropsychological disorders such as dementia or Parkinson’s; (2) aphasia due to non-stroke aetiology; (3) Not proficient in English (group involvement only). Groups will be conducted in English. A significant component will require participating in group discussions; therefore a good knowledge of English language is required.
-
Significant others
Inclusion criteria: (1) family member, close friend, main contact or carer of a person with aphasia; (2) able to participate in group discussions; (3) aged 18 years or older.
Exclusion criteria: Not proficient in English (group involvement only).
-
Clinicians
Inclusion criteria: Speech pathologists currently working with people with aphasia, or their communication partners.
Exclusion criteria: No experience working with people with aphasia.
Recruitment
We aim to recruit 30–60 participants with aphasia, 30–60 significant others and 30–60 clinicians from 21 Hospital and Health Service (HHS) sites in Queensland, Australia. Research sites will be established across areas of varying geographical remoteness (as per the Australian Bureau of Statistics Remoteness Area Classifications51): (1) remote and very remote locations; (2) inner and outer regional locations; (3) Metro North HHS central locations; (4) Metro North HHS north locations; (5) other metropolitan sites; (6) community/other sites (eg, telehealth). At each location, participants will be recruited from participating Queensland HHS sites, professional affiliate groups, aphasia community groups, online community groups, and through opportunistic encounters. Following the completion of each stage, participants will be invited to express interest in participating in subsequent stages of research. It is very common in a codesign project for participants to take part in multiple activities and parts of the process. Separate consent will be sought to participate in experience gathering (EBCD stage 2) and collaborative prioritisation (EBCD stages 4 and 5). Recruitment processes for phase 2 (including codesign—EBCD stage 6) of this research will be developed following completion of phase 1.
Sampling
Maximum variation sampling will be used to ensure the greatest degree of diversity is achieved in the range of experiences captured (see table 1 for details). Prior EBCD projects have recommended approximately 10 participants for information gathering and subsequent codesign stages. These projects have focused on a single health service occurring at 1–2 site locations. As the current EBCD project is considering a system-wide evaluation of experiences across services and service pathways, up to 10 participants will be recruited from each participating region.
Table 1.
Maximum variation participant matrix, including criteria used per stakeholder group
Participant group | Priority | Criteria |
Participants with aphasia | 1. Aphasia severity | Mild, Moderate, Severe |
2. Time post onset | Subacute: 6 weeks to 5 months, Chronic: 6+ months |
|
3. Age | 18–54 years, 55–70 years, 70+ years |
|
4. Sex | Male, Female |
|
5. Cultural ethnicity | Aboriginal and/or Torres Strait Islander, Not Aboriginal and/or Torres Strait Islander, Speaks English as a second language |
|
Clinicians | A. Type of caseload | Acute: 0–7 days, Subacute: 1 week to 6 months, Chronic: 6+ months |
B. Type of health service | Acute, Inpatient rehab, Stroke unit, Outpatient rehab, Community |
|
C. Years of experience | <2 years, 2–10 years, >10 years |
|
Significant others | None | Significant others will be categorised based on the characteristics of their significant other with aphasia. |
Modifications to EBCD methods
Facilitating communication with people with aphasia
All research activities will be facilitated by speech pathologists who are experienced in facilitating group discussions with people with aphasia. Written information pertaining to each group session or interview (including agendas, questions, participant response forms and questionnaires) will be prepared in an communicatively accessible format (eg, use of white space, contextual image supports and reduced linguistic complexity) and distributed prior to each session.52 People with aphasia will be given an opportunity to verbally discuss research processes prior to participation in a given session. Participation within all research activities will be facilitated using supported communication techniques including use of: multimodal communication; written key words; additional processing time for comprehension and response generation; fixed-choice questions; and verification of content.53 Research findings will be made available in lay and communication accessible formats.
Inclusion of diverse populations
Specific modifications have been made to support the inclusion of Aboriginal and/or Torres Strait Islander peoples who wish to participate in this research. Indigenous Liaison Officers have been identified at participating sites to offer expert guidance on a case-by-case basis. Project materials have been developed in line with guidelines provided by The University of Queensland Poche Centre for Indigenous Health (eg, using large size font, use of white space, simplified language and use of (culturally appropriate) image supports) to facilitate comprehension and to show respect for Aboriginal and/or Torres Strait Islander peoples. To ensure Aboriginal and/or Torres Strait Islander Peoples priorities for service development are captured and included, a community engagement plan will be developed in collaboration with the Cultural and Capability Officer—Allied Health Professions (Royal Brisbane and Women’s Hospital). The cultural capability officer sits on the consumer advisory committee to provide expert guidance and project oversight. In response to advice provided during the collection of participant experiences in phase 1, an additional focus group will be offered for Aboriginal and/or Torres Strait Islander participants at each site where it is desired. Participants who would like to share their experiences in a language other than English can do so during an interview with the aid of an appropriate interpreter. Non-English transcripts will be translated and analysis of meanings confirmed.
Assessments
Participants with aphasia will complete a single assessment session prior to their initial research involvement. Assessments will include: (1) ASRS50; (2) Stroke and Aphasia Quality of Life Scale -39 (SAQOL-39)54; (3) Health-related Quality of Life Scale (EQ-5D-5L).55 The SAQOL-39 includes 5-level responses for 39 questions across 3 domains. The ASRS provides an overall rating of language ability or aphasia severity rating between 0 and 5. The ASRS and SAQOL-39 have both been validated for use with people with aphasia. The EQ-5D-5L includes five-level responses across five health domains. Assessments will be used to describe the participant sample, provide additional context during data analysis and inform understanding of unmet needs.
Financial data collection
A prior review of EBCD projects identified that costs associated with EBCD are not routinely collected to inform future research design.33 To the authors’ knowledge, this is the first project to use an EBCD approach with people with aphasia. Therefore, costs of conducting this EBCD approach will be analysed to inform future research design using EBCD with people with aphasia to codesign services. The principal investigator will maintain a running excel spreadsheet that will be calculated on completion of each phase. Details pertaining to the development of resources, costs of participating for participants (including transport costs), associated time and administration costs involved with data collection, and human resource expenses (related to data collection and facilitation) will be collected. A cost analysis comparing costs associated with data collection, administration and facilitation per stakeholder group, based on data collected in this EBCD project will be conducted at the completion of each phase.
Experiences of participation
All participants will complete a short questionnaire on their experiences of involvement following each research activity. This questionnaire will be codeveloped with input from the consumer advisory committee. Inductive content analysis56 will be used to explore themes related to participants experiences of being involved following each stage of the research. Experience of participation data (eg, evaluation of information provided prior to participation, participants capacity to share within the group setting, overall satisfaction and open comment fields) will be used to refine research processes in subsequent stages and to inform the development of an EBCD toolkit for people with communication disability.
Data collection and analysis in phase 1
Information Gathering (EBCD stage 2)
Semistructured interviews (data gathering and understanding)
Eighteen semistructured interviews, (six people with aphasia, six significant others and six clinicians) will be conducted to develop a deeper understanding of the thoughts, feelings and actions that occur during the journey of aphasia service interactions (eg, development of experience maps and touch-points). An interview guide will be developed in consultation with the advisory committee to ensure relevance and methodological consistency. Semistructured interviews use open ended questions with prompts to support communication, making it an appropriate choice for interviewing people with aphasia. Semistructured interviews have been used in EBCD projects to effectively explore patient experiences of care,37 and the experiences of people with aphasia.57 Interview data will provide insights into key experiences per stakeholder group and contextual support for participants generating local ideas and priorities for aphasia service design during NGT focus groups.
NGT (data gathering, understanding and individual site prioritisation)
An adaptation of Harvey and Holmes49 NGT will be used to gather experiences, and ideas for service design from a wider audience at each participating location. NGT48 49 is a structured form of focus group used to identify personal experiences, and to establish group consensus for prioritising actions.49 The structured approach ensures participation in group discussion by all members and allows for the provision of supportive communication strategies for people with communication impairments. Wallace et al58 have effectively used NGT in prior research with people with aphasia and their significant others.
A single NGT will be hosted at each site, per stakeholder group, with an additional NGT group offered at each site for Aboriginal and/or Torres Strait Islander participants where it is desired. Individual prioritisation by geographical area will provide guiding principles for broader consensus prioritisation to follow. Three questions will be posed to each group: (1) describe your best experience of aphasia services; (2) describe an experience that did not go so well and (3) what would the ideal service look like? Groups will be video and audiorecorded with consent. Questions will be piloted to assess clarity of wording and ability to generate desired information.
Field notes (understanding)
Field notes will be recorded at each site and will be used to support interpretation of themes and to provide contextual insights that may shape participant responses. Field notes will be used to keep track of setting details and key features of the interactions for future consideration.
Data Analysis (EBCD stages 2 and 3)
Interview and NGT data will be combined to develop a comprehensive experience map per stakeholder group. Thematic analysis of positive and negative experiences will follow the stepped process described in Braun and Clarke.59 Thematic analysis of negative experiences will be used to explore unmet needs in people with aphasia and their significant others. Themed rankings of service priorities will be used to develop problem statements for review during prioritisation focus groups in EBCD stages 4 and 5. Representations of significant negative themes identified across the data will be developed into a 20 minute touch-point film. Touch-point films use digital storytelling to evoke an emotional connection across stakeholder groups.60 61 Video footage of participants with aphasia and significant others will be selected based on themes identified, as shown in EBCD stages 2 and 3 of figure 2. The touch-point film will be viewed by participants during prioritisation focus groups (EBCD stages 4 and 5). No video data will be viewed by participants without prior consent from all those who appear in the film. When consenting to the use of video footage, participants will have the opportunity to review all clips that contain their voice or image to be used in the film presentation. Inductive content analysis56 will be used to combine and compare ranked consensus prioritisation for service development from combined NGT and interview data. Themes will be compared across groups for differences and commonalities according to geographic remoteness and demographic indicators.
Collaborative understanding and prioritisation (EBCD stages 4 and 5)
Focus groups will be used to collaboratively identify suitable priorities for focused codesign workshops (EBCD stage 6) to follow. Themed service priorities identified in earlier stages (EBCD stages 2 and 3) will be reviewed within a two part prioritisation process (EBCD stages 4 and 5). Focus groups use the interaction between people to generate a collective exploration and understanding of peoples experiences.62 Focus groups typically involve 8–10 people, for a session of 1–2 hours.63 Due to multiple stakeholder involvement, 3–10 participants will be invited to attend each prioritisation focus group. Focus groups have been recommended for comprehensively exploring patient perspectives on health and functioning,64 and have been effectively used in prior EBCD projects to identify priority areas for codesign stages.40 All focus groups will be videorecorded and audiorecorded with consent. Participants will be invited to comment on their experiences of being involved following each group.
Consensus prioritisation part 1 (EBCD stage 4)
Two focus groups will be conducted: (1) clinician only focus group; (2) combined group with people with aphasia and their significant others. These consensus meetings will focus on collectively understanding experiences and ranking priorities identified across all geographical locations within stakeholder groups. Participants will view the touch-point film (significant themes of negative experiences), stakeholder experience maps (see exemplar in online supplemental material 1) and deidentified personas (see example template in online supplemental material 2) during prioritisation processes. An overview of prioritisation procedures is provided in figure 3. All participants unable to attend, will be given the opportunity to vote on their priorities for service development via a 30 min online survey.
Figure 3.
Focus group procedures for prioritisation part 1 (EBCD stage 4), with details of the sequential order and key elements explained.
bmjopen-2020-047398supp001.pdf (296.6KB, pdf)
bmjopen-2020-047398supp002.pdf (403.2KB, pdf)
Consensus prioritisation part 2 (EBCD stage 5)
A final combined focus group will produce a top ten list of priorities for service design that represents all stakeholder group perspectives, and identifies key areas for codesign workshops. Focus group consensus prioritisation part 2 procedures will include a reshowing of the digital presentation of key experiences, personas and experience maps. A matrix, adapted from reference 65, will be used to facilitate prioritisation (see figure 4). Participants will rank their top three priorities for service design until a top ten list of priorities is identified.
Figure 4.
Example of matrix, adapted from reference 65 to facilitate prioritisation based on: (1) degree of impact (low vs high); (2) degree of influence (low vs high); (3) suitability within scope of codesign project (low vs high).
Data analysis (EBCD stages 4 and 5)
Inductive content analysis56 will be used to combine and compare ranked consensus prioritisation for service development from combined prioritisation focus groups and survey data. Rankings will be compared across groups for differences and commonalities according to geographical remoteness and demographic indicators. Participants experiences of involvement in prioritisation processes will also be analysed.
Anticipated outcomes
A comprehensive understanding of the unmet needs of people with aphasia and their significant others.
A consensus-based list of priorities for the future design of aphasia service elements.
Mapped aphasia services and service models from participating sites in Queensland.
An evaluation of participant experiences of involvement during experience gathering and collaborative prioritisation processes.
Future directions, phase 2 data collection and analysis
Outcomes of phase 1 will be used to inform specific methods and processes used in phase 2. A brief overview of phase 2, the next steps for this research (EBCD stages 6 and 7) is provided here. These stages will be developed following completion of phase 1. Phase 2 of this research intends to codesign elements of aphasia services that target identified priorities. Some elements will be generic and will require customisation and some will reflect priorities unique to specific contexts such as rural settings.
Codesign workshops (EBCD stage 6)
Codesign workshops will be used to bring together representatives from all stakeholder groups to work in partnership to design services. Participants will be assigned to codesign workshops using purposive sampling to work on a priority for service development based on prior consensus prioritisation (EBCD stage 6). Published literature will be synthesised into ‘rapid evidence reports’ that align with each priority identified and presented to participants prior to codesign workshops. Participants will be able to use this information during codesign workshops. In prior research, Kearns et al66 effectively used codesign workshops with people with aphasia to develop a new user informed outcome measure. Six participants with aphasia attended six, 90 min workshops, with feedback suggesting 4–5 participants as optimal. However, codesign workshops in this project will include participation by multiple stakeholders, presenting challenges for accurate estimations of participant numbers. Accordingly, codesign workshop agendas and supporting resources will be developed following phase 1 and pilot tested prior to workshops. Specific training for codesign workshops will be developed based on participant feedback from research activities in phase 1 and from pilot testing. Training will cover specifics of: methods used during workshops; aims, objectives and limitations of workshop processes associated with expectation management; roles and decision-making processes and how power will be managed between stakeholder groups. Details for codesign workshops will be informed by phase 1 prioritisation, participant feedback on participation and in response to pilot testing. A research protocol specific to phase 2 of this research will be developed following completion of phase 1.
Celebration event (EBCD stage 7)
All participants involved in the project will be invited to a feedback and celebration event where outcomes and future directions as a result of the codesign project will be reported. Feedback on codesigned services developed in response to consensus prioritisation will be sought from all attendees.
Data analysis (EBCD stages 6 and 7)
Participant experiences of research participation will be collected across all stages of involvement (EBCD stages 2, 4, 5, 6 and 7), and used to inform changes to EBCD procedures and in toolkit development. Combined feedback data collected during celebration and feedback event on responses to coproduced services (EBCD stage 7) will be analysed.59 Themes will be compared across groups for commonalities according to geographic remoteness and demographic indicators.
Ethics and dissemination
Ethics
Ethics approval for phase 1 of this project has been provided by the Royal Brisbane and Women’s Human Research Ethics Committee (HREC/2020/QRBW/61368) and The University of Queensland (2020000965). Ethics approval was granted in April 2020, initial site-specific research contracts were approved in November 2020, and data collection began in December 2020.
Managing expectations of participants has been identified in other codesign studies as a way to reduce possible distress associated with involvement.43 As such, aims and agendas for each group session will be clearly defined and distributed to participants prior to all events. Following the communication principles established by the consumer advisory committee.
To reduce the burden of participation on participants with aphasia, prepaid hospital parking vouchers and reimbursement for the cost of fuel per kilometre of travel (for remote participants who need to travel long distances) will be offered. Participants will be given the option to participate using videoconferencing where it is considered unsafe to meet face to face or due to COVID-19 pandemic measures. Participants who are invited, and who live remote from Brisbane, will be reimbursed for the cost of their travel and accommodation. All people with aphasia and significant other participants will be offered a voucher as a reimbursement for their time.
Informed consent
Informed consent will be sought from all participants involved in this research. All project data will be stored securely in a de-identified format and only accessed by project staff. Individual responses will only be identifiable to other participants, where consent has been provided, such as during the themed media presentation. Additional consent will be sought from all participants where audio-visual footage is used. In providing feedback to sites, care will be taken to ensure that individual responses are not identifiable by other community members.
Dissemination
It is anticipated that the findings and relevant outcomes of this project will be reported in international peer-reviewed journal articles and presented at relevant national and international conferences. Findings and outcomes will also be presented at a celebratory event at the completion of the research project. A summary of research findings will be made available to participating sites. Results will be communicated to clinicians through local, statewide, national and international clinical networks and forums. Plain English summaries (including video abstracts) of this research will be prepared in consultation with the consumer advisory committee, suitable for people with aphasia (see online supplemental material 3).
bmjopen-2020-047398supp003.pdf (168.3KB, pdf)
Supplementary Material
Acknowledgments
We would like to thank speech pathology management staff at participating sites for their support of this research project and their assistance with ethics submissions. We would also like to thank members of the consumer advisory committee for their valuable input and Mr Geoffrey Binge for his expert guidance on respectfully engaging Aboriginal and/or Torres Strait Islander Peoples in this research.
Footnotes
Twitter: @AnemaatLisa, @VictoriaJPalmer, @LNL_Copland, @SarahJWallace
Contributors: All authors contributed to the design and drafting of the research protocol. LA was responsible for the initial draft of this manuscript. SJW, DAC and VJP provided critical review and editing and contributed to revisions and additions. KM (person with aphasia), KD (person with aphasia), JD (family member of a person with aphasia), BA (person with aphasia), GB (cultural capability officer for the Royal Brisbane and Women’s Hospital, and a proud Kamilaroi man from North West New South Wales, Australia), PM (family member of a person with aphasia) contributed to the development of the plain English summary and provided input on methods, recruitment processes and dissemination. All authors have read and approved the final version.
Funding: This work is supported by The University of Queensland Aphasia Rehabilitation Research Fund (2018–2023) and a New Researcher grant awarded by Speech Pathology Australia (2020). SJW is supported by a National Health and Medical Research Council (NHMRC) Emerging Leadership Investigator Grant (APP1175821) and LA is supported by The University of Queensland-Research Training Programme Scholarship (QARC).
Competing interests: None declared.
Provenance and peer review: Not commissioned; externally peer reviewed.
Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
Ethics statements
Patient consent for publication
Not applicable.
References
- 1.O'Sullivan M, Brownsett S, Copland D. Language and language disorders: neuroscience to clinical practice. Pract Neurol 2019;19:380–8. 10.1136/practneurol-2018-001961 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.GBD 2016 Stroke Collaborators . Global, regional, and national burden of stroke, 1990-2016: a systematic analysis for the global burden of disease study 2016. Lancet Neurol 2019;18:439–58. 10.1016/S1474-4422(19)30034-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Mitchell C, Gittins M, Tyson S. Prevalence of aphasia and dysarthria among inpatient stroke survivors: describing the population, therapy provision and outcomes on discharge. Aphasiology 2020:1–11. [Google Scholar]
- 4.Rose M, Ferguson A, Power E, et al. Aphasia rehabilitation in Australia: current practices, challenges and future directions. Int J Speech Lang Pathol 2014;16:169–80. 10.3109/17549507.2013.794474 [DOI] [PubMed] [Google Scholar]
- 5.Brady MC, Godwin CJ, Enderby CP. Speech and language therapy for aphasia after stroke: an updated systematic review and meta-analyses. Stroke 2016;47:e236–7. [Google Scholar]
- 6.Lam JMC, Wodchis WP. The relationship of 60 disease diagnoses and 15 conditions to preference-based health-related quality of life in Ontario hospital-based long-term care residents. Med Care 2010;48:380–7. 10.1097/MLR.0b013e3181ca2647 [DOI] [PubMed] [Google Scholar]
- 7.Kauhanen ML, Korpelainen JT, Hiltunen P, et al. Aphasia, depression, and non-verbal cognitive impairment in ischaemic stroke. Cerebrovasc Dis 2000;10:455. 10.1159/000016107 [DOI] [PubMed] [Google Scholar]
- 8.Hilari K, Northcott S, Roy P, et al. Psychological distress after stroke and aphasia: the first six months. Clin Rehabil 2010;24:181–90. 10.1177/0269215509346090 [DOI] [PubMed] [Google Scholar]
- 9.Flowers HL, Skoretz SA, Silver FL, et al. Poststroke aphasia frequency, recovery, and outcomes: a systematic review and meta-analysis. Arch Phys Med Rehabil 2016;97:2188–201. 10.1016/j.apmr.2016.03.006 [DOI] [PubMed] [Google Scholar]
- 10.Kadojić D, Bijelić BR, Radanović R, et al. Aphasia in patients with ischemic stroke. Acta Clin Croat 2012;51:221–5. [PubMed] [Google Scholar]
- 11.Worrall L, Sherratt S, Rogers P. What people with aphasia want: their goals according to the ICF. Aphasiology 2011;25:309–22. [Google Scholar]
- 12.Baker C, Worrall L, Rose M. ‘It was really dark’: the experiences and preferences of people with aphasia to manage mood changes and depression. Aphasiology 2020;34:19–46. [Google Scholar]
- 13.Pringle J, Hendry C, McLafferty E, et al. Stroke survivors with aphasia: personal experiences of coming home. Br J Community Nurs 2010;15:241–7. 10.12968/bjcn.2010.15.5.47950 [DOI] [PubMed] [Google Scholar]
- 14.Wray F, Clarke D. Longer-term needs of stroke survivors with communication difficulties living in the community: a systematic review and thematic synthesis of qualitative studies. BMJ Open 2017;7:e017944. 10.1136/bmjopen-2017-017944 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Andrew NE, Kilkenny M, Naylor R, et al. Understanding long-term unmet needs in Australian survivors of stroke. Int J Stroke 2014;9(Suppl A100):A100:106–12. 10.1111/ijs.12325 [DOI] [PubMed] [Google Scholar]
- 16.Walsh ME, Galvin R, Loughnane C, et al. Community re-integration and long-term need in the first five years after stroke: results from a national survey. Disabil Rehabil 2015;37:1834–8. 10.3109/09638288.2014.981302 [DOI] [PubMed] [Google Scholar]
- 17.Martin BJ, Yip B, Hearty M, et al. Outcome, functional recovery and unmet needs following acute stroke. experience of patient follow up at 6 to 9 months in a newly established stroke service. Scott Med J 2002;47:136–7. 10.1177/003693300204700605 [DOI] [PubMed] [Google Scholar]
- 18.Ullberg T, Zia E, Petersson J, et al. Perceived unmet rehabilitation needs 1 year after stroke: an observational study from the Swedish stroke register. Stroke 2016;47:539–41. 10.1161/STROKEAHA.115.011670 [DOI] [PubMed] [Google Scholar]
- 19.King RB, Hartke RJ, Lee J, et al. The stroke caregiver unmet resource needs scale: development and psychometric testing. J Neurosci Nurs 2013;45:320–8. 10.1097/JNN.0b013e3182a3ce40 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kersten P, Low JTS, Ashburn A, et al. The unmet needs of young people who have had a stroke: results of a national UK survey. Disabil Rehabil 2002;24:860–6. 10.1080/09638280210142167 [DOI] [PubMed] [Google Scholar]
- 21.Zawawi NSM, Aziz NA, Fisher R, et al. The unmet needs of stroke survivors and stroke caregivers: a systematic narrative review. J Stroke Cerebrovasc Dis 2020;29:104875. 10.1016/j.jstrokecerebrovasdis.2020.104875 [DOI] [PubMed] [Google Scholar]
- 22.Simmons-Mackie N, Holland AL. Aphasia centers in North America: a survey. Semin Speech Lang 2011;32:203–15. 10.1055/s-0031-1286175 [DOI] [PubMed] [Google Scholar]
- 23.Woelders S, van Der Borg W, Schipper K. The meaning of aphasia centres from the perspectives of people with aphasia and their relatives: understanding participation in the Dutch context. Aphasiology 2018;32:1490–512. [Google Scholar]
- 24.Palmer R, Witts H, Chater T. What speech and language therapy do community dwelling stroke survivors with aphasia receive in the UK? PLoS One 2018;13:e0200096. 10.1371/journal.pone.0200096 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Katz RC, Hallowell B, Code C, et al. A multinational comparison of aphasia management practices. Int J Lang Commun Disord 2000;35:303–14. 10.1080/136828200247205 [DOI] [PubMed] [Google Scholar]
- 26.Foster A, O’Halloran R, Rose M, et al. “Communication is taking a back seat”: speech pathologists’ perceptions of aphasia management in acute hospital settings. Aphasiology 2016;30:585–608. 10.1080/02687038.2014.985185 [DOI] [Google Scholar]
- 27.Pedersen PM, Vinter K, Olsen TS. Aphasia after stroke: type, severity and prognosis. Cerebrovasc Dis 2004;17:35–43. 10.1159/000073896 [DOI] [PubMed] [Google Scholar]
- 28.Klebic J, Salihovic N, Softic R, et al. Aphasia disorders outcome after stroke. Med Arh 2011;65:283–6. 10.5455/medarh.2011.65.283-286 [DOI] [PubMed] [Google Scholar]
- 29.Stroke Foundation . No postcode untouched: stroke in Australia 2017, 2017. Available: https://strokefoundation.org.au/No-postcode-untouched [Accessed 28 october 2020].
- 30.Code C, Petheram B. Delivering for aphasia. Int J Speech Lang Pathol 2011;13:3–10. 10.3109/17549507.2010.520090 [DOI] [PubMed] [Google Scholar]
- 31.Lalor E, Cranfield E. Aphasia: a description of the incidence and management in the acute hospital setting. Asia Pac J Speech Lang Hear 2004;9:129–36. 10.1179/136132804805575949 [DOI] [Google Scholar]
- 32.Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open 2013;3:e001570. 10.1136/bmjopen-2012-001570 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Clarke D, Jones F, Harris R, et al. What outcomes are associated with developing and implementing co-produced interventions in acute healthcare settings? A rapid evidence synthesis. BMJ Open 2017;7:e014650. 10.1136/bmjopen-2016-014650 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Bate P, Robert GB. Bringing user experience to healthcare improvement: the concepts, methods and practices of experience-based design. Radcliffe Pub: Oxford, 2007. [Google Scholar]
- 35.Robert G, Cornwell J, Locock L, et al. Patients and staff as codesigners of healthcare services. BMJ 2015;350:g7714. 10.1136/bmj.g7714 [DOI] [PubMed] [Google Scholar]
- 36.Louise L, Annette B, Locock L. Drawing straight lines along blurred boundaries: qualitative research, patient and public involvement in medical research, co-production and co-design. Evid Policy 2019;15:409–21. 10.1332/174426419X15552999451313 [DOI] [Google Scholar]
- 37.Blackwell RWN, Lowton K, Robert G, et al. Using experience-based Co-design with older patients, their families and staff to improve palliative care experiences in the emergency department: a reflective critique on the process and outcomes. Int J Nurs Stud 2017;68:83–94. 10.1016/j.ijnurstu.2017.01.002 [DOI] [PubMed] [Google Scholar]
- 38.Brady GC, Goodrich J, Roe JWG. Using experience-based co-design to improve the pre-treatment care pathway for people diagnosed with head and neck cancer. Support Care Cancer 2020;28:739–45. 10.1007/s00520-019-04877-z [DOI] [PubMed] [Google Scholar]
- 39.Palmer VJ, Weavell W, Callander R, et al. The participatory Zeitgeist: an explanatory theoretical model of change in an era of coproduction and codesign in healthcare improvement. Med Humanit 2019;45:247–57. 10.1136/medhum-2017-011398 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Palmer VJ, Chondros P, Piper D, et al. The core study protocol: a stepped wedge cluster randomised controlled trial to test a co-design technique to optimise psychosocial recovery outcomes for people affected by mental illness in the community mental health setting. BMJ Open 2015;5:e006688. 10.1136/bmjopen-2014-006688 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Piper D, Iedema R, Gray J, et al. Utilizing experience-based co-design to improve the experience of patients accessing emergency departments in New South Wales public hospitals: an evaluation study. Health Serv Manage Res 2012;25:162–72. 10.1177/0951484812474247 [DOI] [PubMed] [Google Scholar]
- 42.Raynor DK, Ismail H, Blenkinsopp A, et al. Experience-based co-design-Adapting the method for a researcher-initiated study in a multi-site setting. Health Expect 2020;23:562-570. 10.1111/hex.13028 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Tsianakas V, Robert G, Maben J, et al. Implementing patient-centred cancer care: using experience-based co-design to improve patient experience in breast and lung cancer services. Support Care Cancer 2012;20:2639–47. 10.1007/s00520-012-1470-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Tsianakas V, Robert G, Richardson A, et al. Enhancing the experience of carers in the chemotherapy outpatient setting: an exploratory randomised controlled trial to test impact, acceptability and feasibility of a complex intervention co-designed by carers and staff. Support Care Cancer 2015;23:3069–80. 10.1007/s00520-015-2677-x [DOI] [PubMed] [Google Scholar]
- 45.Needham C, Carr S. SCIE research briefing 31: co-production: an emerging evidence base for adult social care transformation. London: Social Care Institute for Excellence, 2009. [Google Scholar]
- 46.Staniszewska S, Brett J, Simera I, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ 2017;358:j3453. 10.1136/bmj.j3453 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Dimopoulos-Bick T, O'Connor C, Montgomery J. “Anyone can co-design?”: A case study synthesis of six experience-based co-design (EBCD) projects for healthcare systems improvement in New South Wales, Australia. Patient Exp J 2019;6:93–104. [Google Scholar]
- 48.Delbecq AL, Van de Ven AH, Gustafson DH. Group techniques for program planning: a guide to nominal group and Delphi processes. Glenview, Ill: Scott, Foresman, 1975. [Google Scholar]
- 49.Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract 2012;18:188–94. 10.1111/j.1440-172X.2012.02017.x [DOI] [PubMed] [Google Scholar]
- 50.Goodglass H, Kaplan E, Barresi B. Boston diagnostic aphasia examination. 3rd ed. Philadelphia: Lippincott Williams & Wilkins, 2001. [Google Scholar]
- 51.Australian Bureau of Statistics . Australian Bureau of statistics: ABS maps, 2020. Available: https://itt.abs.gov.au/itt/r.jsp?ABSMaps [Accessed 28 Oct 2020].
- 52.Rose TA, Worrall LE, Hickson LM, et al. Guiding principles for printed education materials: design preferences of people with aphasia. Int J Speech Lang Pathol 2012;14:11–23. 10.3109/17549507.2011.631583 [DOI] [PubMed] [Google Scholar]
- 53.Kagan A. Supported conversation for adults with aphasia: methods and resources for training conversation partners. Aphasiology 1998;12:816–30. [Google Scholar]
- 54.Hilari K, Lamping DL, Smith SC, et al. Psychometric properties of the stroke and aphasia quality of life scale (SAQOL-39) in a generic stroke population. Clin Rehabil 2009;23:544–57. 10.1177/0269215508101729 [DOI] [PubMed] [Google Scholar]
- 55.Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011;20:1727–36. 10.1007/s11136-011-9903-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004;24:105–12. 10.1016/j.nedt.2003.10.001 [DOI] [PubMed] [Google Scholar]
- 57.Tomkins B, Siyambalapitiya S, Worrall L. What do people with aphasia think about their health care? Factors influencing satisfaction and dissatisfaction. Aphasiology 2013;27:972–91. [Google Scholar]
- 58.Wallace SJ, Worrall L, Rose T, et al. Which outcomes are most important to people with aphasia and their families? an international nominal group technique study framed within the ICF. Disabil Rehabil 2017;39:1364–79. 10.1080/09638288.2016.1194899 [DOI] [PubMed] [Google Scholar]
- 59.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. [Google Scholar]
- 60.De Vecchi N, Kenny A, Dickson-Swift V, et al. Exploring the process of digital storytelling in mental health research: a process evaluation of consumer and clinician experiences. Int J Qual Methods 2017;16:1–13. [Google Scholar]
- 61.Donetto S, Tsianakas V, Robert G. Using experience-based Co-design to improve the quality of healthcare: mapping where we are now and establishing future directions. London: King’s College London, 2014. [Google Scholar]
- 62.Morgan D. The focus group Guidebook. Thousand Oaks: SAGE Publications, Inc, 1998. [Google Scholar]
- 63.Lyons R, McAllister L. Qualitative research in communication disorders: an introduction for students and clinicians. Guildford: J&R Press Ltd, 2019. [Google Scholar]
- 64.Coenen M, Stamm TA, Stucki G, et al. Individual interviews and focus groups in patients with rheumatoid arthritis: a comparison of two qualitative methods. Qual Life Res 2012;21:359–70. 10.1007/s11136-011-9943-2 [DOI] [PubMed] [Google Scholar]
- 65.Agency for Clinical Innovation . Redesign - Methodology for improvement and innovation: Factsheets, Diagnostics - Prioritising key issues, 2019. Available: https://www.aci.health.nsw.gov.au/__data/assets/pdf_file/0004/486769/Diagnostics_PRIORITISING-KEY-ISSUES.pdf [Accessed 28 Oct 2020].
- 66.Kearns Á, Kelly H, Pitt I. Rating experience of ICT-delivered aphasia rehabilitation: co-design of a feedback questionnaire. Aphasiology 2019;34:395–7. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2020-047398supp001.pdf (296.6KB, pdf)
bmjopen-2020-047398supp002.pdf (403.2KB, pdf)
bmjopen-2020-047398supp003.pdf (168.3KB, pdf)