Ethical issues raised by incorporating personalized language models into brain-computer interface communication technologies: A qualitative study of individuals with neurological disease

Abstract

Purpose:

To examine the views of individuals with neurodegenerative diseases about ethical issues related to incorporating personalized language models into brain-computer interface (BCI) communication technologies.

Methods:

Fifteen semi-structured interviews and 51 online free response surveys were completed with individuals diagnosed with neurodegenerative disease that could lead to loss of speech and motor skills. Each participant responded to questions after six hypothetical ethics vignettes were presented that address the possibility of building language models with personal words and phrases in BCI communication technologies. Data were analyzed with consensus coding, using modified grounded theory.

Results:

Four themes were identified. (1) The experience of a neurodegenerative disease shapes preferences for personalized language models. (2) An individual’s identity will be affected by the ability to personalize the language model. (3) The motivation for personalization is tied to how relationships can be helped or harmed. (4) Privacy is important to people who may need BCI communication technologies. Responses suggest that the inclusion of personal lexica raises ethical issues. Stakeholders want their values to be considered during development of BCI communication technologies.

Conclusions:

With the rapid development of BCI communication technologies, it is critical to incorporate feedback from individuals regarding their ethical concerns about the storage and use of personalized language models. Stakeholder values and preferences about disability, privacy, identity and relationships should drive design, innovation and implementation.

Introduction

Brain-computer interface (BCI) technology holds great promise as an alternative access method for people who lose their motor skills as a result of progressive neurological disease, such as amyotrophic lateral sclerosis (ALS), Parkinson’s disease, and multiple sclerosis [1,2]. A BCI for communication allows an individual to select language units, whether letters, words or symbols, through brain activity alone, thereby eliminating the need for muscle activations [3]. A neural signal, whether acquired through an externally worn EEG cap or an implantable chip placed directly on an individual’s brain, provides a method of expression by permitting individuals to control communication software through their brainwaves [4]. With the clinical goal to increase participation in decision-making, medical management and daily activities, the BCI communication system falls within the category of augmentative and alternative communication (AAC) technologies.

BCI communication systems are being built with integrated natural language processing (NLP) models [5,6]. The goal of adding NLP to communication technology is to (1) reduce the physical input necessary to produce an utterance; (2) reduce the cognitive load on the user; (3) increase the speed of communication; and (4) reduce the delays between the user formulating what they want to say and the device articulating the appropriate words [7]. NLP models are typically trained on very large data sets, with a goal of achieving near real-time communication. Unfortunately, there are no large data sets that are based on language produced by people who rely on AAC technology [8]. The last mile of NLP-based BCI communication will require training NLP models on personal user data sets. These user data sets will include words, phrases and connected text that appear frequently and have a high probability of re-occurring in one’s expressive output. Adding personal lexica to AAC language models provides a way to increase speed and reduce effort of message generation [9,10]. Wandmacher and Antoine demonstrated that adding a dynamic user model to standard corpora used for word prediction increases keystroke savings and reduces out-of-vocabulary words [11]. Though the promise of personalization has driven recent work in the field of BCI communication, there is little data on how end users understand and value the prospect of adding personal vocabulary to a device’s language corpus.

Inclusion of users and potential users in the design of new devices or technology, often called user-centered design (ISO 9241-210, 2008) [12], is a way to make technology development more responsive to user needs [13,14]. BCI research groups have been gathering data on the preferences and experience of BCI end users [15-22]. The advantages or disadvantages of adding personal vocabulary, especially frequently used words and phrases, to language corpora in devices, however, has not yet been explored with end users.

BCI technology development has been noted to raise ethical issues related to identity, normality, authority, responsibility, privacy, and justice [23, 49, 59]. While ethical issues are not unique to BCI technology, the current and potential capabilities of BCI technology raise them in different and important ways. Does identity shift if one relies on, or even merges with, a BCI device? Will relying on a BCI to get around in the world or communicate be stigmatizing or become normalized over time? Who owns the data that is collected by a BCI system? And if neural data conflicts with a first-person report ("I didn't intend to do that!"), which will we credit with authority? Who or what will we hold responsible when a BCI system fails to achieve its aim or causes harm? And will the same groups of people who volunteer for BCI studies be those who will one day have access to commercialized, and potentially expensive, products?

User perspectives on these and other ethical implications of current or future BCI devices have become a part of user-centered design [24-27]. The choice of an NLP model, and the data it is trained on, has ethical implications. Since how a user communicates is intimately linked to user values about relationships, privacy, identity, among other considerations, understanding user perspectives on the ethical implications of building personalized NLP models for BCI is an important part of adopting user-centered design in BCI communication technology.

In this study, we engaged with adults with neurodegenerative diseases who might need to rely on BCI communication technologies for expression in the future. We explored their values related to the possibility of storing personal lexica as word and phrase sets within the BCI systems.

Method

Participants

A total of 66 participants were recruited and placed into one of two study groups based on participant preference, either an interview cohort or a survey cohort.

The interview cohort included 15 participants (8 men, 7 women) ranging in age from 52 to 73 years with a mean age of 62 years. All but one participant identified their race as white, with one identifying as unknown. Seven participants reported a diagnosis of amyotrophic lateral sclerosis, three reported multiple sclerosis. The remaining five participants listed one of the following diagnoses: primary lateral sclerosis, multiple systems atrophy, spinal muscular atrophy, Parkinson’s disease or a dual diagnosis of multiple sclerosis and Parkinson’s disease. All participants lived in the United States except one survey respondent who resided in Germany. The participants’ current functional communication skills were self-reported via the Amyotrophic Lateral Sclerosis- Speech Severity Scale (ALS-SSS) [29]. For reference, the total range of score for the ALS-SSS is 0 (no speech) to10 (unimpaired speech).

The mean ALS-SSS [29] score was 9.2 (range of 4 to 10). All participants of the interview cohort were administered the Telephone Interview for Cognitive Skills (TICS) [30]. The TICS is an 11-item cognitive screening administered orally and found to have high reliability. The areas addressed include orientation, attention, memory and conceptual knowledge [30]. All participants achieved a score of 32 or greater (maximum score = 50) indicating that they demonstrated adequate cognitive skills to participate in the interviews.

The education level of participants ranged from 13 to 24 years with a mean of 17 years. Ten participants identified as being expert computer users and five identified as having some familiarity with computer use. Participants reported a wide variety of lifetime occupations, from clerical to pediatric physician. Table 1 provides additional details about demographic information.

Table 1.

Interview Cohort Participant Information

ID	Age	Diagnosis	Sex	Education (Years)	Occupation	Familiarity with computers	ALS- SSSa	TICS-mb
I-01	67	PD	M	16	engineer	expert user	10	39
I-02	58	ALS	M	16	sales professional	expert user	9	35
I-03	63	ALS	M	14	engineer	expert user	9	35
I-04	59	MS	M	17	journalist	expert user	9	35
I-05	59	SMA	M	14	IT professional	expert user	9	38
I-06	71	MS, PD	F	13	clerical work	some familiarity	9	37
I-07	52	MS	F	20	professor	some familiarity	9	40
I-08	73	ALS	M	18	editor, monk	some familiarity	7	41
I-09	61	ALS	M	16	engineer	expert user	9	40
I-10	59	PLS	F	17	real estate broker; teacher	some familiarity	9	35
I-11	59	MS	F	24	doctor	expert user	10	41
I-12	61	ALS	F	16	teacher	some familiarity	9	33
I-13	55	ALS	M	18	teacher	some familiarity	7	38
I-14	59	ALS	F	14	lab technician	expert user	4	26
I-15	72	MSA	F	20	teacher	expert user	8	35

Note. I =Interview Respondent, PD = Parkinson’s disease, ALS = Amyotrophic Lateral Sclerosis, MS = Multiple Sclerosis, SMA = Spinal Muscular Atrophy, PLS = Primary Lateral Sclerosis, MSA = Multiple Systems Atrophy, ALS-FRS = ALS Functional Rating Scale, ALS-SS = ALS Severity Scale, TICS-m = Telephone Interview of Cognitive Status – modified.

^aThis assessment has a range of 0-10 points.

^bThis assessment has a range of 0-50 points.

The survey cohort consisted of 51 individuals. Sixteen of them identified as male, 34 identified as female and one participant chose not to report their sex. Their ages ranged from 35 to 80 years old with a mean age of 56. The majority of the participants identified as white. Ten participants reported a diagnosis of amyotrophic lateral sclerosis, 18 reported Parkinson’s disease, and 20 reported multiple sclerosis. The remaining three participants listed one of the following diagnoses: primary lateral sclerosis, multiple systems atrophy or spinal muscular atrophy. The participants’ current functional communication skills were self-reported via the ALS-SSS [29]. The mean ALS-SSS [29] score was 7.82, with a range of 0 to 10. All participants displayed satisfactory performance on the true/false cognitive screening tool designed to assess reading comprehension of the online consent form. The mean level of education was 16.05 years, with a range of 12 to 23 years. Fifty-four percent of participants in the survey cohort said they had some familiarity with computer use, while 43% identified as an expert user. Participants reported on a wide variety of lifetime occupations, from teaching to fire fighter. Table 2 provides additional details about demographic information.

Table 2.

Survey Cohort Participant Information

ID	Age	Diagnosis	Sex	Education (Years)	Occupation	Familiarity with computers	ALS- SSa	Cognitive screeningb
S-01	80	ALS/PLS	F	13	bookkeeper	some familiarity	4	10
S-02	53	ALS	F	13	medical technician	some familiarity	4	10
S-03	77	PD	F	17	marketing/PR specialist	expert user	9	10
S-04	43	MS	F	18	fire fighter/paramedic	expert user	9	10
S-05	45	PD	F	17	teacher	expert user	10	10
S-06	45	MS	F	15	child care provider	expert user	8	10
S-07	57	MS	F	12	instructional aide	some familiarity	9	9
S-08	66	PD	F	14	artist	some familiarity	9	10
S-09	43	MS	F	16	office support specialist	expert user	10	10
S-10	58	ALS	M	14	engineer	expert user	0	9
S-11	35	MS	F	16	scheduler, copy editor	expert user	10	10
S-12	59	PD	F	19	teacher	expert user	8	10
S-13	62	PD	M	16	engineer	expert user	6	10
S-14	71	PD	F	20	scientist, attorney	some familiarity	10	10
S-15	42	MS	F	14	healthcare provider	expert user	10	10
S-16	39	PLS	F	16	property manager	expert user	5	10
S-17	48	ALS	M	14	landscaper	some familiarity	4	9
S-18	55	SMA	M	16	IT support	expert	4	9
S-19	47	ALS	M	20	scientist	expert user	4	7
S-20	73	ALS	F	15	nurse	some familiarity	7	9
S-21	56	ALS	M	13	HVAC technician	some familiarity	6	10
S-22	78	ALS	M	16	professor	expert user	9	9
S-23	69	ALS	M	16	project manager	some familiarity	4	10
S-24	79	PD	F	14	homemaker	some familiarity	9	9
S-25	73	PD	F	18	teacher	some familiarity	8	10
S-26	57	MS	F	14	medical assistant	some familiarity	10	9
S-27	67	PD	M	16	technology sales rep	unreported	9	10
S-28	62	MS	F	20	attorney	some familiarity	10	9
S-29	60	MS	F	12	retired	some familiarity	7	10
S-30	70	PD	F	17	manager	some familiarity	10	10
S-31	65	MS	M	16	grant manager	some familiarity	8	10
S-32	36	MS	F	12	administrative assistant	some familiarity	10	8
S-33	48	PD	F	21	dentist	expert user	10	10
S-34	79	PD	M	16	service technician	some familiarity	8	10
S-35	64	MS	F	17	dental hygienist	some familiarity	8	10
S-36	47	MS	F	18	editor	expert user	10	10
S-37	62	MS	F	14	credit analyst	expert user	8	10
S-38	73	PD	F	16	teacher	some familiarity	10	9
S-39	46	MS	F	14	truck driver	some familiarity	10	9
S-40	70	MSA	M	22	engineer	some familiarity	5	9
S-41	72	PD	F	16	billing clerk	some familiarity	8	10
S-42	75	ALS	M	23	professor	some familiarity	5	10
S-43	77	PD	M	16	teacher	some familiarity	6	8
S-44	69	PD	F	12	bookkeeper	expert user	8	9
S-45	35	MS	other	16	parent	expert user	9	10
S-46	72	PD	F	18	physical therapist	some familiarity	9	10
S-47	50	MS	M	16	researcher	expert user	8	10
S-48	61	MS	F	19	teacher	expert user	10	10
S-49	55	MS	F	12	dispatcher	some familiarity	9	10
S-50	43	MS	M	16	graphic designer	some familiarity	8	10
S-51	71	PD	F	18	nurse practitioner	some familiarity	10	10

Note. S= Survey Respondent, PD = Parkinson’s disease, ALS = Amyotrophic Lateral Sclerosis, MS = Multiple Sclerosis, SMA = Spinal Muscular Atrophy, PLS = Primary Lateral Sclerosis, MSA = Multiple Systems Atrophy, ALS-SS = ALS Severity Scale

^aThis assessment has a range of 0-10 points.

^bThis online assessment has a range of 0-10 points.

Recruitment

Recruitment occurred through the university hospital clinics, as well as national patient advocacy organizations, such as the local ALS Association (ALSA) and the assistive technology listserv for the ALSA. The discovery cohort function of the university’s electronic medical record system provided access to all patients with neurodegenerative diagnoses who were interested in being contacted for research studies.

The study was approved by the IRB at Oregon Health & Sciences University, IRB#20377. All participants provided informed consent prior to enrollment.

Materials

Six vignettes of hypothetical situations and question probes were designed to explore ethical issues related to personalization of vocabulary for future BCI communication devices. An interview guide was modeled on prior studies investigating perspectives of stakeholders on future motor and communication BCI or closed-loop neural devices [25,27]. The interview guide was developed by the multidisciplinary team of scientists, clinicians, and ethicists and piloted with two participants who had worked with the lab’s BCI spelling research. The guide was iteratively refined over the course of the first five interviews. Table 3 summarizes the vignettes and interview questions (the complete interview guide is available upon request).

Table 3.

Hypothetical vignettes and prompts presented to interview cohort

Hypothetical vignettes (summary)	Interview prompts (examples)
An individual with Parkinson’s disease considers providing BCI researchers with a collection of personal letters she has written to her daughter over many years.	What do you think about personalizing a BCI device and its effect on the relationship of a device user and those important to them?
An individual with stroke-related communication impairment considers providing BCI researchers with YouTube videos he made earlier in his life, including ones that involved crass and violent language.	How can the words a person used during an early phase of life connect with how a person views themselves now?
An individual with ALS considers providing BCI researchers with recordings of her comedy shows, an area of her life of which her family is unaware.	What do you think about one’s ability to use technology to paint a picture about who one is or wants to be?
An individual with advanced multiple sclerosis considers providing BCI researchers with personal emails, including ones that involve her romantic history.	What do you think about the ability of technology to reveal things that people don’t even know about themselves?
Individual with ALS considers providing BCI researchers with work-related emails in which her way of speaking differs from how she speaks with friends and family.	How might a person try to change how others view them by choosing different types of words?
Spouse of individual with locked-in syndrome from late-stage ALS considers providing BCI researchers with her partner’s personal diary.	What role should those people important to the person using a BCI play in collecting language sources to improve performance?

The same vignettes were presented to the online survey cohort of 51 individuals. The question probes in the interview guide were transformed into Likert scale questions with accompanying free-response question prompts. All open-ended responses to vignettes were included in this qualitative data analysis. Likert scores will not be reported here.

Before each interview or survey, participants were presented with an 8-minute YouTube video describing a research participant using a BCI spelling system. The video ensured a basic and uniform understanding of the technology presented in the vignettes.

Procedure

For the interview cohort, a reflective-projective interview technique was used to explore views of ethics-related issues. A reflective-projective technique allows individuals to use their personal experience with disease or disability to inform their perspective on ethical issues related to neurotechnology under development [25,27]. The first three interviews were conducted in-person, either in-home or in a location chosen by the participants. Due to the COVID pandemic, 11 interviews were moved to a virtual format. One interview was conducted using an iterative email format due to participant preference. This participant chose to be part of the interview cohort, even though she was too severely dysarthric to participate through speech. To accommodate her participation in the interview cohort, therefore, she was invited to use her most efficient mode of communication, writing, to answer questions. All interviews were conducted and recorded by members of the research team (EK, MK). Interview recordings were anonymized and sent to a third-party service for document transcription. For the online survey cohort, surveys were presented to participants through an electronic survey link, Qualtrics (Copyright © 2020, Qualtrics, Provo, UT). Participants in both cohorts were provided with a small monetary incentive for participation.

Analysis

A total of 995 minutes of audio-recorded interview data were collected from the interview cohort. Each interview ranged in length from 43 to 102 minutes with a mean time of 71 minutes per interview.

Interviews were analyzed using consensus coding and modified grounded theory [31,32]. Grounded theory is an inductive, iterative process of cycling between the qualitative data and emerging themes. A grounded theory approach is particularly useful when there is minimal literature or prior theory to explain a phenomenon of interest, such as was the case in the present study. The research team adopted a modified approach to grounded theory by implementing some deductive initial categorization via directed content analysis [56,57]. Some categories (e.g., privacy, identity, agency) and codes (e.g., hype of technology, technology limitations) were drawn from prior research on stakeholder views of neural technology [25,27].

Transcripts were managed using the qualitative data software program Atlas.ti Cloud (© 2002–2020 – ATLAS.ti Scientific Software Development GmbH, v1.0.15-master–2020-11-25). Analysis began after the first three interviews were conducted and data collection continued until thematic saturation. Corbin and Strauss argue that theoretical saturation occurs at the “the point in category development at which no new properties, dimensions, or relationships emerge” ([58] p143). EK and IS did an initial separate read of transcripts prior to coding to develop a holistic understanding of each interview. This pre-coding read was conducted in parallel to data collection (after the first three interviews) and was important in order to contextualize content, for example if interest or understanding of BCI technology seemed to evolve over the course of an interview. Thematic saturation was achieved, and cessation of interviews occurred, based on this pre-coding process.

EK and IS conducted initial or open coding in which raw data were organized into preliminary thematic categories. The first stage involved focused coding of all interviews by both EK and IS. A comparison of codes from EK and IS was conducted after each interview and code consensus reached for each interview. This consensus process and refinement of codes continued for the first ten interviews. A codebook consisting of 185 distinct codes was generated in this process. IS was the primary coder for the remaining five interviews with EK as a secondary coder. Agreement on coding of all interviews was reached by EK and IS. (For interviews 1-10, both coders independently coded and then compared and reached consensus on all codes. For interviews 11-15, IS coded based on the codebook and EK read coded transcripts and agreed or marked codes for later discussion and consensus). Free responses from the 51 online surveys were collected and the codebook from the interviews was applied by IS and EK independently to this data set, and disagreements resolved through discussion. No new codes were added to the 185 codes in the codebook after analyzing the free responses from the online surveys. A second stage involved categorizing, synthesizing, and integrating the codes into larger groups of data. EK, MFO, and MK reviewed all 185 codes and grouped these into categories of disease, technology, and values. All interview segmented data and free response data associated with each of the codes in each of these three categories were then read independently by EK, MFO, and MK. The most prominent themes in each of these categories were identified (disability, identity, relationships, privacy).

Below, respondents who were part of the interview cohort are indicated with an I and their accompanying participant number after their comments. Respondents who provided written free responses in the online surveys are indicated with an S and their accompanying participant number following their comments.

Results

Individuals with neurodegenerative diseases expressed their opinions on the prospect of personalizing language models within BCI technologies. Respondents demonstrated a general understanding of how communication with BCI technology works and how personalization may influence some aspects of their expression. Respondents were generally interested in the topics of both BCI technology for communication and personalization of language models, and many expressed support for personalization. Based on their comments, it was clear that respondents drew on their own experiences with social media, technology, and mobile devices, as well as their experiences with or their anticipation of future communication impairment. Support for language model personalization was moderated by opinions of how building a customized word set could affect privacy, sense of self, and valued relationships. Respondents wanted researchers who are adding personal lexica to language models to take into account their values related to these three areas.

All study participants had a diagnosis of a neurodegenerative disease that did or could impair communication ability. For many participants, the topic of communication was highly salient and they recognized the value that a personalized lexicon in a BCI device holds.

I would imagine for myself if I didn't have the ability to communicate, I probably would be less likely to want to continue to live because I'm not sure what – just there would be no pleasure in life.

(I-11)

You know, communication is so important in one’s relationship with anyone, I would think that any way of enhancing it would make it better. So, I think a device that would facilitate that would really be important.

(I-08)

I don't really understand how a BCI works but it seems to me like if a machine had more information to learn from about a person the response would be more genuinely that [of the] person's words or intent.

(S-12)

Imagining myself in this position, I would want my loved ones to know that they're actually communicating with me - personalizing the BCI device to match the voice of my written word would make it feel like I'm still the person I was - just with new challenges.

(S-11)

Motor degenerative diseases rob a person’s body of many things, including things that are unique to them; how they walk, their handwriting, how they gesture their arms when they talk, etc. while almost no one realizes this until it’s too late, getting information from sources that are unique to that individual would be the best way to personalize any communication opportunity.

(S-04)

But respondents noted that using assistive technology with a personalized language model is a multifaceted challenge. One respondent summarized this well. And another framed this challenge in terms of inevitable trade-offs.

It’s not black and white and it’s not clear cut. And on the surface, it looks pretty straightforward. Well, you give a bunch of your stuff and then the machine speaks just like you do. Well, yeah, but there’s all these other things, right? And it’s not just, okay, you give us this stuff, we’ll give you a machine that talks like you. But, there’s, okay, what happens between those two points, and particularly with the personal data and stuff?

(I-09)

Communication is always personal. Regional mannerisms, personal sayings, patterns of speech and so on are individual. I do feel that personalization is important, as individuality is integral to a person's well-being, but personalizing things also gives away things about a person that they might not want everyone to know or see.

(S-15)

Four principal themes emerged from the interviews and surveys related to disability, identity, relationships, and privacy (Table 4).

Table 4.

Four principal themes that emerged from interviews and online surveys

Theme
1. Current or anticipated disability influences hopes for a personalized language model
2. Personalization of a language model affects identity
3. Personalized language is important to relationships
4. Building a personalized model affects privacy

Theme #1: Current or anticipated disability influences hopes for a personalized language model

Respondents' views of their disease and anticipated progression of their disease over time shaped what they wanted out of a personalized language model. At a minimum, respondents wanted a language model to facilitate communication around satisfying “physical” (I-11) or basic life needs, like “when you want to eat, when you want to have your bowels emptied, or whether you want to have a discussion about some topic.” (I-04) But many respondents wanted more than just this minimum capability. They wanted a personalized language model that allowed them to communicate in ways and for purposes similar to how they do currently. Respondents noted that having a neurodegenerative disease does not change most of what a person wants out of communication. The potential positive impact of a speech generating device with personalized vocabulary was viewed by participants through the lens of their experience with or anticipation of communication limitations.

[A]ny of the things that we think about now when we are able to communicate completely are still there when we have more difficulty communicating probably…I don’t know why things would change just because you suddenly have a different means of communication. I would think you would still have the same kinds of feelings, and maturity, and knowledge.

(I-06)

I would argue that anyone who has reservations about using technology has not fully experienced the complete and total frustration of not being able to communicate… anything is better than that.

(S-04)

I am losing my ability to communicate with my loved ones effectively and it scares me. This would help so very much.

(S-06)

Neurodegenerative disease can be isolating, both because it can limit interactions with others but also because others may not be willing to adapt to new types or patterns of communication (i.e., using a synthetic voice instead of natural speech). Respondents want personalized language models to exert control and reduce isolation (e.g., keep conversation partners engaged).

As these chronic diseases get worse, oftentimes, there are people in our lives more often than we want. So, certainly, I would want the ability to monitor what was coming out of my BCI machine…So, I think it's important to have that ability to control – to control the technology a little bit.

(I-07)

The person using the device is already at the mercy of their disability. They depend on other people to do many if not all things for them. They should be given as much control over their decisions as possible, and that includes how they communicate. A person's agency is sometimes all they have left, and taking that away without their consent is unjust.

(S-15)

If you think about the ideal device, [it] would allow you to communicate as you did today. No changes, no filters, no delays. I mean, that's the idea. I think it would definitely keep a person from feeling more isolated or different. I think that would be important.

(I-09)

Theme #2: Personalization of a language model affects identity

The words and phrases that one uses for expression and daily conversation can be considered part of one’s identity. The use of code-switching, where a person changes how they speak based on their communication partner, is an example of how people use language to both reflect and shape their identity. Respondents talked about the perceived benefits of a personalized language model in terms of their identity. Succinctly stated, “the words a person uses and how they use them are part of a person’s character.” (S-03)

I think your style of communication, the words you choose, the way you put sentences together, I think that is as individual as each person. And so it's part of what makes you the person you are is your style of communication and what you – and how you string words together into sentences and paragraphs and get your meaning across. And so I think it's important to keep that. Because if we are all the same, we lose our – the essence of who we are.

(I-07)

Respondents expressed a desire for a personalized language model that allowed them to preserve important elements of their identity and allows them to change other elements. Some respondents worried that their disease might constrain their ability to express personality through humor, or sarcasm, or even curse to make a point. Choosing a speech register, where one speaks more informally or more restricted depending on one’s communication partner or situation, was a value that respondents tied to their identities. As such, respondents wanted a language model that allowed them to continue to communicate in familiar and valued ways to them.

So, I would think that [a personalized language model] would allow the program to have your patients’ personalities come out, to allow them to be saying things that are sarcastic or humorous.

(I-11)

This is particularly true in communicating with loved ones. People create personal registers to speak with intimate partners. One respondent poignantly notes how important it is to her to be able to joke with her spouse (and more specifically, the prospect of losing this).

There's a very real chance that I'll lose my ability to speak. I mean, already, I'm very raspy and I never used to be. I used to have a very clear voice. I don’t anymore. But to lose that ability to banter with my husband would be – I'd just shoot myself in the head because there'd be no point. So, to be able to keep that is huge. It keeps your humanness, it keeps your relationship going, it's just – it's part of being alive.

(I-07)

Some viewed the use of a personalized language model as a way to enable self-improvement through the feedback that would occur in using a language model or to gain self-knowledge. For instance, realizing "Wow, that's how I talk?". (I-07) Respondents suggested that a personalized language model could help them improve in other ways. For instance, some wanted to eliminate a tendency to use “like” (I-07) or “uh-huhs” (I-09) in conversation.

Respondents recognized the potential power of personalization to craft an image of how others see them.

If a person wants to affect how people view them, she can deliberately use words that fit the image [she] wants to convey. If a person wants to appear to be intelligent, carefree, tough, sophisticated, caring or serious, for example, she can choose her words accordingly. This may indeed cause a listener to view her in that way, unless her actions, behaviors, or conversational content fail to support the image created.

(S-14)

And one respondent viewed this power to shape how others see them as a kind of temptation.

[I]t’s really tempting to be who I want to be. I think we all have that side to ourselves. I could be a little better than I really am…

(I-13)

Others preferred a personalized language model that would not change how they presented themselves to others. One person noted:

And for myself, I would not try to temper it all that much because, ultimately, how we communicate is part of who we are. And even if I don’t like some of the things – the style that I communicate, it's who I am. And if I didn’t make a concerted effort at a certain point in my life to change it, if I'm still communicating that way now, why should I hide it? Just because now I have an opportunity to censor what the machine might do for me…If I didn’t make the decision to stop swearing or to swear more or whatever, why should I change it now? Because that's what people around me know and understand me to be.

(I-07)

Some respondents viewed designing a language model in order to shine a positive light on themselves as a kind of manipulative “Wizard of Oz” (I-03) or as “self-defeating” (I-08). One respondent summarized the conflict well, noting that while some people would want to “craft a narrative to make them in as positive a light” (I-13) as possible, others would not want this.

Theme #3: Personalized language is important to relationships

A central theme throughout the interviews and survey is a concern with how personalizing a language model might maintain, improve, or even in some ways threaten valued relationships. Respondents talked about their valued relationships in terms of different contexts, roles, and levels of familiarity. They referred to frequently used words or phrases (often called speech registers) that are integral parts of these relationships. Some of the valued relationships referenced included spouses and children, distant relatives, co-workers, friends, clients, and care providers.

Respondents talked about individualizing a language model that could accommodate talking differently to different people. One respondent wrote:

I believe it would be important to have different vocabularies, on a device, to speak to different groups of people.

(I-14).

One bilingual English-Arabic speaking respondent noted that having a language model that accommodates multiple language use or code switching within an utterance also would be important.

Me and my brother, we go grocery shopping these days…so, I tell him “linadhhab shopping,” which is “let’s go shopping.” I mix things.

(I-05)

Some respondents discussed this in terms of different contexts (employment, religious community, medical clinic, family and social gatherings), where different speech registers were required for conversational partners.

I could be more crass with my friends, coworkers after work and hanging out. I don't speak crassly with my mother, mother-in-law, or grandmother. An example would be ‘The f---ing Bears let another game slip through their fingers!' is what I would say to my friends/coworkers & ‘Dang it, the Bears let another game slip through their fingers!' I would say in polite company.

(I-14)

In order to use a range of speech registers or to code-switch with different conversation partners, many respondents expressed a desire for the capability to adjust or “toggle” the “settings”, “modes”, “folders”, “lingo” of their language model depending on their current partner or context. Respondent suggestions included: a “family” setting or a “when I’m comfortable with people around me setting” (I-07), a “business personality” or an “out with friends personality” (I-09), a “baby talk button”, an “out with the guys button”, and a “talking with the wife button” (I-03), or a “mom” setting, a “business person” setting, and a “casual friend” setting (I-10).

Others talked about wanting a personalized language model that allowed them to maintain different levels of comfort or intimacy with different language partners. Again, the personal speech register that was established between intimate partners was important to maintain. This was discussed in terms of the role of communication between spouses or other close family. One respondent noted that such a device would improve relationships with many family members.

Communication between close family and friends often includes private short hand or secret messages that make it intimate and that the rest of the world does not understand. By including this into the BCI system, it would honor those special relationships and help keep the intimacy of the relationship important and private.

(S-08)

[I]t helps you become more independent and a viable companion and better friend, mother, daughter, wife.

(S-12)

Respondents talked about the important role that their family would have in constructing their personalized language model - choosing what data their device was trained on - as a kind of “shared decision” (S-13). For some, the choice of family member involvement was driven by a sense of who best understands a person’s communication style (and who does not).

I think having someone like – in my case, with my husband, because we have a good communication style together, it would make a lot of sense to have him as part of that because he would be able to say, ‘[She] wouldn't say that.’ Whereas my mother would be like, ‘Well, [she] SHOULDN’T say that.’ Thanks, mom.

(I-07)

Some respondents acknowledged that family members sometimes knew more about how they tended to communicate than the respondent themselves and that this might “aid in personalization of a device.” (S-01).

The people in my sphere know my “isms” more than I do. I’m sure they can tell you all the little repetitions I do.

(I-02)

Whereas for others, the choice of family involvement was driven by a sense of who best knows a patient’s values.

I know for me, my wife would have a role, my kids would have a role in this, but that would be the extent of it for me…They know me, more about my progression with the disease, my values, and what I want for myself short- and long-term.

(I-13)

Not all respondents welcomed the involvement of others in building a language model and some would want limitations on others’ involvement.

I don't think people close to them get to have a say in this. It really should be up to the person using the device.

(S-12)

Theme #4: Building a personalized language model affects privacy

Respondents recognized that building a personalized language model for a BCI device might affect their privacy. Some respondents were already attuned to privacy issues inherent to AAC use. For instance, one respondent had observed how privacy was violated when others read messages prior to completion.

Most people don’t wait for somebody to finish [selecting words]. They look at their screen, and they start reading what they’re typing.

(I-03)

Depending on what personal communication data are used or who is involved in the collection or curating of these data, private information could be revealed. As one person noted:

I don’t know whether you want someone to know everything you have thought or [your] reactions… Because I understand in people’s lives there are things you want to keep private.

(I-04)

Many respondents noted that a device incorporating a personalized language model at times might reveal too much or reveal the wrong thing to the wrong person. One respondent wrote that they wouldn’t want “intimate emails to a lover” seen by family (I-14), information about struggling with depression (I-03) revealed to others, or a history of being the subject of “a criminal act, rape, or other type of traumatic event” somehow revealed (S-01).

Some respondents expressed a desire to protect their privacy by maintaining or exerting control over what personal information was used to build their language model. Privacy concerns lead respondents to express a preference for wanting to edit (“keep, delete, keep, delete” (I-02)) what personal communication data are used to build their customized language model.

Respondents recognized that personal communication data might affect not just their privacy but the privacy of family and friends. An individual’s personal communication data might include potentially sensitive information about others, particularly family members. Revealing this information might be experienced by others as “a betrayal of trust” (S-02). Respondents were attuned to the potential privacy harms to family members both because of their inherent concern for family member well-being but also for practical needs in being “dependent upon [others] for mental & physical support” (S-01).

I’m reliant on someone to make sure I get fed and breathe. I don’t want to piss them off…I need those people to not hate me down the road.

(I-02)

A handicapped person can't afford to alienate his/her support people.

(S-01)

It should be noted that a heightened concern about privacy was not shared by all respondents. Some felt that they were not private persons by nature.

I’m not bothered by somebody getting into even my old love letters from when I was just [inaudible] or maybe communication between [spouse] and I, or somebody even getting into my computer. It’s like, help yourself. Seems like a waste of your time. So, for me, what it comes down to, I certainly don’t want them to get into my bank account, but beyond that I’m pretty much an open book.

(I-03)

And yet others lamented that loss of privacy in communication would just be one more kind of privacy loss that often accompanies neurodegenerative disease.

So much of a patient’s modesty and privacy are taken away if they become disabled.

(S-05)

At some point, your level of disability robs you of all physical privacy (others must address your health and hygiene for you), and your mind becomes the one place that you have to yourself, but it is also the one thing controlling all things that are uniquely “you”.

(S-04)

Some viewed privacy as something that might have to be sacrificed in the “war” (I-04) against their neurodegenerative disease.

Personally, I don't worry about [privacy] that much, especially if it were for making a device that would help me to communicate.

(I-04)

On the whole, respondents recognized a tension may exist between providing personal information to improve device capabilities and protecting privacy related to that information and that a “balance must be found individually.” (S-10)

There is always a fear in divulging too much. Information is powerful and in the wrong hands can be used to hurt instead of help. However, sometimes the only way to truly help someone is to have all of the information. It can be a slippery slope.

(S-06)

Discussion

Language databases that drive word prediction increasingly are included in BCI communication technologies, as well as speech generating devices within the AAC field. Often, but not always, personalized vocabulary is stored during device use. Personalized language models can enhance AAC technologies [9], especially for adults with neurodegenerative disease who were fully competent speakers with large vocabularies that varied according to whom they were talking to or what they were talking about [2]. The verbal and nonverbal interaction patterns that are interrupted by the progressive loss of motor speech and physical abilities may be enhanced by technologies that learn these large vocabularies, and provide support for conversation. Similar to current technology platforms, such as text or email, NLP now includes functions that suggest word or phrase completions, autocorrect unfamiliar spellings, or propose infrequent proper names or idiosyncratic language. These functions are becoming expectations for rapid written exchanges. NLP also provides frequency of usage information about words and phrases which could enhance language use within BCI communication technologies. The combination of ubiquitous natural language processing, stored personal vocabulary, and machine learning improves the ability to customize technologies. This current study examined the perspectives of individuals with neurodegenerative disease about the prospect of including personalized language models driven by machine learning in BCI communication technologies, and the ethical trade-offs this might entail. The data suggest that this group of respondents, who may rely one day on AAC, were generally supportive of permitting devices to store personal lexica, but clearly expressed concerns about how personalizing BCI communication technologies might affect their privacy, valued relationships, and identity.

Privacy has been recognized as a critical consideration in the implementation of AAC [33-35] and BCI for communication [36-41]. Privacy is a complicated concept [42]. In the context of BCI communication technologies, privacy involves three different types - informational, decisional, and physical [37]. In the current study, respondents were concerned about how a communication system that stored their personal lexica would affect all three types of privacy. They expressed concern about informational privacy, where partners could violate proxemics by looking over one’s shoulder and reading an incomplete or personal message that was being composed [43], decisional privacy, where the system might affect their intended messages for end-of-life care; and physical privacy, where the system might interfere with personal care by family. The responses shared regarding the personalization of a BCI system for communication raises additional concerns. Individuals who rely on AAC may not want to risk revealing all the data used to train a language model, such as personal history (e.g., depression, traumatic events), previous words that are no longer spoken (e.g., cursing), or latent attitudes toward others (e.g., frustration, resentment). Respondents were concerned about the inclusion of an individual’s vocabulary in the language database and the availability of private information from different sources. For example, the appearance of inappropriate, embarrassing or harmful word choices if the device accessed words from prior uses may not fit within a specific verbal interaction or task, or with a specific partner. Respondents knew that privacy is fluid with a progressive illness, and expressed that personal word and phrase sets need to be sensitive to this. They provided many examples of how communication changes in the setting of new speech and physical challenges. While most people rarely talk about their personal hygiene needs, for example, these words become critical if one comes to rely on others for care. Words used in a business meeting might not reflect current needs when employment is no longer an option. Communication partners who are not familiar may become the very ones who provide intimate care, thus moving from one part of a person’s social network to another.

One of the features that people who rely on AAC want from a communication device is the ability to maintain social closeness [44]. Light and McNaughton [45] argue that sociorelational skills are one of the key domains for individuals who use AAC to attain and demonstrate communicative competence. Maintaining social closeness or intimacy is important to people with neurodegenerative diseases [46]. The current study found that a personalized BCI communication system is a way for individuals to maintain the closeness of their valued relationships. Using familiar phrases or idiosyncratic word choices was viewed as a way to hold onto or even reinforce relationship bonds amidst the challenges and uncertainties of a neurodegenerative condition.

A central finding of the current study is that potential users view personalized language as a tool to modulate levels of social closeness with everyone in their lives. This is consistent with a social networks approach to AAC, where the means of communication, types of messages and communication partners vary according to the social networks available to the person who relies on AAC [47]. People with neurodegenerative conditions establish and nurture rich and diverse social networks as competent adult communicators. As they experience speech and motor loss, they want to use a BCI communication device to modulate their registers when speaking with communication partners in different social networks. They want to generate words and phrases to express one register, for instance, to talk to close friends, another to interact with acquaintances, another for paid caregivers, yet another with distant or estranged family, and finally a separate register for intimate family. The abilities to adjust one’s speaking registers, select different words or phrases and modulate interaction styles based on one’s conversational partners, are expressions of autonomy. Respondents want the ability to toggle between vocabularies in order to control the level of intimacy in their communication. As physical impairments lead to a loss of autonomy in other areas of life (e.g., physical privacy) and a shrinking of their social networks, individuals with neurodegenerative disease want to more finely exert control over their social world.

There has been recent interest in whether neurotechnologies, like BCI, might affect a user’s identity [23,27]. In the case of BCI for communication, concerns about identity are often tied to potential disruptions of agency (e.g., ambiguity over whether a device or person “chooses” a word) [41]. In the current study, a personalized BCI for communication was primarily viewed in terms of positive effects on identity. A person’s identity, both in narrative terms (the “story” a person tells about who they are) and relational terms (how others view them), can be affected by neurotechnology [48-50]. In the current study, respondents viewed a personalized BCI for communication as enhancing their ability to express words and phrases reflective of their identity. Choosing what lexical items or phrases are used to personalize a device was viewed as a way to present oneself to the world. Some respondents wanted family and others to have a say in what gets included in the personalization of a language model, believing that their family or others recognize more about how they communicate, and maybe who they are, than they do.

BCI technology is moving at lightning speed through the three stages of technological innovation that are proposed by Lane and Flagg [51]: scientific research, engineering development, and industrial production [52]. User-centered and values-based design can incorporate the perspectives of users and potential users throughout the BCI innovation process [14,24]. In the current study, the importance of gathering end user perspectives on BCI technology, specifically building personalized language models, was a common theme expressed by participants.

If end user values and perspectives are not included in the initial stages of development, then there is a danger that the products, the technology-based innovations, will not be relevant and will not lead to benefits for the intended population of end users [13,53]. While this is clearly a recommendation derived from the data obtained by people with potentially severe motor impairments who are considering BCI-based communication, it also applies to a more general class of communication device. Indeed, development of any speech generating device within the field of AAC should examine the values of end users regarding personalized language storage. Unfortunately, asking these ethical questions regarding the storage of personalized vocabulary has not been a focus within AAC, thereby amplifying the importance of these results even more. Taylor and Balandin [55] warn us that ableist assumptions may limit the involvement of people with complex communication needs in AAC research and development, thus limiting outcomes for effective device development and negatively affecting the conduct of fair and equitable research that benefits people with complex communication needs and other stakeholders. Light and McNaughton [54] stress that people must be considered first when designing AAC technology. They suggest that the design of AAC technologies has critical implications for end users with the most significant motor, cognitive, language, and sensory impairments.

In order for people with disabilities to want to adopt BCI communication technologies, design must be based on the foundational knowledge that is acquired through understanding their values. This is a critical time in the development of BCI communication technology to explore stakeholder values, such as those related to integrating NLP models and storing personalized vocabulary. As two respondents powerfully expressed, “The ability to communicate and be understood is a vital part of quality of life, and there may come a time when I, myself, am unable to communicate. If my feedback…helps further the development of technology…I've helped fight this illness just a little bit. “(S-15) And, “I'm happy to see that you are trying to help folks who need this! I'm hoping it's sooner than later. With my disease I could wake up tomorrow and need it.” (S-07)

Limitations

There are important limitations to the findings of this study. First, most research participants were white, college educated, and technologically savvy. It is possible that ethical values or concerns were missed or underexplored because the perspectives of marginalized or underrepresented groups were not sufficiently included. Second, the positioning of the interviewers and of the research participants may have influenced the views expressed. The interviewers were both white, clinicians (neurologist, occupational therapist), and members of a BCI research group. Some participants had pre-existing relationships with the BCI research team as participants in previous BCI-related studies and others explicitly expressed interest in participating in future BCI studies (beyond the current qualitative interview study). Though the interview guide was designed and interviews conducted to encourage participants to express a full range of opinions on BCI technology, it is possible that the positioning of participants shaped the views expressed. Third, interviews were conducted remotely once the pandemic restricted in-person visits. This may have affected responses provided by participants who were not as comfortable with videoconferencing. Fourth, the use of hypothetical vignettes, though an accepted method in exploring views about future technology, is not without limitations. Respondents may have over or under-identified with particular vignettes or vignettes may have engaged individual biases in ways that were difficult to account for in the analysis. In addition, though vignettes are intended to make issues related to future technologies more concrete for respondents, it is possible that the topics discussed were still too abstract and that perspectives would have differed had participants had experience with particular devices or different communication disabilities. Fifth, the majority of participants in this study did not have experience using a brain-computer interface. While they were familiarized with this assistive technology through videos, they had not tried spelling with a BCI. Finally, the participants presented at different stages of neurodegenerative disease which may require use of a BCI in the future. No participant needed this new assistive technology at the time of the study.

Recommendations

Based on participant responses, the following recommendations should be considered as this new technology is further developed and made available to individuals with severe speech and physical impairments: (1) User-centred design should be broadened beyond preferences to also include user values and ethical perspectives. (2) BCI communication devices will affect how users experience and understand their privacy, identity, and relationships and BCI devices should be designed with these effects in mind. (3) BCI researchers should establish processes for gathering and incorporating stakeholder input into the design of devices. This does not represent an exhaustive list of ethical considerations but provides some important guideposts as this research goes forward.

Implications for Rehabilitation.

• Individuals with neurodegenerative diseases are important stakeholders to consider in development of natural language processing within brain-computer interface (BCI) communication technologies.

• The incorporation of personalized language models raises issues related to disability, identity, relationships, and privacy.

• People who may one day rely on BCI communication technologies care not just about usability of communication technology but about technology that supports their values and priorities.

• Qualitative ethics-focused research is a valuable tool for exploring stakeholder perspectives on new capabilities of BCI communication technologies, such as the storage and use of personalized language models.