Taking the Test: A Qualitative Analysis of Cultural and Contextual Factors Impacting Neuropsychological Assessment of Xhosa-Speaking South Africans

Abstract

Objective

There is an urgent need to make neuropsychological (NP) testing more acceptable, accessible, and culturally salient, particularly for culturally, educationally, and linguistically diverse individuals from countries who may have little-to-no experience with NP testing. In settings with limited resources such as South Africa, unique cultural and contextual factors (e.g., structural inequality, poverty) may impact the experience of NP evaluation. Research in this area is limited and requires further exploration. This qualitative study explores the role of cultural and contextual factors that may impact the experience of NP evaluation in a sample of Xhosa-speaking South African adults. Participant interviews explored the context from which individuals arrived at the NP assessment (e.g., quality of education, understanding of cognitive disorders), and their experience of completing NP tests.

Method

This qualitative study used data from semistructured interviews to conduct a thematic analysis exploring contextual factors and the experience of completing NP tests for the first time among Xhosa-speaking South African adults (N = 22). Results: Although no participants had prior experience with NP testing, most found testing procedures acceptable. Most participants, however, reported a limited understanding of the purpose of NP testing and cognitive problems. Additionally, some participants reported perceptions and attitudes that could affect test performance, such as misinterpreting standard testing procedures (e.g., no feedback from the examiner, being stopped mid-task) as indicative of poor performance.

Conclusions

This study provided much needed exploration into unique cultural factors that may impact the experience of NP assessment in South Africa, which could bias test performance and interpretation, and may aid the field of cross-cultural NP in better serving culturally and linguistically diverse populations. In these countries, neuropsychologists may need to actively evaluate participants’ understanding of NP testing to help foster optimal assessment conditions. They may also need to educate participants on possible causes of cognitive disorders.

Introduction

Globally, greater demands are being made to conduct neuropsychological (NP) testing for both clinical and research purposes with culturally, educationally, and linguistically diverse individuals from low- and middle-income countries (LMICs) who have little-to-no experience with NP testing (Ardila 2020; Fernandez 2019; Olson and Jacobson 2015; Watts and Shuttleworth-Edwards 2016). Although clinical neuropsychology training programs and professional governing bodies have made commitments to train culturally competent practitioners and to ensure best practices of NP testing with racially and ethnically diverse populations (American Educational Research Association 2014; Rivera Mindt et al. 2010), the NP literature pays little attention to these issues among individuals living in resource limited settings (Hilsabeck and Mindt 2020). Yet, many countries have recent or current histories replete with war, food insecurity, structural and institutionalized inequalities, extreme disparities in wealth, and limited access to resources, quality education, and healthcare services (Badat 2010; Grace 2015; Kruk et al. 2009; Schendel and McCowan 2016; Ward and Viner 2017; Williams et al. 2008)—conditions that may cause or exacerbate neurocognitive problems, as well as limit educational attainment and experience with standardized tests. In the context of NP testing, these experiences may subsequently impact examinee understanding, rapport building, and test performance. Moreover, many countries (e.g., LMICs) with these inequalities, especially those in sub-Saharan Africa, often have a high, or the highest, burden of brain involving diseases, such as HIV, creating a great need for NP testing (Kharsany and Karim 2016; Watts and Shuttleworth-Edwards 2016).

Although there is considerable research examining how sociocultural factors (e.g., quality of education, access to educational opportunities, test-wiseness, cultural salience of test stimuli, appropriateness of norms, and acculturation; Brickman et al. 2006; Manly and Echemendia 2007; Manly et al. 2004; Gutchess and Indeck 2009; Millman et al. 1965; Robbins et al. 2013; Hawkins and Bender 2002; Kempler et al. 1998; Fivush and Nelson 2004; Kashima and Kashima 1998; Tripathi et al. 2014; Zahodne et al. 2015, Ostrosky-Solis et al. 1998; Manly et al. 1999, 2002; Ferrett et al. 2014) can affect NP test performance and interpretation, there is a lack of research exploring how sociocultural and contextual factors may affect the experience of undergoing NP testing. A much smaller literature has focused on the experience of taking NP tests and how aspects of the NP test-taking experience may impact test performance (Gruters et al., 2020; Keady & Gilliard, 2002) and does not include research from LMIC populations. Such data may be especially pertinent to strengthening our approach to cross-cultural neuropsychology and optimizing NP testing among ethnically, linguistically, and educationally diverse people who may be unfamiliar with NP tests.

There is also little research on how individuals from LMICs understand cognitive disorders and the extent to which medical and scientific terms (e.g., diagnostic categories, symptoms, etiologies) are incorporated into individuals’ worldviews. A systematic review examining the understanding of dementia in LMICs and amongst indigenous peoples found that dementia was rarely conceptualized using medical terminology (e.g., pathological, progressive condition); rather it was conceptualized within the context of normal aging, mental illness, cultural beliefs, and trauma caused by colonization (Johnston et al. 2020). Cultural factors that may influence one’s perception and understanding of cognitive problems include the normalization of cognitive concerns (e.g., as a part of “normal aging”), stigma, religious/spiritual beliefs, access to health services, and the causes to which individuals attribute their health (Chin et al. 2011; Cipriani and Borin 2015; Furnham and Baguma 1999; La Fontaine et al. 2007; Mahoney et al. 2005; Sun et al. 2010).

Ardila (2007, 2020) synthesized much of the cross-cultural neuropsychology literature to derive five culturally related factors of NP testing that may affect how examinees experience testing and interact with the neuropsychologist, neuropsychologists understand or interpret behaviors during testing, and NP test performance is interpreted—factors that warrant consideration when testing culturally, educationally, and linguistically diverse examinees. Patterns of abilities refers to those abilities that NP tests measure, which are typically learned abilities that correspond with the test taker’s educational opportunities and sociocultural experiences. Cultural values refer to the culturally constructed models for ways of thinking, behaving, feeling, and cultural variations in cognitive tests scores (e.g., the one-on-one testing paradigm could violate cultural values for some). Familiarity refers to the participant’s familiarity with the testing situation and strategies necessary to perform tasks (e.g., test-wiseness), including familiarity with the strategies used in testing and the cultural salience of the testing environment and stimuli (e.g., certain test stimuli, such as images of animals, may lack salience in geographic regions where those animals do not exist and are not taught in educational curricula). Language refers to the differential impact of phonology, lexicon, grammar, syntactics, and pragmatics on test performance, its relationship with education, and equivalency in meaning across languages. Finally, education refers to socially reinforced attitudes and values that enhance the learning process and vary from one culture to the next, as well as education quality and educational attainment. For a more detailed exploration of these factors and related examples, see Ardila (2007, 2020). To date, no research has been conducted on which of Ardila’s five factors may be of more or less relevance to different LMIC populations.

South Africa, where the current study was conducted, is a LMIC (i.e., an upper middle-income country) with a sociopolitical history that has engrained racialized thinking into much of its citizenry. Beginning in the 17th century, South Africa experienced colonization by the Dutch, then the British in the 18th century (Oliver and Oliver 2017). There have also been strong Portuguese and French influences dating back to the 17th century (Correia and Verhoef 2009; Renou 2002). European cultural influences persist today (e.g., English is the primary language of the national government). Between 1948 and 1994, the authoritarian National Party government institutionalized the system of apartheid (i.e., legislated racial segregation in terms of housing, education, and social contact). At birth, every South African was racially classified as either White, Black, Colored (i.e., of mixed ancestry), or Indian/Asian. Economic protection for the minority White population was enshrined in various wide-ranging pieces of legislation, as was prohibition of mixing across these racial lines. Hence, where children classified as White had access to well-funded schools that emphasized aspiration toward tertiary education and professional careers, those classified as Black, Colored, or Indian/Asian were relegated to underfunded, impoverished schools that emphasized education for the unskilled labor market (Bunting 2006; Chisholm 2012; Jansen 2002; Ndimande 2009). In 1994, Nelson Mandela’s African National Congress defeated the National Party government in South Africa’s first democratic elections, effectively ending the apartheid-era policies.

Today, South Africa is a diverse country with rich ethnic and linguistic diversity (the government recognizes 11 official languages). Most South Africans are multilingual, learning two or more of the official languages. The three most widely spoken of South Africa’s 11 official languages in the household are isiZulu (24.7%), isiXhosa (15.6%) and Afrikaans (12.1%). Despite the end of Apartheid-era policies, tremendous structural inequalities persist, including race-based geographic and social organization (Adams et al. 2012; Leibbrandt et al. 2009; Leibbrandt et al. 2012; Lemanski 2017; Maharaj 2020; Newton and Schuermans 2013), disparities in quality of education (Watts and Shuttleworth-Edwards 2016), hunger and suboptimal housing conditions (e.g., lack of water, electricity; Ataguba et al. 2011).

In more recent decades, the field of neuropsychology has experienced significant growth in South Africa (Watts and Shuttleworth-Edwards 2016), with a growing body of research examining the impact of HIV on neurocognitive functioning. South Africa has the world’s largest population of people living with HIV (PLWH) at approximately 7.1 million (UNAIDS 2018) that predominantly affects Black communities—a result of apartheid and its legacy (poor education, poor infrastructure, poor access to healthcare; Phatlane 2003; Scrubb 2011). Consequently, South Africa also has a high burden of HIV-related cognitive impairment (23–76% of PLWH depending on experience with antiretroviral therapy; Joska et al. 2011, 2012). These high prevalence rates of HIV-related cognitive impairment indicate a great need for NP testing in research and clinical settings (Gouse et al. 2020). Yet, there is little access to NP testing or screening through the South African healthcare system among those communities most vulnerable to and most affected by HIV (Mabuto et al. 2019). Furthermore, it remains unclear how familiar individuals from these communities are with standardized NP tests and testing procedures (Chisholm and Wildeman 2013). Contrary to these concerns and limitations, many research studies have administered NP tests to samples from these populations (Nyamyaro et al. 2019).

Despite a limited literature examining cross-cultural factors that may impact NP test performance (Ardila 2007, 2020), there is a lack of research exploring which of these factors are relevant to NP testing in South Africa. To provide a preliminary exploration of these factors, a qualitative research approach was selected. Qualitative research has particular value for cross-cultural research, as it brings a deeper appreciation and understanding across cultures and may provide a voice to the worldviews and lived experiences of marginalized groups (Ponterotto 2010). The purpose of this qualitative study was to explore cultural and contextual (i.e., education, knowledge of cognitive problems, and familiarity with standardized tests) factors that may impact the experience of NP testing in a sample of Xhosa-speaking South Africans who underwent NP testing for the first time.

Materials and Methods

Study Design

This cross-sectional qualitative study used a guided, semistructured interview to examine cultural factors that may impact NP testing among Xhosa-speaking South African adults who recently underwent NP testing for the first time. The Xhosa-speaking people are indigenous South Africans who reside primarily in the Eastern Cape Province and are the second largest cultural group in South Africa (Mavundla et al. 2010). To better contextualize findings, the interview also explored their knowledge of common cognitive problems (e.g., forgetfulness), educational experience, and familiarity with standardized NP tests. Following Creswell (1998), we aimed for a sample size of between 17 and 25 for this novel phenomenological research.

Sample

Twenty-two adult participants who had agreed to be contacted for follow-up studies were purposively recruited from recently completed or ongoing studies on HIV and cognition that administered (by trained Xhosa-speaking NP testing technicians) a comprehensive NP test battery (Table 1) to HIV+ and HIV− Xhosa-speaking adults. Participants were originally recruited from general medical clinics in predominantly low-income, high-unemployment, and high-HIV prevalence communities in the Western Cape province of South Africa (see Joska et al. 2011). To be included in the parent study, participants had to: be naïve to Highly Active Antiretroviral Therapy and in a pretreatment phase of counseling; aged between 18 and 35 years; have a positive diagnosis for HIV within the last 6 months; and at least 7 years of formal education (see Joska et al. 2011, for complete details). In order to be included in the current study, participants from the parent study provided written agreement to be contacted by members of the research staff. There were no exclusion criteria for the current study.

Table 1.

Neuropsychological test battery used in Joska et al. (2011) parent study

Neuropsychological domain and tests
Attention/working memory
Mental alternation test
Mental control test
Processing speed
Trail making test, Part A
Color trails 1
WAIS-III digit symbol coding
Executive functioning
Trail making test, Part B
Color trails 2
Stroop Color Word test
Wisconsin Card Sorting Task-64 Item Version
Language
Category fluency, animals
Category fluency, fruit and vegetables
Learning and memory
Hopkins verbal learning test
Brief visuospatial memory test
Motor
Finger tapping
Grooved pegboard time (dominant and nondominant hand)

Participants were compensated for their time and travel expenses. This study was approved by the New York State Psychiatric Institute Institutional Review Board (IRB) and the University of Cape Town Human Research Ethics Committee (HREC). Informed consent was obtained for all participants.

Data Collection—Methods and Instruments

The research team developed an interview guide that consisted of semistructured, open-ended questions guided by Ardila’s (2007) framework (included as supplemental material). The interview guide also included questions about knowledge of cognitive difficulties (e.g., memory problems), educational experience (e.g., quality of education), and familiarity with standardized NP tests. The interview guide was not modified during the course of data collection. All interviews were conducted in the participant’s first language by Xhosa-speaking research staff trained in semistructured interviewing. Interviews lasted between 1 and 2 hours and were audio-recorded. All interviews were conducted between March 2009 and August 2010, with the majority taking place in a private research room at Groote Schuur Hospital’s Department of Psychiatry and Mental Health.

Data Processing and Analyses

Audio recordings of interviews were transcribed and translated from Xhosa into English by a fluent Xhosa-speaking study team member with experience conducting NP assessments and mental health interviews with Xhosa-speaking populations (TL). All interview transcripts were deidentified. After transcription, the interviewer reviewed transcripts for accuracy and added related contextual notes, including behavioral observations. For quality assurance purposes and translation accuracy, each transcript was reviewed by a supervisor (RNR) with extensive background in cross-cultural neuropsychology, and feedback was provided to the interviewer, as needed. The interviewer and supervisor discussed difficult-to-translate (from Xhosa to English, and English to Xhosa) phrases and terms to arrive at the most accurate translation as possible. Deidentified transcripts were uploaded to Dedoose (Version 8.2.14, 2019), a web-based application used for the management, analysis, and presentation of narrative data.

An inductive thematic analysis was conducted on data from the in-depth interviews examining the individual experience of Xhosa-speaking South African adults who recently underwent NP testing for the first time. Data were analyzed via an iterative approach, allowing subthemes to emerge from the interview data (Fereday & Muir-Cochrane, 2006). The process was iterative and inductive insofar as the interview data were gathered and organized to understand themes, rather than deductively derived hypotheses to test.

To complete this analysis, we selected the Framework Method. This is the most commonly used method for thematic analysis of semistructured interview data and represents a systematic method of organizing, coding, and analyzing qualitative data into an analytical framework (Gale et al. 2013). The Framework Method, which is described in further detail subsequently, has been utilized widely in health research (Ellis et al. 2012; Gale and Sultan 2013; Murtagh et al. 2006). This method provides concise, systematic steps that identify similarities and differences in the qualitative data to produce highly structured summary outputs (Gale et al. 2013; Smith and Firth 2011). First, two members of the research team (MA and AS) read the interview transcripts multiple times to familiarize themselves with the data. Second, the study team met to discuss emerging themes and develop a preliminary codebook, which is a list of labels (“codes”) that are applied to relevant sections of text in the interview transcripts. A third member of the research team (RNR), who was also familiar with the interview data, reviewed the preliminary codebook and provided feedback to enhance the trustworthiness of codes. The research team then refined the codebook based on each member’s feedback and agreed on the codes to apply to all transcripts (the “working analytical framework”). Third, two members of the research team (MA and AS) coded the transcripts in Dedoose by applying thematic codes to text segments of interviews. Each coder (MA and AS) coded half of the transcripts. During the coding phase, the research team reread and discussed the interviews, identifying response patterns and illustrative quotes. Both coders (MA and AS) independently extracted and coded text segments that mapped onto thematic domains. Coding discrepancies were addressed through collaborative discussion and consensus, and refinements were made to the codebook throughout the process to ensure that codes most accurately and parsimoniously captured the data. As a reliability check, both coders (MA and AS) applied codes to transcripts and the codes were compared, with a coding agreement threshold of 90%. After initially coding each transcript, the coders met and collaboratively coded two transcripts, one of which was initially coded by MA and one of which was initially coded by AS. Coding was discussed extensively until the final selected codes were unanimously agreed upon. One coder reviewed each transcript and updated final codes to ensure that codes were applied uniformly. To facilitate interpretation, data were charted into a framework matrix, and responses were tabulated and tallied to identify the most salient findings (Gale et al. 2013).

Results

Participants’ demographic information is presented in Table 2. They ranged in age from 19 to 46 years (Mdn_age = 31, IQR = 6.50). About two-thirds of the sample was female and half reported receiving less than 10 years of education. The average time elapsed between NP testing and the qualitative interview was approximately 3.5 months (M = 107.10 ± 3.51 days). More than half the sample (59%) completed the interview within 3 months of completing the NP testing.

Table 2.

Sample demographic characteristics (N = 22)

	n	%
Age, Med (IQR)	22	31.00 (6.50)
Years of education	22	10.50 (3.00)
≤10 years	11	50%
>10 years	11	50%
Gender	22
Male	7	32%
Female	15	68%
HIV status	22
HIV+	15	68%
HIV-	7	32%

Notes: Med = median, IQR = interquartile range.

A total of 16 themes emerged related to the experience of completing NP testing. These themes are discussed below and organized within Ardila’s (2007) five factors (i.e., patterns of abilities, cultural values, familiarity, language, education). Table 3 includes illustrativey quotes from participant interviews.

Table 3.

Participant quotes related to experience of completing NP testing

Ardila’s domain	Theme	Quote
Patterns of abilities	Unfamiliar test, familiar ability	“I’m not very familiar with them, but I am a bit… when using public transportation, you have to do some calculations, or if you went to a shop and get short changed.”
		“I am familiar with some of the material… those cards are similar to cards used in card games. And the blocks remind me of a dice game.”
Cultural values	Positive attitude toward examiner	“Once she [the examiner] started administering the tests I discovered that there was no reason to cause me to be nervous. The test giver was friendly”
		“I was nervous. But she [the examiner] told me not to fear anything. She kept me comfortable.”
	Internal/subjective issues	“No it’s not personal [questions about cognitive problems]. I’m not the only one who is interviewed here.”
		“I did find myself wondering why I was being asked these kinds of questions [about memory and cognitive functions]. But I am not worried or upset.”
Familiarity	Pretesting expectations	“I did not know all the details. But I had been told that I was going to work using my hands, my mind, and thinking.”
		“I was under the impression I’ll be learning about certain illnesses, like HIV and sexually transmitted infections… how a person should behave or conduct themselves … what one should do when a person is perhaps HIV+.”
		“I thought that maybe I would have to have my blood drained. Hence, I asked whether there would be pain to be endured… Or whether I would have to donate blood…I had earlier been concerned that I may end up unhappy at the end of the day.”
	Understanding the examiner’s role	“I thought you were the main researcher, testing my brain.”
		“I thought you were a helper.”
		“I do not know whether you are doing the research for yourself or on behalf of someone… nevertheless, I did assume you are the main researcher.”
	Understanding of cognitive problems	“Change in personal circumstances can probably bring about memory loss. If you had a good life living with your parents and then suddenly, have to fend for yourself, without a job, then I think that contribute to forgetfulness.”
		“It is the heavy stress, pressure, domestic problems, broken homes…these lead to problems even at school. And that is how a child starts experiencing memory loss.”
		“As far as I’m concerned, how a person remembers is never noticed or given attention to.”
	Nervousness	“…Of being wrong. That I might not do what’s expected of me.”
		“…Since I did not know what I would be required to do. However, as we continued playing, I realized it was fun. So, I told myself to relax because these are just games. It is nothing serious.”
	Post-testing takeaways	“You know, sometimes I am forgetful. Therefore, this sort of reminded me that I must not be forgetful.… If something had been dictated to me and in the end … even if other things were discussed after the list of words, if I can still remember those from earlier on, then that is good.”
		“I realized that I know a lot. And that my mind is still working fine. In addition, it also trained me not to be forgetful.”
		“I had the feeling after the visit…that I can look after myself. That I’ll take care of my health…During my visit, we worked and worked. But I now had interest in taking ARVs by the time I left…I felt that my life is important.”
	Reactions to standardized testing procedures	“I often think I’m not fast enough or I am not good.”
		“I assumed we ran out of time.”
		“When that happens, I feel worried [when told to stop by examiner]”
	Speed	“When you are doing something for the first time and somebody tells you to be quick, that is not good…You begin in a shaky state.”
		“It is difficult to do something accurately when it is your first time doing it. Therefore, it would be better if you showed me or lead me.”
		“If you do things fast you sometimes get them wrong. If you are being put under pressure, time wise, you could fail the mark. Especially when you consider that you are not that good at doing the required activities, it can cause you to panic. So, if you say I should work fast and accurately, then that means something will go wrong. Or, it will not be accurate… Somewhere somehow, I will screw something up.”
	Purpose of NP testing	“I have occasionally heard that in HIV+ people, the brain also gets affected. That the mind becomes “lazy” and does not grasp well… Maybe she was trying to understand whether it still functions properly!”
		“They are just trying to see/understand my giftedness, how fast I was, whether I still remember the things they had read out to me.”
	Familiarity with animals	“Those [Animals test] were easy because I watch wildlife documentaries.”
		“I only see them on TV.”
	Familiarity with strategies needed for task performance	“I just tell myself to try and do it within the allocated time, even when I do not know the limit. I would be anxious that I might not be able to finish in time. Therefore, I try to push harder.”
		“I think I should do as much as I can as this might be of benefit to someone else”
Language	Acceptability of language	“There were no really difficult words as we carried on.”
		“I understood them [the translations of the tests]. They were in familiar Xhosa.”
Education	Perceptions about the value of education	“School imparted to me the ability to write, to read, and respect.”
		“Education is the key. It is valued. I do not know which subjects are more important than others, but I believe every child should go to school.”
		“There is a disadvantage for someone who attended our schools when looking for a job, due to poor English skills. However, in terms of personal character, I would recommend our rural schools…they can mold learners into better people unlike the schools in the townships or cities.”
		“I got sufficient education. I just could not go further for I did not have the means.”

Notes: NP = neuropsychological, ARV = antiretroviral medications.

Patterns of Abilities

Unfamiliar test, familiar ability

Participants were asked a series of questions about their familiarity with the tests administered during their NP assessment. All reported being unfamiliar with the specific tests administered. Seven participants stated that they had been asked to “make drawings” and complete arithmetic before. Three reported that the tasks on the tests reminded them of familiar puzzles and games they played in their everyday lives and likened the NP tests to word games on television and to gambling. One participant stated, “The one where I was asked to say all the names of the animals… We used to do that. It reminded me of my school days.” (RR11, Female, HIV−, 31-year old).

Participants were also asked how often in their everyday lives they perform tasks similar to those performed during the NP testing, such as drawing, performing mental arithmetic, recalling spans of numbers and word lists, and describing how two items or ideas are alike. Half of the sample (n = 11) reported that they did not engage in any form of drawing in their everyday lives. Seven participants reported that they sometimes engage in activities similar to drawing, referencing helping their children with homework. Ten participants reported that they did not use arithmetic or calculations in their daily lives. Six participants reported performing arithmetic regularly in assisting their children with homework, counting money, and solving mathematical equations. Half the sample (n = 11) reported having to recall both number and word lists regularly, primarily memorizing phone numbers and identification numbers (n = 8), grocery lists (n = 5), and bible excerpts (n = 3). Four participants reported that they rarely memorized number or word lists in their daily lives. More than half of participants (n = 13) reported that they rarely compared items to one another or discussed how two objects are alike. Three participants reported that the first time they were asked to discuss similarities between two items was during the assessment, with one stating, “I never speak in such terms” (RR10, Male, HIV+, 36-year old).

Experience of NP Testing

Positive attitude toward examiner

When participants were asked about their reaction to the one-to-one relationship with the test examiner, most (n = 15) reported feeling comfortable and enjoying the interaction. In the words of one participant, she found the relationship “mutually friendly… a comfortable relationship” and viewed the examiner as a “helper” or “teacher” (RR13, Female, HIV+, 43-year old). Another participant described enjoying the testing relationship “because there was cooperation”. One participant stated:

Before I got here, I expected it to be just a day of work, work, and more work. However, she [the examiner] would stop once in a while to chat with me and try to make acquaintance… Then later she would say, ‘it is time to start’… Therefore, we would joke, laugh, and then be well composed when doing the tests. So, you would feel free because you can share jokes with this person. She is not constantly stern. (RR11, Female, HIV-, 31 y.o.)

Most participants (n = 19) did not feel unduly influenced or coerced by the examiner to complete tasks. However, two participants described inherent examiner–examinee power dynamics, one stating, “You [the examiner] are the one asking me questions, testing my mind. Therefore, that puts us on different levels” (RR54, Male, HIV+, 45-year old). Another participant stated, “I assumed I should do it. I wouldn’t be able to succeed if I didn’t do what you [the examiner] instructed me to do” (RR21, Female, HIV+, 30-year old).

Twenty participants reported feeling comfortable being in an isolated environment with the examiner. No participants reported perceiving being asked to be in a private room with the examiner, a stranger, as inappropriate. Participants likened the testing environment to similar situations they had experienced, such as taking a school test or attending a job interview. When asked directly if they would have preferred to have someone they knew in the testing room with them, most participants (n = 21) reported indifference, and only one participant reported that they would have preferred someone familiar in the room with them.

Participants did not report any problems with the formal language used by the examiner during the testing session. No participant reported feeling uncomfortable with the detached, professional nature of their relationship with the examiner. One participant reported that they expected this type of deportment, stating “I thought it was not appropriate for us to have a close relationship” (RR01, Male, HIV−, 25-year old). Overall, participants reported they were able to establish an effective working relationship with the examiner. Four participants, in particular, reported establishing good rapport, feeling calm and comforted by their interactions with the examiner. One participant described positive interactions with the examiner, reporting, “She [the examiner] was good. She did not get irritated. She would explain how to do the activities. And if you do not understand, she would explain it again” (RR11, Female, HIV−, 31-year old).

Sensitive topics/privacy concerns

Participants were asked whether they perceived the questions asked by the examiner as sensitive or private. Most participants (n = 16) reported they did not view the discussion of medical history as overly personal or as violating their privacy. Participants viewed the testing experience as an opportunity to “use the mind” and “know more about your current condition.” Two participants reported the topics discussed were of a sensitive nature; however, they also reported that they felt comfortable discussing such matters under the assurance of confidentiality. One of these participants stated, “Yes, it is personal. However, although you understand that it’s personal, you also discern that discussing those things could benefit you” (RR01, Male, HIV−, 25-year old).

Familiarity with NP testing and Cognitive Problems

Pretesting expectations

After enrolling in the parent studies, participants were asked about their expectations of the NP testing component. A few (n = 5) noted that they had no knowledge about the specific content of that study visit; three of those participants noted that they believed the visit would involve completing bloodwork. Four participants reported that they were expecting to complete “mind” and “knowledge” tests.

Understanding of cognitive problems

More than half the sample (n = 15) reported having heard about memory problems; none were familiar with the term “dementia.” When asked about causes of forgetfulness, participants provided a number of responses, including stress (n = 5), excessive worrying (n = 4), old age (n = 2), depression (n = 1), and witchcraft (n = 1). One participant suggested that forgetfulness and memory problems might be the result of problems with the brain.

Participants were also asked to provide Xhosa terms for individuals who experience forgetfulness or memory problems. Some participants (n = 6) reported being unaware of any such term in Xhosa. Others provided responses such as “fool/idiot” (“izidenge”; n = 3), “stutterer” (“lithintitha”; n = 2), and “mindless/crazy” (“ligeza”; n = 2). When asked to provide a Xhosa word for someone who has difficulty making decisions, participants described these individuals as “liars/dishonest” (“ixoxi”; n = 5), “stupid person” (“Isiphukuphuku”; n = 2), and “cowards” (“igwala”; n = 1).

Understanding the examiner’s role

Although a small number of participants (n = 5) erroneously identified the examiner as the lead researcher, most (n = 13) reported understanding that the examiner was not in charge of the overall project.

Nervousness

About half the sample (n = 10) reported feelings of nervousness related to testing. Nervousness was attributed to uncertainty regarding what to expect during the testing visit or concerns about appearing “stupid” in front of the examiner. A few participants (n = 3) stated that although they felt nervous at the start of testing, it subsided quickly. Others (n = 7) denied ever feeling nervous.

Post-testing takeaways

When asked about their post-testing takeaways, seven participants described feeling positively about and motivated by completing the NP testing. Generally, participants viewed testing as valuable and as “helping” them improve their current condition. One participant reported feeling motivated to begin taking antiretroviral medications after testing, whereas others described feeling motivated to help their children with schoolwork or work toward improving their memory.

Reactions to standardized testing procedures

Participants were asked about their reaction to a number of standard NP test administration procedures, such as being stopped mid-task and not receiving feedback on their test performance. Eleven participants reported not being bothered when stopped mid-task. Six specifically reported that they were stopped because time had run out. Three reported that they believed they were stopped due to their poor performance or slow speed. One stated, “I must be stupid not to finish the task within the expected time limit” (RR11, Female, HIV−, 31-year old). Participants also reported anxiety (n = 2) and disruption (n = 1) when asked to stop working mid-task. One reported, “Sometimes I felt I was getting into the swing of things, just when I was starting to feel good about it… Then suddenly they [the examiner] say, ‘Stop’” (RR47, Male, HIV+, 31-year old). Another participant explained, “At times, I was shocked. I was starting to get it right and now being told to stop” (RR47, HIV+, Male, 31-year old).

When participants were asked about not receiving feedback from the examiner, more than half did not find this bothersome (n = 15), reporting that they believed the examiner was simply doing their job and that they “weren’t here to be given answers” (RR45, Female, HIV+, 30-year old). In contrast, two participants felt “let down” about not receiving feedback; one said that he “didn’t understand why I wasn’t being told that ‘this is where you’re wrong’” (RR09, Male, HIV−, 31-year old).

Speed

When asked about being asked to perform tasks both quickly and accurately, 12 participants found this acceptable and likened the request to instructions they had been given when performing tasks in school. One participant stated, “At school, you’d be expected to answer a certain question and be told ‘times up,’ so it wasn’t something new” (RR13, Female, HIV+, 43-year old). In contrast, seven participants reported that the instruction to perform tasks both quickly and accurately was contradictory. These participants reported difficulty working both quickly and accurately on tasks that they were performing for the first time and reported concerns that working quickly would result in increased errors.

Purpose of NP testing

Eleven participants believed that the purpose of the NP testing was to “check” their brain and its functionality or “look into the mind.” Six others believed that the purpose of testing was to assess specific cognitive abilities, such as speed and memory. Three reported that the purpose of testing was to enhance cognitive abilities, “enlighten,” and “keep the mind bright.” Two participants reported that the purpose of testing was specifically to assess whether HIV had affected their brain compared to people without HIV.

One participant described the tests, saying, “They are just games. It is nothing serious” (RR45, Female, HIV+, 30-year old). Referring to the tests as “games” or “exercises” (as was done in the parent studies) was not confusing for most participants. Only one participant was initially confused by this terminology, as she understood the word “game” to describe “gambling” (RR22, Female, HIV+, 26-year old).

Familiarity with animals

Participants were asked whether they were familiar with the pictured animals that they were asked to name during the testing session (i.e., lion, tiger, camel). Five participants reported seeing these animals on television; two reported seeing them in real life.

Familiarity with strategies and attitudes needed for task performance

All participants reported that they put forth their best effort on NP tests, and most (n = 19) reported placing importance on performing to the best of their abilities. No participant reported issues with being asked to do their “best,” with some participants finding this instruction both helpful and encouraging: “I want to give good responses so that you may be satisfied that I’m giving my best” (RR10, Male, HIV+, 36-year old). One participant reported feeling altruistically motivated to perform his best to benefit the research study or others, stating, “I think I should do as much as I can as this might be of benefit to someone else” (RR36, Female, HIV−, 39-year old). One participant shared:

I had to do drawings. I am not good at those, yet I just tried all I could. There were some block shapes I had to copy. They were difficult, yet I told myself to try my best. I think the most important is actually making the design. So, I tried to copy it as well as I could. (RR11, Female, HIV-, 31 y.o.).

Language

Acceptability of language

All participants reported understanding the language used and instructions provided by the examiner. In addition to Xhosa, most participants (n = 19) reported speaking and understanding English, whereas some participants reported speaking other languages such as Sesotho (n = 6), Zulu (n = 3), and Afrikaans (n = 5).

Education

Differing perceptions of educational quality

All participants reported attending rural schools for their primary education. Half of the sample (n = 11) reported believing that they had received an adequate education and that they had learned skills in school that they continued to use in their daily lives (e.g., reading, writing, communication, respect for others). In the words of one participant, “Yeah, it [education] is good… because I can write, I can speak, I can keep a job… [education taught me] respect… how to address someone… how to speak to an older person” (RR19, Female, HIV+, 35-year old).

Five participants reported believing their education was inadequate. As one participant described:

Now that I’ve grown up, I realize that everything requires one to be educated… I realize my little education is as good as that of someone who never went to school… When I was in standard 6 [grade 4], we were being asked to draw things like frogs… But nowadays, a standard 6 [student] is already doing career-orientated subjects and thus is able to later get a job aligned with what they learned at school… The education I received is bad compared to what I notice nowadays. (RR10, Male, HIV+, 36 y.o.)

Two participants described attending schools lacking electricity, toilets, and running water. One participant reported that transportation to better schools in their township was scarce and that they lacked proper computer access. She explained, “These days they do a lot of things, such as computers in high school… A lot of what is available nowadays was not available in our time. In those days, we did not find anything wrong, because they were not around. It did not seem strange” (RR20, Female, HIV+, 35-year old). A few participants (n = 5) discussed their frustration about receiving limited formal instruction in the English language, which precluded them from gaining English-language skills at a younger age. They reported that this lack of formal English language instruction was not the case in “White, suburban schools” and was a disadvantage for “someone who attended our [rural] schools when looking for a job, due to poor English skills” (RR11, Female, HIV−, 31-year old).

Valued skills

Participants were asked whether skills such as reading, writing, math, word knowledge, and problem solving were valued in their lives. Seven reported that these skills are highly valued, stating that individuals with education “possess some of these skills and will receive more respect than the person who does not possess them. Because they are more useful” (RR11, Female, HIV-, 31-year old). Some participants (n = 5) explicitly reported valuing education above all other skills, whereas others noted valuing other skills, such as independence, handiwork, and respect. When asked about pursuing further education, participants reported enrolling in courses for security certification (n = 3), computer certification (n = 2), business administration (n = 2), and sewing (n = 2).

Discussion

This study is the first to examine the experience of and perspectives toward NP testing among a group of Xhosa-speaking, South African adults who underwent NP testing for the first time. Though NP testing was a novel experience for all participants and required them to perform tasks that were (for the most part) unfamiliar to them, most found the testing acceptable and half the sample were aware that the purpose of NP testing was to “check” the status of their brain. Participants’ unfamiliarity with some task demands and procedures (e.g., being asked to stop mid-task), and the fact that many of the tasks (e.g., drawing, comparing two objects, remembering numbers or word lists) were not performed in their everyday lives, raises questions about the ecological validity of these tests and hence warrants further research.

In this sample of South African adults, responses related to familiarity with the NP testing procedure provide important insights into the examinee experience. Some participants reported a lack of familiarity with or limited use of commonly utilized NP test activities, including using arithmetic, comparing two items, completing drawings, or remembering word or number lists. These findings align with previous research conducted with Xhosa-speaking South Africans that found significant floor effects on complex drawing tasks (e.g., cube drawing) and confrontation naming tasks (e.g., animal naming), concluding that certain test items may be inappropriate for use with this population (Robbins et al. 2013). According to the International Test Commission’s (ITC; 2019) guidelines for testing linguistically diverse populations, examiners should not assume all participants have previous experience with specific task types and should consider evaluating their familiarity with item formats in order to better ascertain whether they are appropriate for the given population (ITC 1.8 2019). ITC guidelines also state that tests should be selected based upon their suitability for test purpose, while considering the background characteristics of the target population (ITC 5.16 2019). Therefore, examiners may consider paying greater attention and consideration to evaluating which test items are appropriate based on examinee educational and cultural backgrounds.

The Standards for Educational and Psychological Testing (Standard 3, American Educational Research Association 2014) also highlight the importance of fairness in testing. This entails the fair and equitable treatment of test takers while being tested, fairness in measurement quality (e.g., bias, access to the constructs being measured) and validity in test interpretation. Testing procedures that are not tailored for new populations threaten to violate fairness. For example, on timed tests, such as those of processing speed, participants in the current study reported that the instruction to perform tasks both quickly and accurately was contradictory, believing that working quickly would result in increased errors. Instructions stated this way without further explanation may create an unintended conflict or measurement bias where participants decide to sacrifice speed over accuracy or vice versa. Timed tests of processing speed typically are trying to assess how accurately an individual can process information quickly (Lezak et al. 2012)—not whether an individual has consciously made a decision to sacrifice one over the other. With further explanation (e.g., reassurance that it is normal to make mistakes), there might be greater confidence that the construct being measured is being measured accurately, which would result in a more accurate interpretation. Further research needs to evaluate how well test takers from different backgrounds and countries, especially those with limited experience taking standardized tests, actually understand test instructions and stimuli to ensure that the testing procedures and instruments are fair.

This study highlighted important contextual factors that may warrant additional research in South Africa. In regard to educational experiences and valued skills, all participants reported attending rural schools, some lacking restrooms, electricity, and running water. There was some disagreement among participants about the quality of education they received at these schools. Some described feeling disadvantaged and were dissatisfied with having received what they perceived as an inferior education compared to those in urban areas or of more advantaged backgrounds (presumably those with the financial means to attend better-equipped public school). Others believed their education was on par with schooling in the cities. Notably, a number of those who felt their education was of poor quality still stated that they highly valued education and the skills learned in school, such as reading, writing, math, and problem solving. These findings highlight the continued structural inequities that persist in South Africa and underscore the role that disparate quality of education may have on NP test performance.

Furthermore, results from this small sample suggest that knowledge and awareness about medically defined cognitive symptoms and brain pathology is variable. Though specific Xhosa terms for forgetfulness or memory problems were not identified by any participants, a few provided alternate terms that translated to “fool,” “idiot,” and “mindless.” Additional research needs to examine if use of these terms indicates stigmatizing beliefs surrounding these types of symptoms, which may represent barriers to seeking help. Previous research examining help seeking for memory problems among British Black African and Caribbean communities found that experiences of stigma and shame around memory problems and distrust toward the healthcare were common barriers to seeking help (Berwald et al. 2016). Among participants in the present study, some believed memory problems resulted from stress or excessive worrying. One participant responded that memory problems could be related to problems with the brain. Another participant reported that memory problems could be caused by witchcraft. The attribution of memory problems to a number of different causes may differ significantly cross-culturally and may be dependent on the causes to which individuals attribute their health (e.g., health locus of control).

Our findings suggest that in resource limited settings like South Africa, neuropsychologists must not only take great care to choose culturally salient and appropriate tests, but also take an active role when assessing a participant’s understanding of NP testing, both during the consent process and when introducing individual tests. Residents in South Africa may be faced with unique challenges (e.g., systemic racism, socioeconomic disparities, and limited access to quality education and healthcare services) that should be considered when contextualizing NP test performance and response to the NP testing procedures. Consideration of both what occurs in the room between examiner and examinee, and what experiences examinees bring into the room that may impact NP test performance are important data points that can enhance our understanding of NP test results. Neuropsychology researchers should also actively assess the ecological validity of tests in culturally and linguistically diverse communities globally, as particular instructions and task demands may be unfamiliar within some contexts. Further, neuropsychologists may need to provide more explicit explanations and assurance to the participant that particular testing procedures (e.g., being asked to stop working mid-task) are standard for everyone and not indicative that they are performing poorly. In efforts to make NP testing more acceptable, it may be particularly important to clarify to patients/clients what to expect during testing sessions, to correct preconceived misunderstandings, and to reduce feelings of apprehension or nervousness as much as possible. Furthermore, neuropsychologists and other allied health professionals should collaboratively develop public health interventions and programs to enhance the understanding of cognitive disorders and their signs and symptoms within these communities.

The current study has a number of notable strengths. This study serves as a first step in beginning to understand the role of cultural factors that may impact NP assessment, particularly in South Africa—a topic that remains neglected in the field and deserves significantly more research attention. Additionally, the use of a rigorous coding scheme and theoretical framework provided a rich, nuanced analysis of participant experiences and quotations. As the first study to examine this topic, our interview guide may serve as a foundation for the development of future methods and structured interviews that can examine these issues with more depth and rigor. Though the present study provides valuable information regarding the experience of NP testing in an underserved South African population, it also has several limitations. This qualitative analysis is based on a selective sample of Xhosa-speaking South Africans and cannot be considered transferable to the broader adult South African population. The participants were volunteers for the parent study, which may suggest that they have positive attitudes toward research and the requirements associated with participation in research. Therefore, attitudes and perceptions of this sample may differ from a Xhosa referred for testing by a physician. Moreover, the interviews were conducted at varying time points after completing the parent studies, which may have resulted in recall bias, with some participants having a better memory of their experience of the NP testing session than others. This issue is particularly important in a sample that includes PLWH due to the greater prevalence of memory deficits among this population. Additionally, due to the in-depth interview format and time spent with the participant, there is the possibility of social desirability effects affecting participants’ responses (Perinelli and Gremigni 2016). For example, one participant stated they wanted to provide “good responses” so the examiner would be “satisfied” (RR10, Male, HIV+, 36-year old). That said, the fact that participants did share critical views of certain testing procedures, such as being asked to stop working mid-task and being asked to respond both quickly and accurately, lends confidence related to the candidness of participants’ responses.

Improving our understanding of the NP testing experience from the participant’s perspective is increasingly important as the world is becoming rapidly more interconnected and NP testing is becoming accessible to certain populations for the first time. Further investigation into the NP testing experience is needed. Further studies examining the NP testing experience should explore gender and both inter- and intracultural variability within South Africa. Future studies should also examine these experiences in larger samples and diverse populations and should assess how these experiences are associated with NP test performance. Furthermore, although this study focused on individuals living in South Africa, an upper middle-income country, future research should also examine the context of NP testing among individuals in LMIC countries, non-Western countries, and high-income countries that historically or currently experience systematic racism and major socioeconomic disparities. Although it may be true that racially and educationally diverse individuals from high-income countries may have more experience and familiarity with standardized tests (e.g., cognitive assessments) than individuals from resource limited settings, research has demonstrated that ethnically diverse and stigmatized individuals in high-income countries may also perform differently on standardized tests depending on how the tests are presented to them (Krendl et al. 2008; Nguyen and Ryan 2008; Thames et al. 2013).

Funding

Supported by grants from the National Institute of Mental Health: (1) T32 MH19139 Behavioral Sciences Research in HIV Infection (PI: T. Sandfort, PhD); (2) R25MH080663, the Mount Sinai Institute for NeuroAIDS Disparities (PI: S. Morgello, Scholar Grant to RNR); (3) P30 MH43520, HIV Center for Clinical and Behavioral Studies at NY State Psychiatric Institute and Columbia University (PI: R. Remien, PhD).

Conflicts of Interest

None declared.