Evaluation of a screening tool for the identification of neurological disorders in rural Uganda

Abstract

Background:

Neurological disorders are common in sub-Saharan African, but accurate neuroepidemiologic data are lacking from the region. We assessed a neuroepidemiological screening tool in a rural Ugandan cohort with high HIV prevalence.

Methods:

Participants were recruited from the Rakai Neurology Study in rural Rakai District, Uganda. A nurse administered the tool and a sociodemographic survey. 100 participants returned for validation examinations by a neurologist (validation cohort). The diagnostic utility and validity of the instrument were calculated and characteristics of those with and without neurological disorders compared.

Results:

The tool was administered to 392 participants, 48% female, 33% people with HIV, average age 35.1±8.5 years. 33% of the study cohort screened positive for neurologic disorders. These participants were older [mean (SD): 38.3 (9.7) vs. 33.5 (7.1) years, p<0.001], had a lower Karnofsky score [89.8 (8.4) vs. 93.9 (7.5), p<0.001] and had a lower body mass index [21.8 (3.3) vs. 22.8 (3.7), p=0.007] than those who screened negative. Amongst the validation cohort, 54% had a neurological abnormality of which 46% were symptomatic. The tool was 57% sensitive and 74% specific for detecting any neurological abnormality and 80% sensitive and 69% specific for symptomatic abnormalities.

Conclusions:

We found a lower sensitivity and similar specificity for the screening tool compared with two previous studies. The lower validity in this study was likely due in part to the high percentage of asymptomatic neurological abnormalities detected. This screening tool will require further refinement and cultural contextualization before it can be widely implemented across new populations.

Introduction

Neurological disorders are the leading cause of disability-adjusted life years and second leading cause of death globally. The bulk of this disease burden occurs in low- and middle-income countries and is expected to continue growing [1-5]. Little is known about the true prevalence of neurological disorders in sub-Saharan Africa (SSA) where neurological disorders are estimated to account for 2.9% of the total burden of disease, but there is marked uncertainty in these estimates [6]. In contrast, hospital-based studies have found the proportion of hospital admissions due to neurological disorders in SSA range from 7.5% - 25% [3-5, 7, 8]. Exacerbating this burden is the lack of adequate neurological services, with 34 of 53 African nations reporting four or fewer neurologists [9].

Two-phase neuroepidemiological studies are the gold standard for establishing disease burden in low-resourced settings because of their capacity for widespread population assessment while simultaneously optimizing resource utilization [10, 11]. The prototypical instrument for these studies was developed in 1981 by the World Health Organization (WHO) [12] and has since been widely utilized but found to have low specificity [13-17]. This poses a significant barrier to widespread implementation as large numbers of false positive cases may overwhelm a low-resourced healthcare system with few neurologists to assess identified cases. A modified screen developed by Bower, et al accurately identified participants with neurological disorders with improved sensitivity and specificity compared to the original instrument [18, 19]. The improved validity and breadth of this screen may be particularly useful in the detection of neurological disease in people with HIV (PWH), of whom neurological complications pose a significant source of morbidity [20].

In this study, we utilized this modified neuroepidemiologic screening tool in rural Uganda to evaluate the feasibility and validity of the tool in this setting.

Methods

Study participants

A subset of participants from the Rakai Neurology Study (RNS) were enrolled using convenience sampling. RNS participants were enrolled from the Rakai Community Cohort Study, an open 50-village cohort in the Rakai region of southcentral Uganda representative of rural Uganda, and HIV clinics in Rakai District. Enrollment occurred between July 2013 and July 2015. Participants were either antiretroviral (ART)-naïve PWH with advanced immunosuppression (CD4 ≤ 200 cells/μL); ART-naïve PWH with moderate immunosuppression (CD4 350-500 cells/μL); or HIV-uninfected adults who were age-, sex-, and community-matched to the PWH. Exclusion criteria included severe systemic illness, inability to provide informed consent, physical disability which prevented travel to the study site, or plans to leave the Rakai District within the two years of follow-up of the RNS.

Study procedures

The study was conducted using a two-phase design. During phase one, participants were administered a sociodemographic survey and the modified neuroepidemiological screen (Appendix) by a study nurse who received training on tool administration by a study neurologist (D.S.). This tool was developed in Tanzania, validated in Tanzania and Ethiopia, and designed to be administered by non-physician healthcare workers. It consisted of 24 yes/no questions regarding neurological symptoms and 17 simplified physical maneuvers meant to approximate portions of the neurological examination. Two symptom questions were meant to serve as stand-alone diagnostic questions for headache and low-back pain (LBP). The tool was translated from its original English into Luganda (the most commonly used local language in the study area) then back translated for quality control. Administration of the tool was performed in the participant’s preferred language. Peripheral blood draw was also performed in PWH for determination of CD4 cell count.

In phase two, 75 participants who screened positive (defined a ‘yes’ answer to any question or any abnormal examination finding) and 25 participants who screened negative (defined as a ‘no’ answer to all questions and normal examination findings) on the neuroepidemiological screening tool were randomly selected to undergo validation examinations consisting of a neurological history and examination performed by a study neurologist (D.S.). A written summary of each case was independently reviewed by a second neurologist (K.T.). Diagnoses were compared, and a consensus diagnosis reached where discrepancies existed.

Ethical and Institutional Approvals

This study was approved by the Western Institution Review Board, the Uganda Virus Research Institute Research and Ethics Committee, and the Uganda National Council for Science and Technology. All participants provided written informed consent for participation in this study.

Statistical analysis

In the overall study cohort, positive screens were defined as any abnormal finding on the examination component or any affirmative response on the questionnaire component excluding the headache and lower back pain (LBP) questions (Q14 and Q15) since these were meant to be stand-alone diagnostic questions without need for further specialist evaluation to confirm the diagnosis. Some questions were associated with clarifying sub-questions. Analysis was performed with and without the requirement that at least one sub-question was affirmative for the overall question to be classified as affirmative.

Participants in the validation cohort were divided into three groups according to the validation examination: normal (i.e. no neurological diagnosis), pain-only diagnosis (i.e. primary headache disorder, musculoskeletal LBP), and abnormal (i.e. any neurological diagnosis). The abnormal group were then subdivided into those with symptomatic and asymptomatic diagnoses. The asymptomatic diagnoses were primarily those with evidence of peripheral neuropathy on examination but no corresponding sensory complaints. Participants with pain-only diagnoses were classified as normal during statistical analyses [18]

Demographic characteristics are reported and compared between the overall and validation cohorts using chi-squared tests for categorical variables, t tests for continuous parametric variables, and Wilcoxon rank-sum tests for continuous non-parametric variables. The diagnostic utility of the screening tool, including sensitivity, specificity, area under the curve (AUC) of the receiver operator characteristic (ROC) curve, positive predictive value (PPV), negative predictive value (NPV), positive and negative likelihood ratios (+LR and −LR, respectively), and accuracy, were calculated separately for participants with any neurological abnormality and for participants with symptomatic neurological abnormality.

Analyses were performed using Stata 14.0 (College Station, Texas). A p-value < 0.05 was considered statistically significant.

Results

Demographic characteristics of the overall and validation cohorts are listed in Table 1. Participants in the overall cohort had an average age of 35.1 ± 8.4 years, 48% were female, and 33% were PWH. Participants in the validation cohort had an average age of 34.7 ± 8.4 years, 58% were female, and 27% were PWH. Compared to the validation cohort, the overall cohort had fewer PWH with moderate immunosuppression (3% vs. 8%, p=0.006) and more with advanced immunosuppression (31% vs. 19%, p=0.007).

Table 1.

Demographic characteristics of the overall study cohort and then of the sub-cohort of 100 participants who were also included in the validation cohort.

	Study Cohort (n = 392)	Validation Cohort (n = 100)	p
Female [n (%)]	188 (48%)	58 (58%)	0.07
Age (years) [mean (SD)]	35.08 (8.38)	34.73 (8.40)	0.71
Education (years) [mean (SD)]	5.84 (3.42)	5.53 (2.97)	0.41
Alcohol Use in Last 30 Days [n (%)]	178 (45%)	44 (44%)	0.8
Current Tobacco Use [n (%)]	50 (13%)	10 (10%)	0.45
HIV Positive [n (%)]	130 (33%)	27 (27%)	0.24
HIV Positive, CD4 Count 350-500 [n (%)]	10 (3%)	8 (8%)	0.006
CD4 Count [mean (SD)]	405.1 (39.32)	405 (44.08)	1
HIV Positive, CD4 Count <= 200 [n (%)]	120 (31%)	19 (19%)	0.007
CD4 Count [mean (SD)]	85.18 (61.91)	69.32 (54.97)	0.29
Karnofsky Score [mean (SD)]	92.5 (7.49)	92.8 (6.53)	0.71
BMI (kg/m^2) [mean (SD)]	22.49 (3.73)	22.57 (3.33)	0.84
BMI Classifications
Normal [n (%)]	270 (69%)	72 (72%)	0.62
Underweight [n (%)]	44 (11%)	7 (7%)
Overweight [n (%)]	58 (15%)	17 (17%)
Obese [n (%)]	20 (5%)	4 (4%)

Of the overall cohort, 97 (25%) participants reported abnormalities on the questionnaire when excluding headache and LBP questions, and 63 (16%) participants had abnormalities on the examination portion. Overall, 129 (33%) participants had a positive screen. Compared to those with a normal screen, participants with an abnormal screen were older [mean (SD): 38.3 (9.7) vs. 33.5 (7.1) years, p < 0.001], had a lower Karnofsky score [89.6 (8.4) vs. 93.9 (7.5), p < 0.001] and a lower body mass index (BMI) [21.8 (3.3) vs. 22.8 (3.7), p=0.007] (Table 2).

Table 2.

Comparison of demographic characteristics between those who had a normal versus abnormal neuroepidemiologic screening tool result in the overall study cohort and between those with and without neurological abnormalities identified in the validation cohort. Of note, those who answered yes to the questions about headache and low back pain on the neuroepidemiologic screening tool were classified as normal for this analysis if no other abnormality was identified.

	Study Cohort			Validation Cohort
	Normal Screen (n = 263)	Abnormal Screen (n = 129)	p	Normal Validation Exam (n = 46)	Abnormal Validation Exam (n = 54)	p
Female [n (%)]	121 (46%)	67 (52%)	0.27	27 (59%)	31 (57%)	0.9
Age (years) [mean (SD)]	33.48 (7.12)	38.33 (9.74)	< 0.001	33.13 (7.50)	36.09 (8.93)	0.08
Education (years) [mean (SD)]	-	-	-	5.74 (2.37)	5.35 (3.42)	0.51
Alcohol Use in Last 30 Days [n (%)]	120 (46%)	58 (45%)	0.9	21 (46%)	23 (43%)	0.76
Current Tobacco Use [n (%)]	28 (11%)	22 (17%)	0.07	4 (9%)	6 (11%)	0.69
HIV Positive [n (%)]	80 (30%)	50 (39%)	0.1	9 (20%)	18 (33%)	0.12
HIV Positive, CD4 Count 350-500 [n (%)]	6 (7.5%)	4 (8%)	0.92	4 (9%)	4 (7%)	-
HIV Positive, CD4 Count <= 200 [n (%)]	74 (92.5%)	46 (92%)	0.92	5 (11%)	14 (26%)	-
Karnofsky Score [mean (SD)]	93.9 (7.5)	89.6 (8.4)	< 0.001	93.7 (5.7)	92.0 (7.1)	0.21
BMI (kg/m^2) [mean (SD)]	22.8 (3.7)	21.8 (3.3)	0.007	22.5 (3.1)	22.6 (3.6)	0.84

Of the 100 participants in the validation cohort, neurological diagnoses were made in 54, of whom 25 (46%) were symptomatic. There were no differences in demographic characteristics between those with and without neurological diagnoses (Table 2). The most common neurological diagnoses were peripheral neuropathy (n=41), cognitive impairment (n=7), pseudodementia secondary to depression (n=4), and seizures (n=3).

The screening tool was 57.4% sensitive for detecting asymptomatic diagnoses and 80% sensitive for symptomatic diagnoses while specificity was 74% for asymptomatic and 69% for symptomatic cases (Table 3). When an affirmative response to a sub-question was required for the overall question to be considered abnormal [18], sensitivity greatly decreased (any: 33%; symptomatic: 48%) while the specificity increased (any: 83%; symptomatic: 81%). Overall, the instrument correctly classified 66% of the validation cohort when screening for any abnormalities and 71% for symptomatic disorders. The AUC ROC curve was greater for symptomatic than any abnormality at 0.747 and 0.657, respectively (Figure 1). When the validity of only the questionnaire component of the tool was assessed for detecting any neurological abnormality, sensitivity decreased (48%) and specificity improved (87%) while accuracy and AUC ROC remained largely unchanged. However, for symptomatic disorders, sensitivity (76%) and AUC ROC remained largely unchanged while specificity (83%) and overall accuracy markedly increased (79%).

Table 3.

Diagnostic utility of the complete neuroepidemiological screening tool and the questionnaire component of the tool (i.e. excluding the examination components) for detecting any neurological abnormality and symptomatic neurological abnormalities within the validation cohort. Each measure is presented with its 95% confidence interval except for the percent correctly classified.

	Any Abnormality	Symptomatic Abnormality	Questionnaire Only - Any Abnormality	Questionnaire Only - Symptomatic Abnormality
Sensitivity	57% (43%, 71%)	80% (59%, 93%)	48% (34%, 62%)	76% (55%, 91%)
Specificity	73.9% (59%, 93%)	69% (58%, 80%)	87% (74%, 95%)	83% (72%, 90.4%)
ROC Area	0.657 (0.564, 0.749)	0.747 (0.651, 0.842)	0.676 (0.592, 0.759)	0.793 (0.698, 0.889)
Positive Predictive Value	72% (56%, 85%)	46% (31%, 62%)	81% (64%, 93%)	59% (41%, 76%)
Negative Predictive Value	60% (46%, 72%)	91% (81%, 97%)	59% (46%, 71%)	91% (82%, 97%)
Positive Likelihood Ratio	2.2 (1.29, 3.77)	2.61 (1.76, 3.86)	3.69 (1.67, 8.18)	4.38 (2.55, 7.53)
Negative Likelihood Ratio	0.576 (0.404, 0.821)	0.288 (0.130, 0.641)	0.596 (0.451, 0.789)	0.290 (0.143, 0.588)
Correctly Classified [n (%)]	66 (66%)	71 (71%)	66 (66%)	79 (79%)

Figure 1.

Receiver operator characteristic (ROC) curves for the performance of the neuroepidemiology screening tool in detecting (A) any neurological abnormality and (B) symptomatic neurological abnormalities. We also evaluated the performance of the questionnaire portion of the screening tool (i.e. excluding the examination components) in detecting (C) any neurological abnormality and (D) symptomatic neurological abnormalities.

Individual questions with at least one abnormal response amongst the validation cohort are recorded in Table 4 with the proportion that were consistent with the final diagnosis made during the validation examination. Most (72%, n=38) participants who answered the headache question (Q14) affirmatively were confirmed to have a primary headache disorder. The question meant to detect cognitive impairment (Q19) was abnormal in 18 participants, but only 9 (50%) were diagnosed with any form of cognitive impairment. Of the 8 positive respondents who reported prior loss of consciousness (Q1), only 3 (38%) were diagnosed with a corresponding neurological disorder. Of six respondents reporting hospitalizations in the previous year, none were for a neurological cause. For the question regarding unusual movements (Q5), 3 positive responses were received, and all 3 respondents were diagnosed with relevant pathologies including simple partial seizures, benign fasciculations, and functional neurological disorder with psychogenic non-epileptic seizures. Whether participants exhibited the same abnormal finding on the screening exam items and the validation examination is detailed in Table 4. Testing extraocular movements (Ex9) was abnormal in 9 participants, but none exhibited similar abnormalities during the validation exam. Finger tapping (Ex12) was abnormal in six participants, 4 (67%) of whom exhibited similar abnormalities during validation exams.

Table 4.

Number of participants in the validation cohort who responded affirmatively to individual questions or had an abnormal finding on individual examination components of the neuroepidemiological screening tool and the percentage of those who had a related finding or diagnosis on the validation examination.

Question	Affirmative Responses (n)	Related Diagnoses [n (%)]
Q14) Do you get headaches?	53	38 (72%)
Q19) Has there been any deterioration of your memory within the last five years?	18	9 (50%)
Q1) Have you ever lost consciousness?	8	3 (38%)
Q17) In the last year, have you been admitted to the hospital?	6	0 (0%)
Q15) In the past year, have you ever had back pain that caused you to stay in bed all day instead of doing your normal daily activities?	5	3 (60%)
Q12) Do you hear well?	5	1 (20%)
Q5) Have you had other unusual movements lasting longer than a day?	3	3 (100%)
Q2) Have you ever had a time where you didn’t know where you were?	1	1 (100%)
Q9) Have you ever had a time when you couldn’t speak or couldn’t understand what people were saying to you?	1	1 (100%)
Q10) Have you had any change in your speech?	1	0 (0%)
Q11) Has your face or part of your face been paralyzed for more than a day?	1	0 (0%)
Q16) In the last year, have you had times of fever with loss of consciousness?	1	1 (100%)
Exam Component	Abnormal Finding (n)	Related Validation Exam Findings [n (%)]
Ex6) Follow my finger with your eyes.	9	0 (0%)
Ex12a) Tap your right thumb and index finger together rapidly.	6	4 (67%)
Ex12b) Tap your left thumb and index finger together rapidly.	6	4 (67%)
Ex5) Look at my nose. How many fingers am I holding up?	2	1 (50%)
Ex8) Smile widely for me.	2	0 (0%)
Ex14) Walk four meters.	1	0 (0%)

Discussion

This study implementing a previously validated neurological screening tool showed modest validity in detecting neurological abnormalities in a rural Ugandan cohort with a high HIV prevalence. This tool was developed by Bower, et al in 2009 in Moshi, Tanzania with a cohort of 78 participants through the adaptation of a previous WHO screening instrument and was found to have a sensitivity of 100% and specificity of 61% for identifying any neurological disorder [18]. In a follow-up study published in 2012, the tool’s validity was assessed in Butajira, Ethiopia (n = 519) and Hai District, Tanzania (n = 657). The sensitivity and specificity were 100% and 82.4%, respectively, while the questionnaire component alone had a sensitivity of 100% and specificity of 91.2%, although the differences were not statistically significant [19].

Participants in our study and the Bower studies were primarily from rural settings, but participants in the initial study in Tanzania were notably older (mid-40s) than participants in our study and the 2012 Bower study (mid-30’s) [18, 19]. The sensitivity of the instrument was lower in our study compared to both prior studies for detection of both symptomatic abnormalities and any abnormalities.

The higher sensitivity found in the Bower studies is likely due in part to differences in participant selection. In the prior studies, participants were chosen for the sensitivity arm based on neurological conditions previously identified during research studies or routine clinical care whereas participants in the validation arm of the current study were selected based on the results of the screening tool [18, 19]. Participants with known conditions may be more aware of the corresponding symptoms or have more severe symptoms that prompted them to seek medical care and, thus, more likely to screen positive. Lastly in the 2012 study, the sensitivity of the instrument was only calculated for participants with stroke, epilepsy, or Parkinson’s disease which does not allow for a direct comparison with our study [19].

Comparable studies utilizing the WHO instrument or its adaptations have reported sensitivities ranging from 84% to 100% and specificities that range from 80% to 99.9% [21]. The lower sensitivity in our study was likely due to the high prevalence of asymptomatic neurological disorders in our cohort. Of the 54 validated participants who were diagnosed with a neurological disease, less than half (46%) were symptomatic. Still, while the sensitivity improved for symptomatic disorders, it remained lower than previously reported studies.

The accuracy of the overall instrument in our cohort was lower than anticipated as well. Accuracy ranged from 0% to 72% for questions that received ≥3 affirmative responses. Likewise, examination components with ≥3 abnormal findings demonstrated accuracies ranging 0 to 67%. In contrast, questions and examination components corresponding to frequently diagnosed conditions during the validation examination, including peripheral neuropathy, received no affirmative responses or abnormal findings at all. Cultural subjectivity of illness is one possible explanation. For example, participants who reported memory problems (Q19) were often diagnosed with pseudodementia secondary to depression. Similar findings have been demonstrated with other adaptations of the WHO neuroepidemiologic screening tool. In an Indian study of 2,510 individuals, participants reporting memory deterioration within the last five years were likely to be children with poor scholastic performance or learning disabilities when the question was meant to identify objective cognitive impairment [22].

Because language and culture shape how patients and practitioners categorize and label diseases and symptoms, diagnostic tools often have cultural biases [23, 24], especially when questionnaires are translated into other languages resulting in alterations of the connotative meanings and missed nuances [25]. This phenomenon was evident in Q14. While designed as stand-alone diagnostic question for primary headache disorders, its accuracy was only 72%. Similar inaccuracies were noted with this question in a study of rural Mexican heads-of-household and were attributed to differing cultural interpretations of the question’s wording [26].

Peripheral neuropathy was the most prevalent neurological diagnosis in our validation cohort, detected in 41% of participants but symptomatic in only one-quarter of those. In a larger study conducted in the same Rakai Neurology Study, 33% of all participants had objective signs of peripheral neuropathy while 13% of participants had both signs and symptoms. [27]. The larger proportion in our validation cohort with both symptomatic and asymptomatic peripheral neuropathy is likely due to saturation with participants who had abnormalities on the screening tool. However, this study and others demonstrating high rates of peripheral neuropathy in PWH (11% - 44%) [28-35] and HIV-uninfected participants (0.2% - 12%) [31, 36-40] in SSA lends credence to the validity of our findings.

This study had several limitations, including the use of convenience sampling for participant recruitment and the small size of the validation cohort. Additionally, the training of research staff in screening tool administration spanned several hours as opposed to days or weeks as in prior comparable studies [18, 38, 39, 41]. However, this better imitates the constraints found in the real-world large-scale implementation of a screening instrument. Finally, the saturation of participants with abnormal screens in our validation cohort does not allow for disease prevalence to be extrapolated from the frequency of observed diagnoses in this study as it was not a representative sample of the community. Despite these limitations, this study was the first to assess the validity of the Bower adaptation of the WHO neurological screening tool in Uganda.

Although this instrument demonstrated modest validity in this pilot study, it would require further refinement before it would be feasible to widely implement it to characterize neurological disease burden in rural Uganda. The differing performance of this tool in our study compared to prior studies in Tanzania and Ethiopia demonstrates the importance of local validation and cultural contextualization of screening tools for neurological disorders.