Sensitivity and Specificity of Stroke Symptom Questions to Detect Stroke or Transient Ischemic Attack

Abstract

Background/Aims

Undiagnosed stroke is a major public health problem. The Questionnaire for Verifying Stroke-Free Status (QVSS) includes eight items and was originally designed to detect stroke-free individuals. Its six symptom-related questions could potentially be used to screen for undiagnosed stroke or transient ischemic attack (TIA), but the sensitivity and specificity of just the six symptom-related questions are unknown.

Methods

A research assistant administered the QVSS to outpatients from Veterans Administration stroke and general medicine clinics. Neurologists, blinded to QVSS scores, interviewed and examined all subjects to determine stroke status. Responses to the six symptom questions of the QVSS were compared against the neurologist-determined stroke/TIA status.

Results

The sensitivity of the individual symptom questions ranged from 0.22 to 0.60, and the specificity ranged from 0.79 to 0.95. The sensitivity of any of the six symptom questions was 0.82, and the specificity was 0.62.

Conclusion

The six symptom-related questions of the QVSS demonstrate a high sensitivity and moderate specificity for the diagnosis of stroke or TIA compared with neurological exam. Though these findings should be validated in a more representative general population, these questions have potential for meeting the public health objective of detecting clinically unrecognized but symptomatic stroke.

Background

Strokes are a major cause of morbidity and mortality in the US, yet not all strokes are reliably detected. Large epidemiologic studies such as the Reasons for Geographic and Racial Differences in Stroke (REGARDS) and the Atherosclerosis Risk in Communities (ARIC) have shown that a large percentage of persons without prior diagnosis of stroke report stroke symptoms. These individuals also have decrements in physical and mental functioning and are more likely to experience an incident stroke, suggesting that many may have undiagnosed strokes and ongoing cerebrovascular disease [1,2,3]. Reliably identifying these individuals becomes an important public health objective.

Several surveys to assess a patient's stroke status do exist [1,4,5,6,7,8,9,10,11]. However, all include questions about a known previous diagnosis of stroke or transient ischemic attack (TIA), limiting their usefulness in detecting clinically unrecognized stroke. The Questionnaire for Verifying Stroke-Free Status (QVSS; table 1), an 8-item structured questionnaire originally designed to identify study subjects who are free of symptomatic cerebrovascular disease, was also found to have a high sensitivity (0.97) and moderate specificity (0.60) for detecting stroke [12,13,14]. Just as for other available surveys, the QVSS includes two questions about a previous diagnosis of stroke or TIA, plus six questions about stroke symptoms. These six symptom-related questions alone could be used to screen for undiagnosed stroke. We evaluated the sensitivity and specificity of the six QVSS symptom questions for detecting stroke by reanalyzing existing data from the original study by Jones et al. [12], which described the operating characteristics of the whole 8-item scale.

Table 1.

Questionnaire for Verifying Stroke-Free Status (QVSS)

1 Were you ever told by a physician that you had a stroke?

2 Were you ever told by a physician that you had a TIA, ministroke, or transient ischemic attack?

3 Have you ever had sudden painless weakness on one side of your body?

4 Have you ever had sudden numbness or a dead feeling on one side of your body?

5 Have you ever had sudden painless loss of vision in one or both eyes?

6 Have you ever suddenly lost one half of your vision?

7 Have you ever suddenly lost the ability to understand what people are saying?

8 Have you ever suddenly lost the ability to express yourself verbally or in writing?

Methods

As has been published previously, subjects were outpatients of general internal medicine or stroke specialty clinics at a tertiary Veterans Administration hospital [12]. Not all stroke clinic patients had a stroke diagnosis, and some patients in the general medicine clinics would have had a prior stroke. Any patient who presented for routine care was eligible, and sequentially eligible patients presenting to the clinics were approached until accrual was complete. Intentionally, twice as many patients were sampled from the general medicine clinics to more closely approximate stroke prevalence in a general population. The research assistant administered the QVSS (table 1) to patients in the clinic, recording all demographic information and questionnaire responses into a database, with any question answered ‘unknown’ scored as a negative response. During the same clinic visit, a neurologist blinded to the QVSS results interviewed each patient and performed a neurological examination. The examining neurologist determined each subject's stroke status based on the combination of history and physical examination [12]. Subjects were scored as ‘yes’ for stroke or TIA if abnormal examination correlated with a history consistent with stroke or TIA, or if the subject had a normal examination but history suggested remote stroke or TIA. All other combinations of normal/abnormal history and physical examination were scored as ‘no’ for stroke or TIA. This determination was considered the benchmark for stroke status in this study. Both the research assistant and study neurologist were unaware of the stroke status of participants prior to the evaluation. Study subjects were not told the results of either the QVSS or the neurologists’ determination of their stroke status.

Results of all subject responses and results of neurologist history and physical examination were compiled in a database. Responses to the six symptom questions were then compared to the neurologist-determined stroke status, and sensitivities and specificities were calculated for individual questions, any of the six symptom-related questions, and clinically relevant combinations of questions (questions 3 and 4 – lateralized weakness and numbness; questions 7 and 8 – abnormalities of speech comprehension and production, and questions 3 and 8 – lateralized weakness with abnormal speech production, all of which represent common stroke presentations). All analyses were carried out in SAS version 9.1 (Cary, N.C., USA). The study was approved by the local human subjects review board.

Results

One hundred and fifty-five subjects were enrolled in the study: 98 from the general internal medicine clinic and 57 from the stroke specialty clinic (table 2). Of the 155 subjects, 77 (49.7%) were found to have stroke/TIA. Of these 77 subjects, 63 had a ‘yes’ response to one of the six symptom questions (sensitivity 0.82). Of the 78 subjects who were found not to have had a stroke or TIA, 48 answered ‘no’ to all six symptom questions (specificity 0.62).

Table 2.

Demographics of study subjects

Subject characteristics	Overall (n = 155)	Stroke status as determined by
		neurologist			symptom-related questions (3–8)
		stroke or TIA (n = 77)	no stroke or TIA (n = 78)	P	≥ 1 item positive (n = 93)	all items negative (n = 62)	P
Age, years
Mean ± SD	70 ± 8	68 ± 9	71 ± 7	0.03	70 ± 8	70 ± 8	0.84
Range	43–93	43–93	61–84		45–84	43–93
Sex
Male	152 (98.1%)	75 (97.4%)	77 (98.7%)	0.62	93 (100%)	59 (95.2%)	0.06
Female	3 (1.9%)	2 (2.6%)	1 (1.3%)		0	3 (4.8%)
Educational level, years1
Mean ± SD	12 ± 3	12 ± 3	12 ± 3	0.51	12 ± 3	12 ± 3	0.31
Range	0–20	3–18	0–20		0–20	6–20
General internal medicine clinic	98 (63.2%)	20 (26.0%)	78 (100%)		45 (48.4%)	53 (85.5%)
Stroke specialty clinic	57 (36.8%)	57 (74.0%)	0		48 (51.6%)	9 (14.5%)

¹Data missing in 13 subjects.

Characteristics of the individual and combinations of symptom questions are shown in table 3. The percentage of subjects answering ‘yes’ to each of the six symptom questions ranged from 14.8% for question 6 to 39.4% for question 3. None of the individual symptom questions was as sensitive as all six symptom questions together.

Table 3.

Symptom-related questions statistics

	% positive	Sensitivity	Specificity
Any questions (3–8)	60.0	0.82 (0.71,0.89)	0.62 (0.49, 0.72)
Questions 3 and 4	31.0	0.48 (0.36,0.59)	0.86 (0.76,0.92)
Questions 7 and 8	9.0	0.17 (0.09,0.27)	0.99 (0.93,0.99)
Questions 3 and 8	15.5	0.30 (0.19,0.41)	0.99 (0.93,0.99)
Question 3	39.4	0.60 (0.47, 0.70)	0.81 (0.70,0.88)
Question 4	38.7	0.57 (0.45,0.68)	0.79 (0.68,0.87)
Question 5	22.6	0.30 (0.19,0.41)	0.85 (0.74,0.91)
Question 6	14.8	0.22 (0.13,0.32)	0.92 (0.84, 0.97)
Question 7	17.4	0.23 (0.14,0.34)	0.88 (0.79, 0.94)
Question 8	21.3	0.38 (0.26,0.49)	0.95 (0.87,0.98)

Percent positive for ‘Any questions 3–8’ is the proportion of respondents who answered yes to one or more questions.

Percent positive for ‘Questions 3 and 4’, ‘Questions 7 and 8’, and ‘Questions 3 and 8’ is the proportion of respondents who answered yes to both of the included questions.

Values in parentheses are 95% confidence intervals calculated using binomial expansion.

The combination of symptom questions representing possible lateralized weakness and numbness (questions 3 and 4) was positive in 48 (31%) subjects, with a sensitivity of 0.48 and a specificity of 0.86. The combination representing possible mixed aphasia (questions 7 and 8) was positive in 14 (9%) subjects, with a sensitivity of 0.19 but a specificity of 0.99. The combination representing possible dysarthria-clumsy hand syndrome (questions 3 and 8) was positive in 23 (15%) subjects, with a sensitivity of 0.30 and a specificity of 0.99.

Conclusions

The six symptom questions of the QVSS demonstrated high sensitivity and moderate specificity for the detection of stroke or TIA in this sample, showing promise as a stand-alone screening tool to detect previously undetected stroke or TIA. Relatively nonspecific neurologic symptoms such as vision loss, speech disturbance, weakness, and numbness showed high specificity for stroke or TIA (0.79–0.95). The specificity of the individual symptom questions remained moderately high even when all six questions were evaluated jointly. The sensitivity of all six symptom questions together was higher than any of the individual questions alone, and any of the clinically relevant question combinations, in an acceptable range for a screening test.

Though the sensitivity of the symptom questions (0.82) was lower than the sensitivity of the original QVSS (0.97), this is not surprising given the high sensitivity of the questions regarding knowledge of a previous stroke or TIA [12]. As expected, combining symptom questions approximating common stroke syndromes does not improve sensitivity over the individual questions but significantly increases the specificity. The combinations of weakness with speech deficit (questions 3 and 8) and mixed aphasia (questions 7 and 8) have excellent specificity at 0.99. This is likely due to their sharing of the highly specific question 8, which asks about loss of the ability to express oneself, thus highlighting the specificity of speech disturbance for stroke. In future refinements of this or other screening questionnaires for stroke/TIA, our findings suggest that weighting the above symptom combinations and speech disturbance in particular may be wise.

The public health significance of this study lies in its support of the previous suggestion made by Howard et al. [2] that the symptom questions of the QVSS could potentially be used to detect individuals with undiagnosed stroke and TIA. They used the 6-item QVSS without knowing its sensitivity or specificity. As such, the decreased quality of life detected in these patients could not be attributed to stroke per se. Our findings suggest that much of the decrements in quality of life described could in fact be due to undiagnosed strokes. The alarming prevalence of stroke symptoms (nearly 18% of all subjects who reported no known history of stroke or TIA) suggests that their identification is an urgent and important public health issue.

There are few questionnaires to which the 6-item QVSS can be compared. Since they include questions about a known history of stroke or TIA, they are not useful for the purpose of identifying patients with undiagnosed stroke [4,5,6,7,9,11,15]. The Sicilian Neuro-Epidemiologic Study Group employed exclusively symptom-related questions, but their reported specificity (0.86) was for a variety of neurologic conditions and not specific for stroke [16]. Even the Stroke Symptom Questionnaire of Berger et al. [8] contains a question about a past diagnosis of stroke or TIA and thus is not a purely symptom-related questionnaire.

The limitations of this study include the characteristics of the study population, which are not representative of the population at large. Since the original design and intent of the QVSS was to detect stroke-free phenotype, the study population intentionally had much higher stroke prevalence than a normal population. For this reason, we did not present positive or negative predictive values (although they are available in the original paper [12]). In contrast, sensitivity and specificity should be independent of disease prevalence. Nevertheless, our findings should be validated in other populations, ideally among individuals not reporting a past history of stroke or TIA. Also, future studies correlating responses to the 6-item QVSS with brain imaging in addition to neurologic history and examination would strengthen our findings.

We conclude that the symptom questions of the QVSS, the 6-item QVSS, may be a useful tool for detecting clinically unrecognized stroke or TIA. In contrast to other screening questionnaires that either do not have exclusively symptom-related questions or have not been validated by neurologic history and examination, the 6-item QVSS has good sensitivity and reasonable specificity compared with neurological exam, and has great potential for meeting the important public health objective of detecting clinically unrecognized but symptomatic stroke or TIA.

Disclosure Statement

The authors have nothing to disclose.