Normative Performance on the Brief Smell Identification Test (BSIT) in a Multi-Ethnic Bilingual Cohort: A Project FRONTIER Study

Abstract

The Brief Smell Identification Test (BSIT) is a commonly used measure of olfactory functioning in elderly populations. Few studies have provided normative data for this measure, and minimal data are available regarding the impact of sociodemographic factors on test scores. This study presents normative data for the BSIT in a sample of English- and Spanish-speaking Hispanic and non-Hispanic Whites. A Rasch analysis was also conducted to identify the items that best discriminated between varying levels of olfactory functioning, as measured by the BSIT. The total sample included 302 older adults seen as part of an ongoing study of rural cognitive aging, Project FRONTIER. Hierarchical regression analyses revealed that BSIT scores require adjustment by age and gender, but years of education, ethnicity, and language did not significantly influence BSIT performance. Four items best discriminated between varying levels of smell identification, accounting for 59.44% of total information provided by the measure. However, items did not represent a continuum of difficulty on the BSIT. The results of this study indicate that the BSIT appears to be well-suited for assessing odor identification deficits in older adults of diverse backgrounds, but that fine-tuning of this instrument may be recommended in light of its items’ difficulty and discrimination parameters. Clinical and empirical implications are discussed.

Introduction

Over the past several decades, increased empirical attention has been directed at evaluating the utility of olfactory measures in detecting and diagnosing neurodegenerative disorders (Devanand et al., 2000; Kjelvik et al., 2007; Suzuki et al., 2004; Wilson, Arnold, Tang, & Bennett, 2006; Westervelt et al., 2003). Although these measures are often assumed to be “culture-free,” (Rosselli & Ardila, 2003), there has been little research investigating whether these measures can be appropriately applied to different ethnic, racial, and language populations.

The Brief Smell Identification Test (BSIT; Doty, Marcus & Lee, 1996) was developed as a quick tool to measure odor identification deficits. The BSIT is a brief, easily administered, and convenient instrument derived from the 40-item University of Pennsylvania Smell Identification Test (UPSIT; Doty, Shaman, & Dann, 1984). It is composed of twelve odorants embedded on scent strips that are released when scratched with a pencil tip. While the BSIT has been utilized among North-American (Wilson et al., 2007), Greek (Economou, 2003), Japanese-American (Graves et al., 1999), Norwegian (Kjelvik, Sando, Aasly, Engedal, & White, 2007), Mexican (Rodriguez-Violante, Lees, Cervantes-Arriaga, Corona, & Silveira-Moriyama, 2010), and Chinese samples (Wang et al., 2002), little work has been conducted to formally assess the impact of demographic factors on BSIT scores among culturally and linguistically diverse populations.

In addition to the dearth of studies evaluating the impact of sociodemographic factors on BSIT scores among diverse ethnic and language groups, there is also relatively little normative data available on this measure. Although normative data are provided with the test (Doty et al., 1996), they were extrapolated from the original 40-item University of Pennsylvania Smell Identification Test (Doty et al., 1984). This is a significant potential limitation to the extant normative data given that the distractor choices were altered in 9 of the 12 BSIT items. To our knowledge, the possible impact of this alteration has not been examined, though Hawkins et al. (2011) have provided some initial support for the use of the BSIT in a sample of African Americans. The authors demonstrate that BSIT performance is relatively resistant to sociodemographic influences including education in their cohort. However, the use of the BSIT has not yet been examined in a Hispanic sample, the largest ethnic minority group within the U.S. The importance of culturally-appropriate normative reference data has been increasingly recognized (Manly, Jacobs, Touradji, Small, & Stern, 2002), and has received increased empirical attention in the context of other smell tests including the UPSIT (e.g., Silveira-Moriyama, 2010) and the Sniffin’ Sticks test (Yuan, Lee, Lee, Lin, Shu, 2010). Efforts are needed to extend these findings to the BSIT in samples of varying sociodemographic backgrounds.

This study aimed to evaluate the use of the BSIT in a bilingual sample of Hispanics (primarily Mexican-Americans) and non-Hispanic Whites and to provide normative data for this population. The influence of various sociodemographic factors such as age, gender, education, language, and ethnicity on BSIT performance was examined. Lastly, item-level analyses were conducted using Rasch analysis. Rasch analyses have been applied to the UPSIT in previous studies, and have found that UPSIT items cover a range of difficulty (Minor et al., 2004; Lange, Donathan, & Hughes, 2002; Minor, Wright, & Park, 2004). However, no studies to date have analyzed the BSIT using a Rasch model. This study sought to replicate previous findings for the BSIT, identifying the items that best discriminate between varying levels of smell identification, as well as the items that provide the most information about the construct being measured (Thissen & Steinberg, 2009). This study also sought to identify a cutoff score to differentiate cognitively impaired from unimpaired participants using ROC curve analysis. It was hypothesized that BSIT performance would be resistant to sociodemographic influences including education, language, and ethnicity, and that most BSIT items would discriminate well at varying levels of difficulty on the smell identification ability continuum.

Materials and Methods

Participants

Data were analyzed from a sample of 302 Hispanic and non-Hispanic White older adults recruited as part of Project FRONTIER, an ongoing epidemiological study of rural health. Inclusion criteria are (1) age 40 and above; and (2) residing in one of the counties part of Project FRONTIER, including Cochran County and Parmer County, both located on the Texas-New Mexico border. These criteria are reflective of the nature of Project FRONTIER, which is an investigation of aging among rural-dwelling adults and older adults who may be at increased risk for cognitive and medical problems. Participant recruitment is conducted by community recruiters through brochures/flyers, presentations and events, and in-person or door-to-door solicitation. Project FRONTIER is conducted under an Institutional Review Board approved protocol that includes a standardized medical examination, clinical labs, interviews with participants and informants, and neuropsychological tests. The BSIT was administered as part of an extensive test battery used in Project FRONTIER. The medical exam and clinical labs were conducted by the local hospital and the interview and neuropsychological testing completed at the Project FRONTIER office at that same hospital by psychological technicians. Project FRONTIER research staff has received extensive training on interview and neuropsychological testing procedures with annual quality checks (to protect against experimenter drift) performed. The detailed interview collected information regarding demographic background, SES (current and past income as well as current and past occupational history), residential history, medical history, and included portions of the CDC Behavioral Risk Factor Surveillance System (BRFSS) questionnaire capturing information regarding hypertension, cholesterol, diabetes, cardiovascular disease, and cancer.

All participants signed written informed consent. Self-report of race and Hispanic origin (ethnicity) was collected using census methods. The full Project FRONTIER protocol has been described elsewhere (more detail provided in O’Bryant, Schrimsher, Johnson, & Zhang, 2011). Self-reported number of years of education was used as an indicator of educational attainment.

Approximately half of the Hispanic subgroup (48.5%) reported being of Mexican origin and the remainder indicated that they were born and raised in the United States (51.5%). A number of Hispanic participants described having greater mastery of Spanish than English (22.1%), and all other non-Spanish speakers of Hispanic origin endorsed English as their primary language (77.9%). The test instructions were provided in Spanish among participants who reported being more proficient in Spanish than English. All participants who reported adequate mastery of English were tested in this language.

For the acquisition of a well-defined “healthy” sample, all information obtained for each participant was systematically examined by consensus review, and participants were excluded if the data suggested the presence of cognitive impairment (e.g., current or preexisting diagnosis of dementia, mild cognitive impairment) and medical conditions susceptible of influencing neurocognitive functioning (e.g., histories of stroke, transient ischemic attack, head injury/concussion, seizures, Parkinson’s disease, brain hemorrhage, and significant upper respiratory conditions). Participants with nasal congestion resulting from significant upper respiratory conditions (e.g., colds, allergies) were also excluded. The Project FRONTIER Consensus Diagnostic Committee is composed of an internist, a psychologist, and a neuropsychologist, and diagnoses are assigned according to published criteria (i.e., NINCDS-ADRDA possible/probable Alzheimer’s disease, vascular dementia, “other” dementia [do not meet criteria for AD or VaD], mild cognitive impairment [Mayo Criteria], and “other cognitive impairment” not meeting one of the above categories. “Cognitively normal” cases were consensus reviewed and deemed to have performed within normal limits on the cognitive test battery. The neuropsychological tests measured functioning in many domains including memory, executive functioning, visuospatial and constructional abilities, intellectual functioning, motor abilities, attentional and verbal abilities. The consensus diagnosis procedures have been more thoroughly described elsewhere (O’Bryant et al., 2011). Because Rasch analysis requires a continuous latent variable to meet assumptions of the analysis, no exclusion criteria applied to the participants included in the Rasch analyses.

Measures

The Brief Smell Identification Test (BSIT; Doty, Marcus, & Lee, 1996) includes 12 odorants embedded on scent strips and released when scratched with a pencil tip. Participants were presented a four-category multiple-choice questionnaire and asked to identify which smell corresponded to the scent strip for each odorant. The BSIT is a forced-choice test, with participants being instructed to identify each odorant even if no particular smell is perceived. The total olfaction score using the BSIT was defined as the number of odorants correctly identified out of the 12 tested, with higher scores designating better olfactory performance and abnormal olfactory functioning being defined as correctly identifying fewer than nine odorants (Doty, 2001). This test has been found to have good internal reliability and validity.

Statistical Analyses

Independent samples t-tests and chi-square analyses were conducted to examine any systematic differences between the two groups (cognitively impaired versus healthy) on basic sociodemographic variables including age, education, gender, and ethnicity. Any significant differences found on the demographic variables prompted their inclusion as covariates in subsequent analyses. Linear regression models were created to further delineate the relationship between demographic factors and BSIT scores. This was done to determine if and how the sample should be stratified based on these factors. In an effort to test for any significant differences in BSIT performance across groupings, one-way analyses of variance were conducted among the different age groups and an independent t-test was used with gender categories. Cronbach’s α coefficient was obtained to examine the level of agreement among the BSIT test items. Corrected item-total correlations were also calculated as a complementary index of consistency, which involve examining the correspondence between each item score and the composite total score extracted from all other items. Correlations and chi-square analyses were also conducted to examine the relationship between key sociodemographic variables and specific BSIT items. Finally, receiver operating characteristic (ROC) curve analyses were conducted to identify the best cutoff scores on the BSIT to identify cognitive impairment (dichotomized as cognitively healthy versus any cognitive impairment [all categories]), as the only cutoffs available to date are for smell abilities. ROC curves are commonly utilized to plot sensitivity (the likelihood of correctly detecting positive cases) against rates of false positives (1 - specificity, designating the proportion of correctly identified negative cases) for all possible cut-scores. These analyses used the Statistical Package for the Social Sciences Software (SPSS), version 16.0, and significance was set at p ≤ .05.

Following these analyses to provide normative data for BSIT, a Rasch analysis was conducted with all participants included as one group, using the ltm package (Rizopoulos, 2006) for R (R Development Core Team, 2006). Rasch analysis assumes that all items are identically related to an underlying latent trait (e.g., smell identification as it relates to smell identification ability), and identifies the difficulty of each item along the latent trait, as well as how well each item discriminates between varying levels of the latent trait (Thissen & Steinberg, 2009). A two parameter logistic model, which allows for both a difficulty score and a discrimination score per item, was most appropriate for the data. A likelihood ratio test revealed that the two-parameter logistic model was preferable to both the constrained and unconstrained models (p < .001). Akaike Information Criterion (AIC) for the constrained model was 2544.34; AIC for the unconstrained model was 2541.50, and AIC for the two-parameter logistic model was 2512.76. Using the two-parameter logistic model, we examined the discrimination scores, difficulty scores, item characteristic curves, and item information curves for each of the 12 items on the BSIT.

A Rasch analysis was first run with only cognitively healthy participants, but the goodness of fit statistics indicated that models did not have acceptable fit, regardless of whether they were constrained, unconstrained, two-parameter logistic, or introduced a guessing parameter. This is likely because an arbitrary cut-point was introduced in the latent trait when participants were excluded for cognitive impairment, and this analysis requires a continuous latent trait that represents all levels of performance to provide appropriate difficulty and discrimination parameters. Therefore, all participants were included in the Rasch analysis; this was the only way to ensure that all levels of difficulty would be represented, and the fit of the models was acceptable when all participants were included.

Results

Table 1 presents the descriptive characteristics of the sample across ethnicities. In the total sample of 302 individuals, the mean age of participants was 61.71 (SD = 13.17), with a mean of 10.29 (SD = 4.33) years of education. Forty-three percent of the sample was Hispanic and 57% was non-Hispanic White. The majority of the sample was women (68.2%).

Table 1.

Descriptive Statistics of BSIT

Attributes	Complete Sample (n = 302)	Cognitively Impaired Sample (n = 108)	Total Healthy Sample (n = 194)	Healthy Subgroup Hispanic (n = 81)	Healthy Subgroup Non- Hispanic Whites (n = 113)	Comparison of Total Healthy Sample to Cognitively Impaired	p
Age, M (SD), rg.	61.71(13.17), 40–96	66.7(13.36), 40–94	58.92(12.25), 40–96	54.60(9.82), 40–79	63.0(13.0), 40–96	t(300) = −5.11	<.001
Education, M (SD), rg.	10.29(4.33), 0–20	9.46(4.0), 0–16	10.75(4.44), 0–20	6.99(3.70), 0–15	13.62(2.61), 6–20	t(300) = 2.49	<.05
Hispanics (%)	43.0	45.4	41.8	--	--	X2(1, N = 277), d = 1.54	0.52
Females, %	68.2	68.5	68.0	69.1	66.7	X2(1, N = 302), d = 0.01	0.13
MMSE, M (SD), rg.	27.1 (3.10), 12–30	22.85 (2.74), 12–23	28.05 (2.24), 24–30	27.10 (2.70), 24–30	29.0 (1.50), 24–30	t(300) = 7.77	<.001
CDR, M (SD), rg.	0.2 (0.2), 0–1	0.30 (0.25), 0–1	0.12 (0.21), 0–0.5	0.09 (0.20), 0–0.5	0.14 (0.23), 0.0.5	t(300) = −6.63	<.001
BSIT, M (SD), rg.	10.2 (2.00), 2–12	9.40 (2.24), 2–12	10.63 (1.42), 4–12	11.00 (1.30), 7–12	10.66 (1.50), 4–12	t(300) = 5.97	<.001

Notes: M = mean; SD = standard deviation; rg. = range; MMSE = Mini Mental Status Exam; CDR = Clinical Dementia Rating scale; BSIT = Brief Smell Identification Test. X²chi-square; d = Cohen’s d. p-values obtained for comparison of those excluded and the normative sample using t-test or chi-square analysis.

A total of 108 (35.76%) participants were excluded from the BSIT normative sample based on the exclusion criteria described previously. Among the group of cognitively impaired participants, 89 met diagnostic criteria for MCI, 15 for dementia, and 4 for “other cognitive impairment.” The final normative sample included 194 participants with a mean age and education of 58.92 (SD = 12.25, range = 40–96) and 10.75 (SD = 4.44, range = 0–20), respectively. Cognitively impaired cases were older and less educated compared to the healthy sample.

A hierarchical regression analysis was conducted using only the cognitively healthy sample to determine the relative relationship of key sociodemographic variables with BSIT performance and to aid in establishing proper stratification criteria. Age, gender, years of education, ethnicity, and language of test administration were entered simultaneously as a first step. The full model was significant, accounting for 12.0% of the total variance in BSIT total scores, F(5, 271) = 7.22, p <.001. Both age and gender were significantly related to BSIT total scores, β = −.04, t = −4.70, p < .001 and β = .60, t = 2.80, p < .01, respectively. In contrast, years of education, β = .02, t = .06, p = .95, ethnicity, β = .17, t = .54, p = .60, and language β = −.35, t = −1.13, p = .26, did not predict BSIT performance after controlling for age and gender.

Based on these results, BSIT total scores were stratified by age and gender, with four distinct age subsamples created with fairly equivalent sample size representation across groups. The four age groups consisted of participants between the ages of 40 and 49, 50 and 59, 60 and 69, or 70 or more years old. The mean scores for the BSIT in the different age and gender groupings are reported in Table 2. One-way ANOVAs indicated significant differences in BSIT performance among the different age groupings among women (F (3, 104) = 3.28, p = .02) and men (F (3, 44) = 2.82, p = .05). Independent t-tests also revealed significant BSIT performance differences across gender groups, t (154)=−3.31, p=.001. Additionally, the ROC curves were generated for each of these cells, and a score of 11 maximized sensitivity and specificity across all cells in terms of identifying cognitive impairment in our sample (see Table 3).

Table 2.

Normative Performance of BSIT

	Age
	40–49		50–59		60–69		70+
	Women	Men	Women	Men	Women	Men	Women	Men
n	37	13	35	18	29	17	31	14
Mean	10.67	10.76	10.94	10.61	11.14	10.06	10.32	9.93
SEM	0.20	0.28	0.21	0.38	0.14	0.42	0.35	0.24
SD	1.27	1.01	1.28	1.64	1.00	1.75	1.94	1.00
Range	8–12	9–12	7–12	7–12	10–12	6–12	4–12	8–11
Median	11	11	11	11	11	11	11	10
5%	8	9	8	7	10	6	5	8
10%	9	9	9	7	10	7	7	8
25%	10	10	10	10	10	9	10	9
50%	11	11	11	11	11	11	11	10
75%	12	12	12	12	12	11	11	11
90%	12	12	12	12	12	12	12	11
95%	12	12	12	12	12	12	12	11

Table 3.

ROC Curve Analyses Results.

	Age
	40–49		50–59		60–69		70+
	Women	Men	Women	Men	Women	Men	Women	Men
Recommended Cutoff	11	11	11	11	11	11	11	11
Sensitivity	.800	1.000	.813	.667	.643	1.000	.846	.947
Specificity	.684	.692	.600	.667	.621	.875	.800	1.000

Inter-item correlations and identification rates were also examined. Cronbach’s α coefficient was .60 in our sample. Significant differences in item identification rates were noted, with 98% of the cohort correctly identifying onion, but only 66% identifying turpentine. Correlation coefficients between specific items also varied substantially, with the lowest correlation coefficient observed for cinnamon (.11), and the highest noted for onion (.36). Item-total correlations for four items were significantly higher for women than men, and identification rates for five items were significantly and negatively correlated with age (See Table 4).

Table 4.

Item Identification Rates, Gender and Age Effects in BSIT Total Scores

Item #	Odor	Correct Identification Rates	Corrected Item-Total Correlation	Gender Differences	Correlation with Age
12	Onion	98.0	0.36	X2(1, N = 194), d = 1.68, p = .19	−0.13, p = 0.40
11	Soap	98.0	0.20	X2(1, N = 194), d = 6.49, p < .01**	−0.12, p = .11
10	Gasoline	98.0	0.18	X2 = (1, N=194), d = 6.49, p <.01**	−0.21, p <.01**
7	Paint Thinner	96.0	0.15	X2 = (1, N=194), d = 9.65, p <.01**	−0.20, p <.05*
5	Chocolate	95.0	0.15	X2 = (1, N=194), d = 1.58, p =.21	−0.20, p <.05*
1	Cinnamon	95.0	0.11	X2 = (1, N=194), d = 0.41, p =.52	−0.03, p =0.66
4	Smoke	95.0	0.22	X2 = (1, N=194), d = 0.67, p =.41	−0.24, p <.01**
9	Pineapple	90.0	0.25	X2 = (1, N=194), d = 0.31, p =.58	−0.12, p =.13
3	Lemon	81.0	0.21	X2 = (1, N=194), d = 0.04, p =.84	−0.17, p <.05*
8	Banana	75.0	0.25	X2 = (1, N=194), d = 0.01, p <.90	−0.14, p =.07
6	Rose	73.0	0.36	X2 = (1, N=194), d = 4.92, p <.05*	−0.04, p =.62
2	Turpentine	66.0	0.19	X2 = (1, N=194), d = 0.38, p =.53	−0.01, p =.92

Note: X²chi-square; d = Cohen’s d; p-values obtained for comparison of gender groups using chi-square analyses, and correlation analyses.

^*p ≤ .05, two-tailed;

^**p< .01, two-tailed;

^***p < .001, two-tailed.

The Rasch analysis conducted with both cognitively healthy and cognitively impaired participants identified four items with good discrimination scores (i.e., paint thinner, gasoline, soap, and onion; see Table 5). Item characteristic curves verified this information, as the lines representing these four items had the steepest curves, indicating that responses discriminate well between varying levels of the latent trait (see Figure 1 for item characteristic curves). These four items provided 59.44% of all information assessed by BSIT. However, all four of these items had similar difficulty scores (see Table 5), indicating that they each measure approximately the same level of the latent trait. Other items had difficulty scores that differed from these four items, but had poor discrimination scores, indicating that they do not discriminate well between varying levels of the underlying latent trait (i.e., smell identification ability). This is reflected in the item information curves (see Figure 2), which show that the four curves representing the items that discriminate well overlap significantly with each other, meaning that they all measure similar difficulty levels along the latent trait. The flat or nearly flat lines are those items that do not discriminate well, though slight elevations were noted at other points along the ability axis, indicating that they measure different difficulty levels.

Table 5.

Rasch Analysis Item Scores.

Item	Discrimination Score	Difficulty Score
Cinnamon	0.8412	−4.1786
Turpentine	0.5678	−0.8465
Lemon	0.7599	−1.7447
Smoke	1.3347	−2.3434
Chocolate	1.3804	−2.1187
Rose	1.2183	−0.7658
Paint Thinner	2.1871*	−1.8082
Banana	1.0006	−1.0559
Pineapple	1.1506	−2.0754
Gasoline	3.7693*	−1.9386
Soap	2.7896*	−1.9152
Onion	3.2838*	−1.9497

Note:

^*Selected items

Figure 1.

Item characteristic curves, showing discrimination scores for each BSIT item. Top plot provides curves for selected items only (for clarity) and bottom plot provides curves for all 12 items (for comparison to non-selected items).

Figure 2.

Item information curves, showing difficulty scores for each BSIT item. Top plot provides curves for selected items only (for clarity) and bottom plot provides curves for all 12 items (for comparison to non-selected items).

Discussion

This study provided normative data for BSIT in a previously unexamined bilingual sample of Hispanic and non-Hispanic Whites. Significant differences in BSIT performance were noted between men and women and among different age groups, with higher BSIT scores found in women and younger individuals. In contrast, years of education, ethnicity, and language were not significantly associated with BSIT performance. This pattern of results corroborates previous findings showing that BSIT scores are relatively resistant to sociodemographic factors, and suggests that this test can be applied to these different groups.

Age had a significant negative influence on BSIT performance, which is consistent with previous studies indicating that aging is associated with heightened deficits in olfaction (Murphy et al. 2002; Bramerson et al. 2004; Strauss, Sherman, & Spreen, 2006). In part, the relationship between aging and olfactory impairment has been attributed to degeneration of the olfactory bulb and olfactory epithelium (Bhatnagar, 1987; Duda et al., 1999), ossification of the olfactory foramina (Kalmey, Thewissen, Dluzen, 1998), and neuropathological changes connected with neurodegenerative diseases (Hawkes & Doty, 2009). Although progress has been made in understanding the relationship between aging and olfactory decline, more research is needed to further delineate the respective influences of peripheral versus cortically-based olfactory pathology on BSIT scores, as this differentiation is essential to ensuring valid clinical interpretations (Doty et al., 1984; Martzke et al., 1997).

Our findings indicate that gender is a significant predictor of BSIT performance, with women found to outperform men on this measure. This pattern of results is congruent with previous research suggesting that women tend to be less vulnerable to smell deficiencies across the lifespan (e.g., Economou, 2003; Doty et al., 1985; Hawkins et al., 2011). Although the exact cause of this gender gap is not fully understood, gender differences in circulating hormones have been proposed to account for females’ better sense of smell throughout the lifespan (e.g., Doty & Cameron, 2009; Doty, 1986). Further, discrepancies in verbal abilities have been found to mediate the relation between gender and olfactory recollection (Larsson, Lovden, & Nilsson, 2003) and could also have contributed to this finding.

Educational attainment did not significantly affect BSIT performance in our cohort. The lack of association between these two factors offers support to the literature and suggests that the BSIT is relatively resistant to sociodemographic factors (Doty et al., 1996; Goudsmit et al., 2003; Hawkins & Pearlson, 2011). The fact that this sample comprised a greater range of educational attainment compared to previous examinations is a strength of this study, as it does not merely replicate but also expands on previous reports by demonstrating that the BSIT is a valuable assessment tool in both highly educated and non-educated individuals.

Consistent with previous research, ethnicity did not account for a significant proportion of variance in BSIT performance (Hawkins et al., 2011), which suggests that the BSIT is appropriate for use across these ethnic groups. Similarly, language of administration did not significantly affect BSIT scores. To our knowledge, this study is the first to examine these effects in a bilingual sample of Hispanics and non-Hispanic Whites. Although more research is needed to replicate these findings with a larger sample size, these preliminary results are encouraging in as much as they suggest that the BSIT has value in assessing for odor identification deficits in diverse populations of varying ethnic and linguistic backgrounds.

Nonetheless, our follow-up examination of internal consistency and item performance revealed that the reliability of the BSIT is less than ideal, and that fine-tuning of this instrument may be recommended. Specifically, the internal consistency of the BSIT was moderate in our cohort, with a Cronbach’s α coefficient of .60, which is comparable to previous reports relying on samples of African-Americans and Greek participants (Economou, 2003; Hawkins et al., 2011) and represents a substantial decrease from reliability estimates attributed to the UPSIT (Doty et al., 1996). While the relative brevity of this measure may account for this pattern, disparities in item identification rates and specific item correlations could also play a role in this issue.

In particular, important differences were noted in the psychometric performance of certain BSIT items, with the lowest reliability coefficient identified for turpentine (.66) and the highest coefficient for onion (.98). Turpentine was one of the two items that had two changes to the distractor choices when extrapolated from the UPSIT, which could have contributed to the lower reliability coefficients for this item. The effects of gender and age were relatively inconsistent across items. In addition, only four items appeared to discriminate well between those with varying levels of the underlying trait of smell identification, though these items all measure approximately the same level of difficulty in this trait. These four items (paint thinner, gasoline, soap, and onion) account for a majority of the information assessed by BSIT, which suggests that these items may be used when a very brief measure is needed, and fine-tuning of this instrument may therefore be recommended. In particular, items that assess for and discriminate well among other levels of difficulty should be developed to ensure that the BSIT assesses for the full continuum of ability. The current version of the BSIT is appropriate for use among adults and older adults, but may cause undue participant burden as eight items do not provide much information about the participants’ ability. It is important to note that the sample used for the Rasch analyses included both cognitively healthy and cognitively impaired participants, allowing for a model that fit the data well but also potentially suggesting that these results may differ in samples that are only cognitively healthy or only cognitively impaired. However, objective measurement is an important component of data collected for Rasch analyses (Thissen & Steinberg, 2009) and previous research suggests that the difficulty estimates provided by Rasch analyses are invariant across samples (Slinde & Linn, 1979).

Results from the ROC analyses identified 11 as a recommended cutoff score for identifying cognitively impaired from cognitively healthy individuals in this sample. Sensitivity and specificity are commonly used to evaluate clinical measures, with a balance of both suggesting greater detection power. Blake et al. (2002)’s recommendations suggest sensitivity indices higher than 80% as good with specificity values above 60% to be appropriate, though the optimal balance is dependent on the use a the test in a given setting along with the base rate of the given condition. Although the diagnostic performance in this sample was generally appropriate based on these criteria, it should be noted that the sensitivity and specificity values were less than ideal among some of the age and gender subgroups, including among men in the 50 to 59 age range (sensitivity and specificity = .667) and among women in the 60 to 69 age range (sensitivity =.643). Thus, these results suggest that the BSIT alone would not be a sufficient screener of cognitive impairment among these specific subgroups. Furthermore, future studies focusing on the fine-tuning of this instrument should take into account the complex interplay of various sociodemographic variables including age and gender in determining the diagnostic performance and differential item functioning profiles of such abbreviated versions.

Future studies are needed to replicate these findings in other subgroups as well. Our results parallel previous studies (e.g., Kjelvik et al., 2007) and underscore the need for efforts to examine the utility of BSIT items among different subgroups. In particular, the current findings underscore the need to explore the potential of a shortened version of the BSIT excluding these items found to have relatively low discriminatory power (e.g., turpentine and lemon). Importantly, previous item analysis research indicated different patterns of BSIT item sensitivities for varying disease states (e.g., Double et al., 2003). Given this, it is possible that the item composition specified in this study may not apply well to other groups. Thus, future research should further examine and cross-validate the specific misidentification patterns associated with different disease states and sociodemographic domains before being utilized as a point of reference in clinical and research settings.

While the current study has a number of strengths and contributes significantly to the literature, this study has limitations. First, future studies should replicate this investigation with larger samples, as the sample size in each cell was relatively low. Additionally, it is possible that our specified subgroups encompassed significant heterogeneity in culture, primary language, immigration history, and socioeconomic status. As such, efforts are needed to replicate and expand on these findings while accounting for these potentially influential dimensions. In particular, it is possible that some of the participants had limited knowledge or previous exposure to certain target odors, such as turpentine, and that this lack of familiarity influenced their performance on certain items. Future research should seek to rely on corroborating assessments of relevant odor knowledge or exposure (e.g., the Picture Identification Test) in an effort to minimize this possible source of bias. In addition, future studies should examine the role of bilingualism in BSIT performance, as dual linguistic mastery has been consistently shown to influence neurocognitive functioning (Bialystok, Craik, Green, & Gollan, 2010). Furthermore, the fact that the screening tests utilized to delineate our healthy sample were not primarily created or validated for use in this diverse cohort is also of concern, as it may have inadvertently introduced construct, item, or method biases (Ardila & Moreno, 2001; Greenfield, 1997; Rogoff & Chavajay, 1995; Van de Vijver & Tanzer, 1997). This underscores the need for continued efforts to cross-validate various neuropsychological assessments in heterogeneous populations.

Lastly, it is possible that discrepancies in quality of education were present among the subgroups of interest, as different sociodemographic subgroups may be exposed to very different educational contexts (Manly, Jacobs, Touradji, Small, & Stern, 2002). In turn, this may have resulted in a somewhat biased representation of the relationship between education and BSIT performance. While quantity of education was included in the present analyses, quality of education has been offered as a more precise operationalization of educational attainment as it accounts for variability in educational experiences (Cosentino, Manly, & Mungas, 2007; Manly et al., 2002). Thus, future research should examine the effect of quality of education on BSIT performance among diverse samples.

Despite these limitations, this study is to our knowledge the first to provide normative data for the BSIT in a sample of English- and Spanish-speaking Hispanics and non-Hispanic Whites, and was the first to apply a Rasch analysis to the BSIT. Data was also provided for a more comprehensive age and education range compared to previous normative studies on this measure (e.g., Hawkins et al., 2011). As noted earlier, this represents an important first step in moving towards more socioculturally adapted and valid diagnostic decision-making. The results of this study indicate that the BSIT appears to be well-suited for assessing odor identification deficits in older adults of diverse backgrounds, but that fine-tuning of this instrument may be recommended in light of its item’s difficulty and discrimination parameters.

Key Points.

Adequate normative data is needed for neuropsychological assessments with diverse population. The BSIT appears to be well-suited for assessing odor identification deficits in older adults of diverse backgrounds, although fine-tuning may be recommended based on its item profiles.