Skip to main content
. 2020 Sep 4;12:270. doi: 10.3389/fnagi.2020.00270

Table 1.

Evaluation criteria for studies on BCS.

Type of criteria Definition Criteria for a positive score
Content validity Refers to the extent to which the BCS represents all aspects of the domains assessed (Streiner and Norman, 2003; Terwee et al., 2007). Includes description of the population to which the BCS is to be applied and evaluation of the BCS by dementia experts with at least two years of experience in cognitive and neuropsychological evaluation who responded to the content validity questionnaire. This questionnaire should contain the conceptual and operational definitions of the cognitive domains assessed and their respective indicators. The definition of each indicator should indicate how to administer the measure and provide instructions for scoring the corresponding domain. The experts should also be asked to assess the capacity of each subtest to evaluate the given cognitive domain, the capacity of each subtest to measure the corresponding indicator, and the clarity of the administration and scoring instructions. The experts should have the opportunity to provide observations and commentary as a basis for consensus discussions within the research team (which should consist of neurologists, geriatricians, psychiatrists, neuropsychologists, and/or other specialists) as well as discussions regarding suggestions and changes to the initially-proposed version of the tool. A positive score was assigned if the authors:
-Described the population and used a content validity questionnaire to obtain approval by expert consensus.
Internal consistency Refers to the homogeneity of items within a cognitive domain on the BCS, the correlation between domain and composite scores, and the assessment of whether these measures truly evaluate the same concept (Streiner and Norman, 2003; Terwee et al., 2007). Internal consistency should be measured using Cronbach's alpha. The effect of successively removing single items from the BCS on the Cronbach's alpha value should be evaluated. A positive score was assigned if Cronbach's alpha was ≥070.
Criterion validity Refers to the extent to which a BCS score is related to another applicable measure, ideally a “gold standard.” A positive score was assigned if there was a precise sample selection method, a detailed description of the sample, and positive correlation between the BCS and a “gold standard.”
Construct validity Refers to the use of indirect evidence to measure validity in the event that a “gold standard” is not available (Terwee et al., 2007). A positive score was assigned if the total score on the BCS and its cognitive domains were correlated with the MMSE, functional scales, or clinical dementia rating (CDR) scores in the individuals evaluated and if Spearman's correlation coefficient was applied when the data distribution was not normal.
Reproducibility Refers to the degree to which repeated measures in stable patients produce the same results (Terwee et al., 2007). The concept of reproducibility includes two elements: agreement and reliability. Agreement reflects the extent to which repeated measures produce the same results, which may be expressed as the standard error of measurement (SEM) or a Bland-Altman plot (de Vet et al., 2006). Reliability assures us that patient groups evaluated with the BCS can be distinguished from controls or other patient groups despite measurement error. Reliability can be evaluated using a statistic such as the intraclass correlation coefficient (ICC) (McGraw and Wong, 1996). A positive score was assigned if the ICC ≥0.70.
Diagnostic accuracy Diagnostic accuracy was evaluated according to the results of the receiver operating characteristic (ROC) curve analysis. ROC analysis can be used to identify cut-off points and calculate the area under the curve (AUC) in order to assess the sensitivity, specificity, and predictive value of the various cut-off points (Streiner and Norman, 2003; Bravo-Grau and Cruz, 2015). A positive score was assigned if the if AUC was ≥0.70.
Floor and ceiling effects A floor or ceiling effect was determined to be present if more than 15% of patients obtained the lowest (floor) or highest (ceiling) scores possible. When these effects are present, patients above or below these limits cannot be distinguished from one another, and change or variability cannot be measured (Terwee et al., 2007). A positive score was assigned if these effects were absent.
Interpretability Refers to the capacity to assign qualitative meaning to the quantitative scores, so that the BCS results can be interpreted. Adequate information should be available to determine whether a score or a change in score is clinically significant (Terwee et al., 2007). A positive score was assigned if the authors provided statistics for:
-A reference population (controls)
-Subgroups of relevant patients (dementia, subtype of dementia, MCI).