Table 4.

Criteria for assessing quality of individual studies (internal validity)⁵⁵

Analytic validity	Clinical validity	Clinical utility
Adequate descriptions of the index test (test under evaluation)	Clear description of the disorder/phenotype and outcomes of interest	Clear description of the outcomes of interest
Source and inclusion of positive and negative control materials	Status verified for all cases	What was the relative importance of outcomes measured; which were prespecified primary outcomes and which were secondary?
Reproducibility of test results	Appropriate verification of controls	Clear presentation of the study design
Quality control/assurance measures	Verification does not rely on index test result	Was there clear definition of the specific outcomes or decision options to be studied (clinical and other endpoints)?
Adequate descriptions of the test under evaluation	Prevalence estimates are provided	Was interpretation of outcomes/endpoints blinded?
Specific methods/platforms evaluated	Adequate description of study design and test/methodology	Were negative results verified?
Number of positive samples and negative controls tested	Adequate description of the study population	Was data collection prospective or retrospective?
Adequate descriptions of the basis for the “right answer”	Inclusion/exclusion criteria	If an experimental study design was used, were subjects randomized? Were intervention and evaluation of outcomes blinded?
Comparison to a “gold standard” referent test	Sample size, demographics	Did the study include comparison with current practice/empirical treatment (value added)?
Consensus (e.g., external proficiency testing)	Study population defined and representative of the clinical population to be tested	Intervention
Characterized control materials (e.g., NIST, sequenced)	Allele/genotype frequencies or analyte distributions known in general and subpopulations	What interventions were used?
Avoidance of biases	Independent blind comparison with appropriate, credible reference standard(s)	What were the criteria for the use of the interventions?
Blinded testing and interpretation	Independent of the test	Analysis of data
Specimens represent routinely analyzed clinical specimens in all aspects (e.g., collection, transport, processing)	Used regardless of test results	Is the information provided sufficient to rate the quality of the studies?
Reporting of test failures and uninterpretable or indeterminate results	Description of handling of indeterminate results and outliers	Are the data relevant to each outcome identified?
Analysis of data	Blinded testing and interpretation of results	Is the analysis or modeling explicit and understandable?
Point estimates of analytic sensitivity and specificity with 95% confidence intervals	Analysis of data	Are analytic methods prespecified, adequately described, and appropriate for the study design?
Sample size/power calculations addressed	Possible biases are identified and potential impact discussed	Were losses to follow-up and resulting potential for bias accounted for?
	Point estimates of clinical sensitivity and specificity with 95% confidence intervals	Is there assessment of other sources of bias and confounding?
	Estimates of positive and negative predictive values	Are there point estimates of impact with 95% CI?
		Is the analysis adequate for the proposed use?

NIST, National Institute of Standards and Quality.