Table 15.

Test Based Validity Evidence

Type of Evidence	Aspect of Validity	Validity Evidence
Test based evidence	Content	✓ Documentation of purpose & uses of the instrument ✓ Documentation of use of domain analysis ✓ Development of test blueprint & item templates ✓ Documentation of test development process ✓ Expert Review (Readability, Clarity, Clarity of instructions, Fairness, Sensitivity)
	Substantive	✓ Development of operationalized definition & theoretical framework of the construct (including internal, external, & processing models) ✓ Documentation of use of expertise ☓ Verifying of use of the proposed processes by respondents (think aloud)
	Structural	✓ Development of internal model ✓ Rationale for adoption of reference framework (criterion versus normative) ✓ Rationale for developing response format (rating scale and/or distracters) ✓ Rationale for adoption of measurement model
	Generalizability	✓ Specification of target population ✓ Selection of norming population
	External	✓ Demonstrating that the construct theory is sufficiently developed to support investigations of external aspect evidence
	Consequential	☓ Detecting positive or negative impact on systems (systemic validity)