Table 1.

Sources of error and mitigating strategies for outcome assessment in multi-site TBI studies

Source of Error	Example	Optimization Strategies
Participant	non-injury related premorbid patient characteristics/demographics	conduct thorough interview to ensure pre-morbid factors can be accounted for monitor participant demographics and adjust enrollment strategies to ensure balanced sample
	altered cognitive status	use Test Completion Codes to indicate validity of assessment ensure outcome assessments are available to test participants at every level of cognitive function
	executive deficits	be alert to signs of difficulty offer cues and redirection offer breaks between measures
	illness/pain	use Test Completion Codes to indicate validity of assessment offer to take breaks between measures to attain best effort/performance reschedule assessment
	fatigue	offer breaks between measures
	motor impairment	use Test Completion Codes to indicate validity of assessment
	language barrier	exclude subjects who are not fluent in the language of the outcome battery assessment consider offering testing in other languages
	poor effort	use Test Completion Codes to indicate validity of assessment offer breaks between measures attempt to address the issue impacting effort
	failure to return for follow-up	plan follow-up around the participant’s schedule arrange and pay for transportation obtain contact information from others who will be able to locate the participant in the event of any changes in home or email address or phone number start contacting and scheduling participants early into the follow-up window to allow for re-scheduling if needed offer assessment locations that require minimal travel conduct assessments in the subject’s home If all efforts fail for inperson evaluation, administer measures that can be done by phone
Examiner	administration of incorrect form	ensure only one form of each assessment is available to examiners store all current forms in a centralized, easy-to-access location immediately communicate and disseminate changes in forms to examiners
	improper adherence to standardized administration guidelines	provide detailed Standard Operating Procedures and ample training provide frequent opportunities for examiners to present questions to other examiners and the expert working group
	poor adherence to time and event anchors	conduct extensive in-person and tele-training and re-training to ensure all examiners are familiar with assessment instructions
	errors in scoring	minimize the number of assessments that require manual scoring develop automated scoring algorithms create and adhere to a data quality monitoring and contingency plan
	data entry, transcription and conversion errors	create and adhere to a data quality monitoring and contingency plan (e.g., double score all forms, double check all paper to electronic record data entry)
Measure	lack of clear administration and scoring instructions	thoroughly review literature to ensure validated instructions and scoring are used train and re-train examiners throughout study use case series to illustrate scoring provide a forum for discussing questions and ensure clarity
	multiple forms and versions	search for and thoroughly review all available forms ensure investigators and examiners from all sites are aware of the assessment version being used provide access to the accepted form in one central location
	alternate forms required to avoid practice effects across assessments	identify alternate forms of equal difficulty ensure alternate forms are validated provide extensive training on when to use each alternate form and provide written guidance in SOP

Examples of the three sources of error in outcome assessment and suggestions for optimization