Table 1.

A summary of the multisite evaluation approach

	Measurement study	Demonstration study
Quantitative results	Number of reviewers	PPV NPV Sensitivity Specificity
Qualitative results	Perceptions of evaluation approach effectiveness: ▸ General evaluation approach ▸ Reviewer expertise	Perceptions of benefit of results: ▸ Themes in FP ▸ Themes in FN

Evaluations included measurement studies (to determine the effectiveness of our evaluation approach) and demonstration studies (to demonstrate the value of our algorithm). The results included both quantitative and qualitative data.

FN, false negative; FP, false positive; NPV, negative predictive value; PPV, positive predictive value.