Table 1.

Summary of raters, material types, and PEMAT version for each round of reliability testing.

Round	Raters (#)	Material characteristics			PEMAT characteristics
		Materialsa (#)	Topics (#)	Language(s)	Total items	Understandability items	Actionability items	Response scale	Item instruction and examples
1	8	16 (P = 11, A/V = 5)	3	English and Spanishb	32	26	6	4 point	None
2	12	12 (P = 8, A/V = 4)	2	English	32	24	8	2 point	Few items had examples
3	2	46 (P = 23, A/V = 23)	33	English	32	25	7	2 point	Full user’s guide
4	2	46 (P = 23, A/V = 23)	33	English	32	25	7	2 point	Full user’s guide

^aP = print materials, A/V = audiovisual materials; in rounds 1 and 2, Print refers to both websites and pdfs.

^bSpanish materials were not tested after round 1 because the reliability results were poor and we needed to limit the points of variation in our testing approach.