Table 2.

Item-level reliability and validity results for items in the final PEMAT.

		Content validitya	External consistency (inter-rater reliability)			Internal consistency		Construct validity
Item #	Item (relevant for printable [P] and/or audiovisual [A/V] materials)	Average (1–3)	% Agree	Kappa	Gwet’s AC1	Cronbach’s α	Item-total correlation	Qualitative Findingsb
	Understandability
1	The material makes its purpose completely evident (P and A/V)	1.00	88	0.71	0.80	0.68	0.62	(+)
2	The material does not include information or content that distracts from its purpose (P)	1.25	81	0.48	0.70	0.80 (P)c	0.62 (P)	(+)
3	The material uses common, everyday language (P and A/V)	1.00	81	0.57	0.66	0.66	0.70	(+)
4	Medical terms are used only to familiarize audience with the terms. When used, medical terms are defined (P and A/V)	1.25	81	0.54	0.68	0.66	0.73	(+)
5	The material uses the active voice (P and A/V)	1.00	88	0.40	0.85	0.75	0.31
6	Numbers appearing in the material are clear and easy to understand (P)	1.00	81	0.55	0.76	0.79 (P)	0.79 (P)
7	The material does not expect the user to perform calculations (P)	1.00	86	0.69	0.74	0.82 (P)	0.32 (P)	(+)
8	The material breaks or “chunks” information into short sections (P and A/V)	1.00	79	0.56	0.72	0.70	0.55	(+)
9	The material’s sections have informative headers (P and A/V)	1.00	83	0.70	0.77	0.69	0.60	(+)
10	The material presents information in a logical sequence (P and A/V)	1.25	86	0.42	0.81	0.67	0.68
11	The material provides a summary (P and A/V)	1.75	69	0.46	0.58	0.73	0.41	(+)
12	The material uses visual cues (e.g., arrows, boxes, bullets, bold, larger font, highlighting) to draw attention to key points (P and A/V)	1.00	74	0.51	0.65	0.71	0.52	(+)
13	Text on the screen is easy to read (A/V)	e	71	0.40	0.63	0.77 (A/V)	0.50 (A/V)	(+)
14	The material allows the user to hear the words clearly (e.g., not too fast, not garbled) (A/V)	1.00	86	0.66	0.82	0.76 (A/V)	0.53 (A/V)	(+)
15	The material uses visual aids whenever they could make content more easily understood (e.g., illustration of healthy portion size) (P)	1.00	76	0.48	0.56	0.80 (P)	0.59 (P)	(+)
16	The material’s visual aids reinforce rather than distract from the content (P)	1.00	86	0.70	0.81	0.81 (P)	0.56 (P)	(+)
17	The material’s visual aids have clear titles or captions (P)	1.25	90	0.84	0.86	0.81 (P)	0.51 (P)	(+)
18	The material uses illustrations and photographs that are clear and uncluttered (P and A/V)	1.25	88	0.67	0.85	0.74	0.34
19	The material uses simple tables with short and clear row and column headings (P and A/V)	1.00	86	0.49	0.80	d	d
	Actionability
20	The material clearly identifies at least one action the user can take (P and A/V)	1.00	74	0.35	0.67	0.71	0.88	(+)
21	The material addresses the user directly when describing actions (P and A/V)	1.75	88	0.60	0.86	0.85	0.81
22	The material breaks down any action into manageable, explicit steps (P and A/V)	1.25	76	0.52	0.68	0.69	0.89	(+)
23	The material provides a tangible tool (e.g., menu planners, checklists) whenever it could help the user take action (P)	1.50	86	0.71	0.72	0.83 (P)	0.84 (P)	(+)
24	The material provides simple instructions or examples of how to perform calculations (P)	1.50	86	0.76	0.80	d	d	(+)
25	The material explains how to use the charts, graphs, tables, or diagrams to take actions (P and A/V)	1.25	81	0.47	0.77	d	d
26	The material uses visual aids whenever they could make it easier to act on the instructions (P)	1.25	81	0.64	0.74	0.85 (P)	0.77 (P)	(+)

^aAverage from 1 to 3; Yes = 1, Maybe = 2, and No = 3.

^b(+) indicates qualitative findings from consumer testing that supported the item/criterion.

^cAll internal consistency results presented are for both material types unless specified: print (P) or audiovisual (A/V).

^dInternal consistency could not be calculated because too few materials received ratings on the item.

^eItem was added after content validation was completed.