Table 2.
Content validitya | External consistency (inter-rater reliability) | Internal consistency | Construct validity | |||||
---|---|---|---|---|---|---|---|---|
|
|
|||||||
Item # | Item (relevant for printable [P] and/or audiovisual [A/V] materials) | Average (1–3) | % Agree | Kappa | Gwet’s AC1 | Cronbach’s α | Item-total correlation | Qualitative Findingsb |
Understandability | ||||||||
1 | The material makes its purpose completely evident (P and A/V) | 1.00 | 88 | 0.71 | 0.80 | 0.68 | 0.62 | (+) |
2 | The material does not include information or content that distracts from its purpose (P) | 1.25 | 81 | 0.48 | 0.70 | 0.80 (P)c | 0.62 (P) | (+) |
3 | The material uses common, everyday language (P and A/V) | 1.00 | 81 | 0.57 | 0.66 | 0.66 | 0.70 | (+) |
4 | Medical terms are used only to familiarize audience with the terms. When used, medical terms are defined (P and A/V) | 1.25 | 81 | 0.54 | 0.68 | 0.66 | 0.73 | (+) |
5 | The material uses the active voice (P and A/V) | 1.00 | 88 | 0.40 | 0.85 | 0.75 | 0.31 | |
6 | Numbers appearing in the material are clear and easy to understand (P) | 1.00 | 81 | 0.55 | 0.76 | 0.79 (P) | 0.79 (P) | |
7 | The material does not expect the user to perform calculations (P) | 1.00 | 86 | 0.69 | 0.74 | 0.82 (P) | 0.32 (P) | (+) |
8 | The material breaks or “chunks” information into short sections (P and A/V) | 1.00 | 79 | 0.56 | 0.72 | 0.70 | 0.55 | (+) |
9 | The material’s sections have informative headers (P and A/V) | 1.00 | 83 | 0.70 | 0.77 | 0.69 | 0.60 | (+) |
10 | The material presents information in a logical sequence (P and A/V) | 1.25 | 86 | 0.42 | 0.81 | 0.67 | 0.68 | |
11 | The material provides a summary (P and A/V) | 1.75 | 69 | 0.46 | 0.58 | 0.73 | 0.41 | (+) |
12 | The material uses visual cues (e.g., arrows, boxes, bullets, bold, larger font, highlighting) to draw attention to key points (P and A/V) | 1.00 | 74 | 0.51 | 0.65 | 0.71 | 0.52 | (+) |
13 | Text on the screen is easy to read (A/V) | e | 71 | 0.40 | 0.63 | 0.77 (A/V) | 0.50 (A/V) | (+) |
14 | The material allows the user to hear the words clearly (e.g., not too fast, not garbled) (A/V) | 1.00 | 86 | 0.66 | 0.82 | 0.76 (A/V) | 0.53 (A/V) | (+) |
15 | The material uses visual aids whenever they could make content more easily understood (e.g., illustration of healthy portion size) (P) | 1.00 | 76 | 0.48 | 0.56 | 0.80 (P) | 0.59 (P) | (+) |
16 | The material’s visual aids reinforce rather than distract from the content (P) | 1.00 | 86 | 0.70 | 0.81 | 0.81 (P) | 0.56 (P) | (+) |
17 | The material’s visual aids have clear titles or captions (P) | 1.25 | 90 | 0.84 | 0.86 | 0.81 (P) | 0.51 (P) | (+) |
18 | The material uses illustrations and photographs that are clear and uncluttered (P and A/V) | 1.25 | 88 | 0.67 | 0.85 | 0.74 | 0.34 | |
19 | The material uses simple tables with short and clear row and column headings (P and A/V) | 1.00 | 86 | 0.49 | 0.80 | d | d | |
Actionability | ||||||||
20 | The material clearly identifies at least one action the user can take (P and A/V) | 1.00 | 74 | 0.35 | 0.67 | 0.71 | 0.88 | (+) |
21 | The material addresses the user directly when describing actions (P and A/V) | 1.75 | 88 | 0.60 | 0.86 | 0.85 | 0.81 | |
22 | The material breaks down any action into manageable, explicit steps (P and A/V) | 1.25 | 76 | 0.52 | 0.68 | 0.69 | 0.89 | (+) |
23 | The material provides a tangible tool (e.g., menu planners, checklists) whenever it could help the user take action (P) | 1.50 | 86 | 0.71 | 0.72 | 0.83 (P) | 0.84 (P) | (+) |
24 | The material provides simple instructions or examples of how to perform calculations (P) | 1.50 | 86 | 0.76 | 0.80 | d | d | (+) |
25 | The material explains how to use the charts, graphs, tables, or diagrams to take actions (P and A/V) | 1.25 | 81 | 0.47 | 0.77 | d | d | |
26 | The material uses visual aids whenever they could make it easier to act on the instructions (P) | 1.25 | 81 | 0.64 | 0.74 | 0.85 (P) | 0.77 (P) | (+) |
Average from 1 to 3; Yes = 1, Maybe = 2, and No = 3.
(+) indicates qualitative findings from consumer testing that supported the item/criterion.
All internal consistency results presented are for both material types unless specified: print (P) or audiovisual (A/V).
Internal consistency could not be calculated because too few materials received ratings on the item.
Item was added after content validation was completed.