Abstract
Ensuring that written materials used in behavioral interventions are clear is important to support behavior change. This study used the Clear Communication Index (CCI) to assess the original and revised versions of three types of written participant materials from the SIPsmartER intervention. Materials were revised based on original scoring. Scores for the entire index were significantly higher among revised versions than originals (57% versus 41%, P<.001); however, few revised materials (n=2 of 53) achieved the benchmark of ≥90%. Handouts scored higher than worksheets and slide sets for both versions. The proportion of materials scored as having “a single main message” significantly increased between versions for worksheets (7% to 57%, P=.003) and slide sets (33% to 67%, P=.004). Across individual items, most significant improvements were in Core, with four-items related to the material having a single main message. Findings demonstrate that SIPsmartER’s revised materials improved after CCI-informed edits. They advance the evidence and application of the CCI, suggesting it can be effectively used to support improvement in clarity of different types of written materials used in behavioral interventions. Implications for practical considerations of using the tool and suggestions for modifications for specific types of materials are presented.
Keywords: communication, health communication, information dissemination, medically underserved area, adult
Introduction
Behavioral health interventions communicate information as a means to build the motivation and skills necessary to change targeted health behaviors and to improve health-related health outcomes (Contento, 2010). Therefore, these interventions target individuals and populations more likely to engage in preventable unhealthful behaviors and have a higher rate of health conditions. These targeted populations are at higher risk for having low health literacy (Institute of Medicine, 2004). Low health literate individuals have compromised abilities to access, understand, and act on health information (Institute of Medicine, 2004), and these skills are needed to drive behavior change. Therefore, given this risk for lower health literacy among those targeted for behavioral interventions, it is imperative that written materials used in behavioral interventions must be clear.
There are guidelines, such as the Federal Plain Language Guidelines (plainlanguage.gov, 2011), that describe attributes that clear written materials should contain, and there are also toolkits, such as the Universal Health Literacy Precautions Toolkit (Brega et al., 2015), that support the development of clearly written materials. Additionally, there are tools that researchers and interventionists can use to actually assess the clarity of their written materials before or after considering guidelines or toolkits. While early tools provided guidance for estimating the grade-level (e.g., Flesch Reading Ease Index (Flesch, 1974); Simplified Measure of Gobbledygook (Hedman, 2008)), recently developed tools have moved beyond a singular focus on readability and are more sophisticated. These tools include the Health Literacy INDEX (Kaphingst et al., 2012), Patient Education Materials Assessment Tool (Shoemaker, Wold, & Brach, 2013), and the Centers for Disease Control and Prevention’s Clear Communication Index (CCI) (Baur & Prue, 2014). Each assesses different aspects of communication, including identifying the target audience, readability, material design, actionability, and/or numeracy. However, only the CCI incorporates all these aspects of communication.
The CCI was designed using best practices from the fields of health communication, risk communication, health and science literacy, and behavioral sciences (Baur & Prue, 2014). In addition to its comprehensiveness, the CCI’s scoring system allows for reduced subjectivity in the assessment. The CCI’s design allows it to be used to assess written health communication materials designed for audiences ranging from patients to health professionals. These characteristics make the CCI a very suitable and comprehensive tool to assess written materials used in behavioral interventions as these materials need to clearly express scientific basis, risk, and how to engage in more healthful behaviors.
In addition to the CDC using the CCI to assess and revising its own materials, there are five known studies that have reported using the CCI. These studies evaluated a patient electronic portal (Alpert, Desens, Krist, Aycock, & Kreps, 2017), education materials for sickle cell disease (McClure, Ng, Vitzthum, & Rudd, 2016), water quality reports (Phetxumphou et al., 2016), internet information on meningiomas (Saeed & Anderson, 2017), and a web-based toolkit (Prue et al., 2015). Notably, these studies all focused on materials designed to be used with little interaction between provider/educator and patient/participant, and none reported revising and/or re-assessing the clarity of the materials after the revision.
In the spring of 2017, the CCI was used to assess and inform revisions to the written materials used in the SIPsmartER intervention prior to its dissemination and implementation trial through health department offices. SIPsmartER is an effective, 6-month, community-based behavioral intervention that effectively reduces sugar-sweetened beverage (SSB) consumption among rural Appalachian adults (Zoellner et al, 2016). The purpose of this study is to describe changes in SIPsmartER’s written materials – handouts, slide sets, and worksheets – between original and revised versions and explore differences in scoring and changes in scores by material type. Though SIPsmartER is an intervention focused on reducing sugar-sweetened beverage (SSB), the CCI methods and interpretations of findings have broad application across all interventions that use written materials to improve patient/participant behaviors, regardless of behavioral target.
Methods
SIPsmartER and its Written Materials
Participants in SIPsmartER significantly reduced SSB intake compared to those in a matched-contact comparison group (−19 ounces versus −5 ounces of SSB per day, P<.001) (Zoellner et al., 2016). In a recent meta-analysis of the effect of interventions targeting sugary drink intake, SIPsmartER demonstrated the largest mean difference had the largest mean difference in intake between intervention and control groups of the twelve interventions targeting adults (Vargas-Garcia et al., 2017).
Guided by the Theory of Planned Behavior (Ajzen, 1991) and health literacy strategies (Brega et al., 2015), SIPsmartER consists of three small-group classes, one live teach-back call, and 11 interactive voice response calls (Porter et al., 2016, Zoellner et al., 2014; Zoellner et al., 2016). The group classes, consisting of two to ten participants, were each approximately two hours long. During these classes, participants learned and applied new content and skills (e.g., health risks of sugary drinks, tracking sugary drink intake), discussed barriers and strategies to reducing their own sugary drink intake, and completed a personal action planning process. These activities were reinforced with three types of written materials: handouts, slide sets, and worksheets. These materials are used in or out of class time and were designed with varying levels of anticipated educator/participant interaction. If a participant missed a group class, these materials were mailed to them and reviewed with research staff during a missed class call (Zoellner et al., 2014). Table 1 provides a detailed description of each material.
Table 1.
Description of written materials used in SIPsmartER
| Material Type | # | Purpose | Content | Participant/ Educator Interaction Level |
Design | Example Material Names | Possible Pointsa |
|---|---|---|---|---|---|---|---|
| Handout | 12 | To provide participants with a reference they can use outside of class and to complete personal action plans | Motivation and strategies for overcoming common barriers to drinking fewer sugary drinks | Limited | ■ Color ■ Full-page ■ Double-sided page (original), Single-sided page (revised) |
■ How to break a sugary drink habit ■ How to make non-sugary drinks convenient ■ How to get loved ones on board |
14.00 (1.81) |
| Worksheets | 14 | To provide participants with a means to complete activities and/or record personal information during or after class | Reflect lesson content | Medium or High | ■ Color ■ Single-sided page |
■ Personal action plan ■ Reading food labels ■ How much can you save? |
11.43 (1.09) |
| Slide Sets | 27 | To provide participants with visuals to reinforce lesson content | Key lesson messages in written and pictorial. Revised slides also provide step-by-step examples of how to complete worksheets | High | ■ Color ■ 1 to 39 slides per set (average 10) |
■ What do I drink? ■ Advertising is everywhere ■ Nutrition label reading activities |
12.70 (2.23) |
Average number of the potential 20 scored CCI items that were applicable for each type of material for original versions of materials. The total possible scores did not significantly differ by type of material between original and revised versions. For both original and revised versions, handouts had significantly higher possible scores than the worksheets
Clear Communication Index
The CCI consists of 24 items. The first four items are unscored and open-ended items; they allow the reviewer to identify the needs of the audience, primary communication objective(s), and main message(s). The remaining 20 items are scored and organized into four parts: (i) Core, which addresses main message, language, information design, and state of science, (ii) Behavioral Recommendations, (iii) Numbers, and (iv) Risk (Baur & Prue, 2014). Items in Core are scored for all materials, while the other three parts are only scored if a written material includes the specific content. All items except one are scored Yes=1 or No=0. Materials scoring ≥90% on applicable questions are considered to be clear; those scoring less should be revised (Baur & Prue, 2014).
Coding Timeline and Revisions of SIPsmartER Materials
Two of the authors (RA and NK) informally coded the original versions of the materials with the CCI between June and September of 2015. The general findings from this process were used to guide revisions to SIPsmartER’s written materials between September 2015 and July 2016. After reviewing the revised products, we recognized that a systematic CCI coding to compare the original and revised materials would be beneficial to advance evidence and application of the CCI. This systematic coding, described below, occurred in the spring of 2017.
Coding Procedure
The lead author had been trained to use the CCI during a 90-minute, in-person training conducted by one of the index’s developers. She trained the other coders by reviewing the CCI coding guidelines (Centers for Disease Control and Prevention, 2015) in detail with them, having them independently assess the original and revised versions of six materials (~10% of the materials), and meeting as a group to review the coding, discuss discrepancies, and recap the CCI coding guidelines. The intra-class correlation for this coding was 0.923.
Each of the remaining materials were coded by two reviewers. KMP and NK or KJP coded original versions while KMP and RA coded revised versions. Coders met to resolve discrepancies three times, or once for each type of material. To resolve discrepancies, coders identified differences, discussed rationale for decisions, reviewed scoring guidelines, and came to agreement. If agreement could not be reached, KJP or NK were brought in to break the stalemate. Kappa scores were calculated.
Handouts and worksheets were coded as individual materials. Slide sets were coded in sets as multiple slides were used together to convey a message.
Data Analysis
Data were entered into SPSS 24.0 (2016, Aramonk, NY). We calculated proportional scores (i.e., the percentage of possible points earned) for the overall index and for each of its four parts. Based on the overall proportional scores, we coded materials as achieving or not achieving an overall score of ≥90%. We also tabulated the proportion of original and revised materials with desirable scores for each of the 20 individual items.
To identify differences in proportional scores between versions, we conducted paired t-tests for overall index, Core, and individual Core items and independent t-tests for Behavioral Recommendations, Numbers, and Risk parts as well as their individual items. Independent t-tests were necessary for these parts/items as not all materials were scored for these parts in both the original and revised versions. For both versions, one-way ANOVAs, with Tukey post-hoc tests, were conducted to assess differences in overall and part scores by material type.
Results
We coded the original and revised versions of 53 materials: 12 (19%) handouts, 14 (26%) worksheets, and 27 (51%) slide sets. Reliability among coders was substantial (κ=0.70 to 0.83). Changes in CCI proportional scores for each material as a whole and by part are presented in Table 2, while changes in the proportion of materials with a desirable score for individual index items are presented in Table 3
Table 2.
Overall and part proportional scores by material type
| All (n=53) |
Handouts (n=12) |
Worksheets (n=14) | Slide sets (n=27) | Sigx | |||
|---|---|---|---|---|---|---|---|
| Proportional Score by Complete Index | |||||||
| Overall Proportional Score | Original | Mean (SD) | .41 (.27) | .77 (.12)a | .17 (.12)b | .38 (.21)c | <.001 |
| Revised | Mean (SD) | .57 (.23) | .82 (.09)a | .46 (.23)b | .52 (.20)b | <.001 | |
| Changey | Sig | <.001 | NS | .002 | .002 | ||
| % with score ≥ 90% | Original | % (n) | 2% (1) | 8% (1) | 0% (0) | 0% (0) | NS |
| Revised | % (n) | 4% (2) | 17% (2) | 0% (0) | 0% (0) | .029 | |
| Changey | Sig | NS | NS | -- | -- | ||
| Proportional Score by Index Part | |||||||
| Core | Original | Mean (SD) | .39 (.29) | .79 (.14)a | .15 (.10)b | .33 (.21)c | <.001 |
| Revised | Mean (SD) | .56 (.25) | .83 (.12)a | .46 (.22)b | .48 (.21)b | <.001 | |
| Changey | Sig | <.001 | NS | <.001 | <.001 | ||
| Behavioral Recommendation | Original | n | 13 | 10 | 0 | 3 | NS |
| Mean (SD) | .64 (.16) | .67 (.00) | -- | .56 (.38) | |||
| Revised | n | 13 | 9 | 0 | 4 | .030 | |
| Mean (SD) | .62 (.23) | .70 (.20) | -- | .42 (.17) | |||
| Changez | Sig | NS | NS | -- | .011 | ||
| Numbers | Original | n | 15 | 2 | 2 | 11 | NS |
| Mean (SD) | .89 (.16) | 1.00 (.00) | .83 (.24) | .88 (.17) | |||
| Revised | n | 25 | 6 | 6 | 13 | .004 | |
| Mean (SD) | .83 (.20) | .89 (.17)a | .61 (.14)b | .90 (.16)a | |||
| Changez | Sig | NS | NS | NS | NS | ||
| Risk | Original | n | 4 | 0 | 0 | 4 | -- |
| Mean (SD) | .75 (.50) | -- | -- | .75 (.50) | |||
| Revised | n | 4 | 0 | 1 | 3 | -- | |
| Mean (SD) | 1.00 (.00) | -- | 1.00 (--) | 1.00 (.00) | |||
| Changez | Sig | NS | -- | -- | NS | ||
one-way ANOVA
paired t-test
independent t-test
Table 3.
Proportion of materials with desirable score for each CCI item
| All | Handouts | Worksheets | Slide Sets | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Original | Revised | Original | Revised | Original | Revised | Original | Revised | ||
| Core | n | 53 | 53 | 12 | 12 | 14 | 14 | 27 | 27 |
| Item 1: One main message | % (n) | 40% (21) | 72% (38)* | 92% (11) | 100% (12) | 7% (1) | 57% (8)* | 33% (9) | 67% (18)* |
| Item 2: Message at front | % (n) | 26% (14) | 57% (30)* | 92% (11) | 100% (12) | 0% (0) | 57% (8)* | 11% (3) | 37% (10)* |
| Item 3: Visual cues | % (n) | 36% (19) | 68% (36)* | 92% (11) | 92% (11) | 0% (0) | 57% (8)* | 30% (8) | 63% (17)* |
| Item 4: Supportive images | % (n) | 28% (15) | 68% (36)* | 33% (4) | 92% (11)* | 7% (1) | 57% (8)* | 37% (10) | 70% (19)* |
| Item 5: Call to action | % (n) | 30% (16) | 38% (20) | 100% (12) | 92% (11) | 7% (1) | 14% (2) | 11% (3) | 26% (7) |
| Item 6: Active voice | % (n) | 26% (14) | 30% (16) | 92% (11) | 84% (10) | 0% (0) | 7% (1) | 11% (3) | 19% (5) |
| Item 7: Simple language | % (n) | 94% (50) | 98% (52) | 100% (12) | 92% (11) | 79% (11) | 100% (14) | 100% (27) | 100% (27) |
| Item 8: Bullet/number lists | % (n) | 53% (28) | 49% (26) | 100% (12) | 92% (11) | 14% (2) | 29% (4) | 52% (14) | 41% (11) |
| Item 9: Headings | % (n) | 70% (70) | 89% (47)* | 100% (12) | 92% (11) | 50% (7) | 71% (10) | 67% (18) | 96% (26)* |
| Item 10: Summary at top | % (n) | 17% (9) | 43% (23)* | 50% (6) | 84% (10) | 0% (0) | 71% (10)* | 11% (3) | 11% (3) |
| Item 11: State of science | % (n) | 4% (2) | 0% (0) | 17% (2) | 0% (0) | 0% (0) | 0% (0) | 0% (0) | 0% (0) |
| Behavioral Recommend-ations | n | 13 | 13 | 10 | 9 | 0 | 0 | 3 | 4 |
| Item 12: Behavioral recommend | % (n) | 100% (13) | 100% (13) | 100% (10) | 100% (9) | -- | -- | 100% (3) | 100% (4) |
| Item 13: Explain importance | % (n) | 8% (1) | 23% (3) | 0% (0) | 17% (2) | -- | -- | 33% (1) | 25% (1) |
| Item 14: Directions how-to | % (n) | 85% (11) | 62% (8) | 100% (10) | 89% (8) | -- | -- | 33% (1) | 0% (0) |
| Numbers | n | 15 | 25 | 2 | 6 | 2 | 6 | 11 | 13 |
| Item 15: Presentation | % (n) | 100% (15) | 100% (15) | 100% (2) | 100% (6) | 100% (2) | 100% (2) | 100% (11) | 100% (13) |
| Item 16: Explain meaning | % (n) | 100% (15) | 80% (20)* | 100% (2) | 67% (4) | 100% (2) | 50% (3) | 100% (11) | 100% (13) |
| Item 17: No calculations | % (n) | 67% (10) | 68% (17) | 100% (2) | 100% (6) | 50% (1) | 33% (2) | 64% (7) | 70% (9) |
| Risk | na | 4/0/0 | 4/1/0 | 0/0/0 | 0/0/0 | 0/0/0 | 1/0/0 | 4/0/0 | 3/1/0 |
| Item 18: Explain risk | % (n) | 75% (3) | 100% (4) | -- | -- | -- | 100% (1) | 75% (3) | 100% (3) |
| Item 19: Risks & benefits | % (n) | -- | 100% (1) | -- | -- | -- | -- | -- | 100% (1) |
| Item 20: Explain probability | % (n) | -- | -- | -- | -- | -- | -- | -- | -- |
number of items eligible for Item 18/ Item 19/ Item 20
significant difference (P<.05) between original and revised materials determined by paired or independent t-test
Overall Proportional Scores
Across all materials, there was a significant increase in the overall proportional scores between the original and revised version (41% to 57%, P<.001). Worksheets and slide sets had significant increases in proportional scores after revisions (17% to 46%, P=.002 and 38% to 52%, P=.002, respectively), while handouts did not (77% to 82%). The proportion of revised materials achieving a score of ≥90% was 4% (n=2), and only handouts achieved this benchmark.
Core
As presented in Table 2, overall proportional scores for Core increased significantly across all materials between original and revised versions (39% to 56%, P<.001). Worksheets and slide sets saw significant improvement in scores between versions; however, the revised versions of both these materials had proportional scores less than 50%.
Handouts had significantly higher Core proportional scores than worksheets and slide sets for both the original and revised versions (P<.001). Original versions of slide sets had significantly higher proportional scores than worksheets (P=.007) while the revised versions were not statistically different.
Across all materials, there were significant increases in proportion of materials having desirable scores for six of these eleven individual Core items (Table 3). Worksheets and slide sets each significantly improved in five of these items while there were no significant changes for handouts. Worksheets and slide sets both had significant improvements related to the presence of a single main message (items 1 through 4). This increase is due in large part to the revised versions including clear written messages in the material (item 1) as the incorporation of this item allowed items 2, 3, and 4 to be scored. For many of the original versions of these two materials, there were single communication objectives; however, they were not explicitly written out, instead the main message had to be assumed. These revisions included written statements about why the information collected through the worksheets was important as well as opening or closing messages for slide sets to allow their purpose to be clearly stated. Lastly, materials across types and versions scored very low in relation to the explanation of the state of science (item 11).
Behavioral Recommendations
Behavioral recommendations were included in 13 handouts and slide sets. While total scores across materials did not significantly change between versions, proportional scores for slide sets significantly decreased (56% to 42%, P=.01, Table 2). Revised handouts had a significantly higher proportional score for this domain than revised slide sets (70% versus 42%, P=.03). There are non-significant but notable changes in two of the individual items in this part (Table 3): description of the behavior’s importance (item 13, increase from 8% to 23%) and inclusion of how-to directions (item 14, decrease from 85% to 62%). Improved descriptions of the behavior’s importance were included in the revised handouts by clearly stating on the sheet that the how-to tips for overcoming the specific barrier to drinking fewer SSBs would help the participant’s health and well-being. In the original version, this connection to health and well-being was, for the most part, assumed.
Numbers
Numbers were used to emphasize the explicit main message of 15 original materials and 25 revised materials. These messages ranged from “you can overcome the size barrier and drink fewer SSBs” to “you can save money by drinking fewer SSBs” to “using a food label can help you identify how much sugar is in a SSB.” The number of materials significantly increased between versions (P<.05).
There were no significant differences in proportional score for this part between original and revised materials overall and by material type; however, scores appear to decrease (Table 2). Revised handouts and slide sets had significantly higher scores than worksheets (89% and 90% versus 61%, P=.02 and P=.004, respectively). Among individual items (Table 3), the proportion of materials that explained the meaning of the numbers was significantly lower among revised materials compared to original materials (item 16, 100% to 80%, P=0.2). Additionally, though non-significant, there were decreases in the proportion of materials that did not require math to be conducted (item 17).
Risk
Risk was included in four original materials (slide sets) and four revised materials (one worksheet and three slide sets). These materials highlight the health risks and potential financial costs of consuming too many SSBs. The majority of materials were only eligible to be coded for explanation of risk (item 18). Revised materials had a proportional score of 100%. Due to the small number of eligible materials, it was not possible to statistically compare types of materials.
Discussion
Using the CCI to revise SIPsmartER’s written materials led to improvements in their clarity. Significant changes were found across all materials and, when looking by type of material, for slide sets and worksheets. Ensuring the clarity of written materials is a critical step in any intervention, regardless of behavioral focus, that uses written materials to improve patient/participant behaviors. In the context of this behavioral intervention, focusing on SSBs, is important because of the ubiquity and negative public health impact of SSBs. SSBs contribute approximately 7% of adults’ daily calorie intake, and high intake of SSBs is linked to numerous health conditions, including obesity, diabetes, cardiovascular disease, and dental caries (Malik, Pan, Willett, & Hu, 2013; Malik et al., 2010; Tahmassebi, Duggal, Malik-Kotru, & Curzon, 2006). Additionally, SSB intake is disproportionately high among adults with low health literate skills (Zoellner et al., 2011). These improvements in material clarity are all the more important because of the twelve studies targeting SSB intake among adults that were included in a recent meta-analysis, only SIPsmartER and another trial demonstrated effectiveness (Ostbye et al., 2012; Vargas-Garcia et al., 2017; Zoellner et al., 2016). To our knowledge none of these other 11 SSB studies applied health literacy or clear communication concepts in the design of their intervention materials.
Findings from this study also advance the evidence and application of the CCI related to its use for guiding changes in written materials and for assessing materials that are implemented with greater patient/participant and provider/educator interaction, specifically worksheets and slide sets. When considering individual items, many of the significant changes are due to the increase of materials with one explicit main message. Proportion of worksheets and slide sets with an explicit main message increased from 7% to 57% and 33% to 67%, respectively. By increasing the number of materials with a main message, four more items in Core were eligible to be scored as “Yes” in the revised versions (Centers for Disease Control and Prevention, 2015). This assumption is supported by the improvements in three of these items for both of these types of materials: message at front (item 2), visual cues (item 3), and supportive images (item 4).
Also, our experience coding and revising materials using the CCI identified four practical suggestions that may enhance the thoughtfulness and efficiency of this process, which can be time and resource consuming. First, using the CCI to assess, revise, or develop written materials is very important even if inclusion of clear communications principles is a priority. Without a tangible tool, these considerations are just driven by curriculum developer judgement and intuition, and there is no means to objectively assess them. Also, using the CCI extends considerations for clarity beyond the language aspect and to include presentation of numbers, risk, and behavioral recommendations. This impact of only using intuition is evident in SIPsmartER’s original materials, which were purposefully developed with clear communications principles in mind, but not with an objective tool (Zoellner et al., 2014). For example, while these considerations did allow SIPsmartER’s original handouts to have higher overall proportional scores than other similar materials that have had their overall scores reported (77% versus a range of 57% to 72% (Alpert et al., 2017; McClure et al., 2016; Saeed & Anderson, 2017)), the proportional scores of the original materials did not meet the 90% benchmark.
Second, after completing this process, we recommend recording specific suggestions for material revisions during the initial coding process instead of solely relying on scores. The close reading of materials that occurs when scoring materials identifies specific weaknesses that might not be recaptured when making changes. Although the revised SIPsmartER materials improved, scores may have been higher had there been a specific list of changes for each material created at the time of coding the original materials. Currently, the CCI scoring sheet does not provide space for these comments to be recorded.
Third, when using the CCI to revise materials, it is important to “unofficially” score items in Numbers and Risk, even if these parts are not technically eligible for scoring due to lacking an explicit main message. Not reviewing and noting about these parts may hinder the improvement of materials as items in these parts that would need revision once a main message was present would not have be assessed. This impact is seen in the decrease of Numbers proportional score for worksheets. As the worksheet content did not change, this decrease was driven by the increase in number of items scored due to the increase in number of worksheets with a main message. If these materials had been eligible to be scored for Numbers during the original round of coding, these areas needing improvement would have been noted.
Fourth, it might be pragmatic to consider scoring modifications for materials which are intentionally designed to be used in conjunction with participant-educator interaction, such as worksheets and slide sets, and for materials that are part of a larger evidenced-based intervention delivered by an authoritative source. The CCI has already been modified to better meet the needs of short form and oral communications (Centers for Disease Control and Prevention, 2014). Specifically, these modifications may include (i) allowing materials designed to always be used with high participant-educator interaction (e.g., worksheets and slide sets) to be considered as having a main message (item 1) if there is a clear, single communication objective and (ii) allowing an “n/a” reply for state of the science (item 11). Regarding the former modification, though proportion of materials with an explicit main message (item 1) increased significantly among revised worksheets and slide sets between version, only 57% and 67% were scored as “yes;” however, all had single clear communication objective and all were completed and/or reviewed with the educators. This modification would also open up opportunities for four other items with Core to be scored as “yes” as well as, if applicable, allow Numbers and Risk to be scored. Regarding the second possible modification, information about the state of science was left off SIPsmartER’s materials to make the materials less technical while knowing that the curriculum content was informed by the literature and the educators who deliver the same content in the class were the subject matter experts. These modifications may allow for more materials to achieve the 90% benchmark without impacting their overall clarity.
Limitations
Findings from this study are limited by two notable factors. First, the sample size of materials scored for Behavioral Recommendations, Numbers, and Risk is small and limits the power to statistically compare changes in materials for these parts. Second, two of the authors (KJP and NK) were both a part of material assessment and revision, which could have added bias. To reduce bias, KJP and NK only assessed original materials with a second coder who did not have a role with material revision. Additionally, Kappa scores indicate high interrater reliability between NK/KMP and KJP/KMP.
Implications and Conclusion
The use of the CCI to revise SIPsmartER’s written materials has allowed them to become clearer which may allow it to better impact behaviors by making it easier for participants to access, understand, and act on the behavioral messages in the materials. Though this study was conducted within the context of an SSB intervention, the findings from this study have broad generalizability related to the use of the CCI to assess and improve the clarity of written materials in behavioral interventions targeting other health behaviors. Specifically, this study advances the evidence and application of the CCI as a tool to enhance written material clarity when developing and revising different types of written materials, even when materials are developed with the intention of incorporating clear communication techniques. Also, findings suggest modifications to the CCI scoring criteria may be necessary to allow for consideration of difference in due to the anticipated level of interaction between patients/participants and provider/educator (e.g., how the single main message is determined). However, more research is needed to create standardized CCI scoring criteria for these types of materials, such that exists for short form (e.g., text messages) and oral communication materials (Centers for Disease Control and Prevention, 2014).
Acknowledgments
This research was funded by NIH/NCI 1R01CA154364–01A1 (PI: Zoellner). The authors acknowledge the contribution of Talking Health team members who have been instrumental in the development and implementation of the trial.
Contributor Information
Kathleen J. Porter, Department of Public Health Sciences, University of Virginia.
Ramine Alexander, Department of Nutrition, University of North Carolina.
Katelynn M. Perzynski, Department of Public Health Sciences, University of Virginia.
Natalie Kruzliakova, Department of Human Nutrition, Foods and Exercise, Virginia Tech.
Jamie M. Zoellner, Department of Public Health Sciences, University of Virginia.
References
- Ajzen I (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179–211. doi: 10.1016/0749-5978(91)90020-T [DOI] [Google Scholar]
- Alpert JM, Desens L, Krist AH, Aycock RA, & Kreps GL (2017). Measuring health literacy levels of a patient portal using the CDC’s Clear Communication Index. Health Promotion and Practice, 18, 140–149. doi: 10.1177/1524839916643703 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baur C, & Prue C (2014). The CDC Clear Communication Index is a new evidence-based tool to prepare and review health information. Health Promotion and Practice, 15, 629–637. doi: 10.1177/1524839914538969 [DOI] [PubMed] [Google Scholar]
- Brega AG, Barnard J, Mabachi NM, Weiss BD, DeWalt DA, Brach C, West DR (2015). Health literacy universal precautions toolkit (HHSA290200710008, TO#10). Rockville, MD: Agency for Healthcare Research and Quality. [Google Scholar]
- Centers for Disease Control and Prevention. (2014). Modified CDC Clear Communications Index score sheet. Retrieved from https://www.cdc.gov/ccindex/pdf/modified-index-scoresheet.pdf
- Centers for Disease Control and Prevention. (2015). Clear Communication Index user guide. Retrieved from https://www.cdc.gov/ccindex/tool/index.html
- Contento IR (2010). Nutrition education: Linking research, theory, and practice (2nd ed.). Sudbury, MA: Jones & Bartlett Learning. [PubMed] [Google Scholar]
- Flesch RF (1974). The art of readable writing. New York, NY: Harper Row. [Google Scholar]
- Hedman AS (2008). Using the SMOG formula to revise a health-related document. American Journal of Health Education, 39, 61–64. doi: 10.1080/19325037.2008.10599016 [DOI] [Google Scholar]
- Institute of Medicine. (2004). Health literacy: A prescription to end confusion. Washington, DC: The National Academies Press. [PubMed] [Google Scholar]
- Kaphingst KA, Kreuter MW, Casey C, Leme L, Thompson T, Cheng MR, Lapka C (2012). Health literacy INDEX: Development, reliability, and validity of a new tool for evaluating the health literacy demands of health information materials. Journal of Health and Community, 17, 203–221. doi: 10.1080/10810730.2012.712612 [DOI] [PubMed] [Google Scholar]
- Malik VS, Pan A, Willett WC, & Hu FB (2013). Sugar-sweetened beverages and weight gain in children and adults: A systematic review and meta-analysis. American Journal of Clinical Nutrition, 98, 1084–1102. doi: 10.3945/ajcn.113.058362 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Malik VS, Popkin BM, Bray GA, Despres JP, Willett WC, & Hu FB (2010). Sugar-sweetened beverages and risk of metabolic syndrome and type 2 diabetes: A meta-analysis. Diabetes Care, 33, 2477–2483. doi: 10.2337/dc10-1079 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McClure E, Ng J, Vitzthum K, & Rudd R (2016). A mismatch between patient education materials about sickle cell disease and the literacy level of their intended audience. Preventing Chronic Disease, 13, E64. doi: 10.5888/pcd13.150478 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ostbye T, Krause KM, Stroo M, Lovelady CA, Evenson KR, Peterson BL, . . . Zucker NL (2012). Parent-focused change to prevent obesity in preschoolers: Results from the KAN-DO study. Preventive Medicine, 55, 188–195. doi: 10.1016/j.ypmed.2012.06.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Phetxumphou K, Roy S, Davy BM, Estabrooks PA, You W, & Dietrich AM (2016). Assessing clarity of message communication for mandated USEPA drinking water quality reports. Journal of Water and Health, 14, 223–235. doi: 10.2166/wh.2015.134 [DOI] [PubMed] [Google Scholar]
- PlainLanguage.gov. (2011). Federal plain language guidelines. Retrieved from http://www.plainlanguage.gov/howto/guidelines/FederalPLGuidelines/index.cfm?CFID=4820890&CFTOKEN=dd548e2c774a0912-8D257A7F-F1C4-6400-B63471316DCC72E9&jsessionid=2B45C8E3A03B7CF25B5898AA846D7278.chh
- Porter K, Chen Y, Estabrooks P, Noel L, Bailey A, & Zoellner J (2016). Using teach-back to understand participant behavioral self-monitoring skills across health literacy level and behavioral condition. Journal of Nutrition Education and Behavior, 48, 20–26 e21. doi: 10.1016/j.jneb.2015.08.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Prue C, Raber A, Mothershed E, Cairns C, Sperber B, Meinhold L, Penn M (2015). Using the Clear Communication Index to review a web-based toolkit. Paper presented at the National Conference on Health Communication, Marketing, & Media, Atlanta, GA Retreived from https://cdc.confex.com/cdc/nphic15/webprogram/Paper36486.html [Google Scholar]
- Saeed F, & Anderson I (2017). Evaluating the quality and readability of internet information on meningiomas. World Neurosurgery, 97, 312–316. doi: 10.1016/j.wneu.2016.10.001 [DOI] [PubMed] [Google Scholar]
- Shoemaker SJ, Wold MS, & Brach C (2013). The patient education materials assessment tool (PEMAT) and user’s guide. Retrieved from https://www.ahrq.gov/professionals/prevention-chronic-care/improve/self-mgmt/pemat/index.html [DOI] [PMC free article] [PubMed]
- Tahmassebi JF, Duggal MS, Malik-Kotru G, & Curzon ME (2006). Soft drinks and dental health: A review of the current literature. Journal of Dentistry, 34, 2–11. doi: 10.1016/j.jdent.2004.11.006 [DOI] [PubMed] [Google Scholar]
- Vargas-Garcia EJ, Evans CEL, Prestwich A, Sykes-Muskett BJ, Hooson J, & Cade JE (2017). Interventions to reduce consumption of sugar-sweetened beverages or increase water intake: Evidence from a systematic review and meta-analysis. Obesity Reviews, 18, 1350–1363. doi: 10.1111/obr.12580 [DOI] [PubMed] [Google Scholar]
- Zoellner J, Chen Y, Davy B, You W, Hedrick V, Corsi T, & Estabrooks P (2014). Talking health, a pragmatic randomized-controlled health literacy trial targeting sugar-sweetened beverage consumption among adults: Rationale, design & methods. Contemporary Clinical Trials, 37, 43–57. doi: 10.1016/j.cct.2013.11.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zoellner J, Hedrick VE, You W, Chen Y, Davy BM, Porter KJ, Estabrooks PA (2016). Effects of a behavioral and health literacy intervention to reduce sugar-sweetened beverages: A randomized-controlled trial. International Journal of Behavioral Nutrition and Physical Activity, 13, 38. doi: 10.1186/s12966-016-0362-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zoellner J, You W, Connell C, Smith-Ray RL, Allen K, Tucker KL, Estabrooks P (2011). Health literacy is associated with healthy eating index scores and sugar-sweetened beverage intake: Findings from the rural Lower Mississippi Delta. Journal of the American Dietetic Association, 111, 1012–1020. doi: 10.1016/j.jada.2011.04.010 [DOI] [PMC free article] [PubMed] [Google Scholar]