Participatory Design of Mass Health Communication in Three Languages for Seniors and People With Disabilities on Medicaid

Abstract

Objectives. We used participatory design methods to develop and test guidebooks about health care choices intended for 600 000 English-, Spanish-, and Chinese-speaking seniors and people with disabilities receiving Medicaid in California.

Methods. Design and testing processes were conducted with consumers and professionals; they included 24 advisory group interviews, 36 usability tests, 18 focus groups (105 participants), 51 key informant interviews, guidebook readability and suitability testing, linguistic adaptation, and iterative revisions of 4 prototypes.

Results. Participatory design processes identified preferences of intended audiences for guidebook content, linguistic adaptation, and format; guidebook readability was scored at the sixth- to eighth-grade level and suitability at 95%. These findings informed the design of a separate efficacy study that showed high guidebook usage and satisfaction, and better gains in knowledge, confidence, and intended behaviors among intervention participants than among control participants.

Conclusions. Participatory design can be used effectively in mass communication to inform vulnerable audiences of health care choices. The techniques described can be adapted for a broad range of health communication interventions.

Reaching vulnerable populations and communicating effectively with them is a critical public health challenge. The estimated 55.6 million Medicaid beneficiaries¹ are one of the most vulnerable US populations and face serious communication barriers related to limited literacy, language, culture, and disability.^2,3 Of this population, the most at-risk subgroups are the more than 13.5 million seniors and people with disabilities.¹ Their communication barriers are even greater: many have limited health literacy skills (difficulty accessing, understanding, and acting on health information), limited English proficiency,^2,4 or physical or cognitive conditions that impede access to information.

With the rapid proliferation of “consumer choice models” in many states, millions of Medicaid beneficiaries are now required to make complex decisions about health plan options and effectively using health care.³ These decisions have important implications for the targeted beneficiaries regarding access to needed preventive, curative, and supportive services.^3,5–9 However, studies show that these populations have difficulty making such decisions—in part because of the poor quality of information they receive.^3–6,8,9 For example, in studies in which seniors on Medicaid used printed materials about health care choices, only 32% of those in Florida³ and 25% of those in California¹⁰ could understand the information. This is not surprising, as more than 250 studies indicate that printed materials related to health are written at reading levels greatly exceeding the average literacy skills of adults in the United States.¹¹

Theoretical guidance from socio-ecological,¹² social cognitive,¹³ and transtheoretical models,¹⁴ as well as 40 years of empirical research, suggest that health communication is more effective when it is relevant to people's personal and social contexts.^15–29 For this reason, interpersonal and tailored computer-mediated communication, which can be personalized, typically show better results than materials-based mass communication.^{18,22,24,25,30–32} However, Medicaid programs primarily communicate with seniors and people with disabilities through printed materials, given the need for low-cost mass distribution and this population's limited access to both in-person advice and the Internet. Is there a way to reach these highly vulnerable groups effectively and affordably?

Research from the past 20 years shows that using strategies to adapt printed mass communication resources more closely to the needs of population subgroups can improve outcomes.^15–29 These design principles include matching readability more closely to users' literacy levels and using “clear communication” formatting criteria,^33–35 incorporating culturally relevant concepts and graphics,^{4,17,19,23,36–40} and adapting rather than literally translating material into other languages.^{4,19,23,39,41}

Such design principles are invaluable, but they can only approximate how real audiences use and react to health communication. For this reason, increasing emphasis is being placed on “participatory” or “user-centered” design—a structured approach that employs varied formative research methods to involve intended consumer and professional audiences as codevelopers of communication.^{4,19,25,42–45}

Guidelines from the US Department of Health and Human Services and its centers, including the Centers for Disease Control and Prevention and the National Cancer Institute, recommend participatory design as a primary strategy to develop health communication.^{33–35,43,46} For the past 20 years, our center (Health Research for Action, www.uchealthaction.org) has leveraged such guidance to create large-scale mass communication materials by and for diverse populations.

Two decades of studies confirm that when users participate in designing and testing communication, outcomes are more successful, including those for vulnerable groups.^{4,25,28,42,44,47–51} However, limited research is available about the detailed, iterative methods and steps involved in developing user-centered, materials-based mass communication—especially about how to adapt it for specific situations.

In California, the Department of Health Care Services (DHCS) was concerned that fewer than 25% of Medicaid beneficiaries who were seniors or people with disabilities made active choices about health plans. By default, many beneficiaries could have plans ill suited for their health care needs. In 2006, the DHCS commissioned our center to use participatory processes to develop and test a guidebook about choosing and navigating health plans, intended for approximately 600 000 statewide Medicaid beneficiaries who were seniors and people with disabilities, in multiple language versions. Our objectives were to determine the (1) importance of participatory design in improving mass communication for diverse and vulnerable audiences, (2) methods to engage consumers and professionals in communication design and testing, (3) processes and outcomes of a project to create a user-designed guidebook for Medicaid beneficiaries, and (4) implications for health communication researchers and practitioners.

METHODS

We used 9 formative research methods in multiple phases to iteratively design and test 5 prototypes of the guidebook in English, Spanish, and Chinese (Mandarin and Cantonese) over 18 months, as shown in Figure 1. We selected methods, as well as processes to integrate those methods, from recommendations in past studies and from our center's experience. The findings from each method shown in Figure 1 were used to develop each successive prototype of the guidebooks in each language and to refine the questions or testing done sequentially over time.

FIGURE 1.

Timeline of formative research and development for the What Are My Medi-Cal Choices? guidebook: California, 2006.

At each formative research step, researchers immediately provided findings to the writers, translators, and graphic designers to incorporate into revised prototypes. This process continued until we achieved “saturation”—negligible recommendations for changes from consumers and professionals. At that point, the final guidebooks (fifth version) were produced in 3 languages. The overall design was intended to provide a comprehensive approach to link consumer and professional input to guidebook development and testing, and to allow “triangulation” of results for stronger content and convergent validity. The total cost of this formative research (not including guidebook production costs) was approximately $385 000.

We reviewed the literature related to (1) health care access, decision making, and communication for recipients of Medicaid and Medicare and for other vulnerable groups; (2) materials-based mass health communication; and (3) participatory design of communication resources. Key references from the review are cited in the introduction.^1–51

Advisory Group Input and Key Informant Interviews

To guide the project, we established a 24-member Advisory Group of advocates for diverse groups of seniors and people with disabilities, managed care plan representatives, health care providers, policymakers, and Medicaid beneficiaries. We obtained feedback through initial individual interviews with each Advisory Group member and 3 group meetings about issues related to access to health care, decision making, existing information resources, and recommended communication approaches for these audiences. The Advisory Group members reviewed and commented on the guidebook prototypes. Major issues and recommendations were synthesized from interviews and meetings.

We recruited and interviewed 51 statewide providers, policymakers (including DHCS staff), and advocates representing a broad range of organizational, service, and cultural and ethnic groups relevant to the target populations. Topics were similar to those for the Advisory Group. Responses to structured questions were quantified and reported as frequencies; open-ended questions were analyzed thematically.

Consumer Recruitment, Interviews, and Focus Groups

After the California DHCS sent letters to 5000 target Medicaid beneficiaries in 1 county, we screened 326 respondents to draw convenience samples for interviews, focus groups, or usability testing, and selected participants to ensure diverse representation by age, disability, education, and language. Additional participants were recruited in 2 other counties for focus groups. Caregivers who made decisions for recruited Medicaid recipients were included, when necessary, as proxies. Consumer samples included seniors, people with disabilities, and proxies for the aforementioned groups, with approximately equal proportions by language spoken.

We conducted 24 in-depth, semistructured interviews with target Medicaid recipients and proxies. Topics covered were informed by the Advisory Group interviews, with added emphasis on disability, literacy, and language issues. The interviewer took detailed notes and analyzed them thematically using Atlas.ti software version 5.3 (Scientific Software Development GmbH, Berlin, Germany). Demographic data were analyzed with SPSS software version 11 (SPSS Inc, Chicago, Illinois).

We conducted 18 focus groups for the target Medicaid consumers and their proxies (n = 105) in 3 California counties. Both Advisory Group and consumer interviews informed selection of focus group topics. Participants also reviewed prototype guidebook content for 1 or 2 specific topics (Table 1). Discussions were audio-recorded, transcribed, and analyzed thematically with Atlas.ti software, and demographic data were summarized with SPSS.

TABLE 1.

Examples of Content Testing of What Are My Medi-Cal Choices? Guidebook With Medicaid Beneficiaries: California, 2006

Content Tested	No. of Times Tested for Usability	No. of Times Tested in a Focus Group	Languages Tested
The 2 types of Medicaid health plans	9	6	English, Chinese, Spanish
Who can choose between the 2 types of health plans	4	6	English, Chinese, Spanish
County-specific information for each type of health plan	16	9	English, Chinese, Spanish, ASL
Consumer costs and billing information for each type of health plan	6	2	English, Spanish
Costs and availability of prescription drugs in each type of health plan; drug safety tips	5	3	English, Spanish
Doctors and other providers in each type of health plan	3	5	English, Spanish
Language and interpretation in each type of health plan	5	3	English, Chinese, Spanish, ASL
Disability access in each type of health plan	3	2	English, Chinese

Note. ASL = American Sign Language. Testing with consumers was conducted over a 10-month period.

Usability, Readability, and Suitability Testing

We conducted 36 semistructured, in-person usability tests^4,5 on guidebook prototypes to assess how well participants could read, use, and understand specific parts of the guidebooks. Detailed participant responses and observations were recorded in interviewer notes. Table 1 illustrates testing of selected guidebook topics.

We used the 4 most common standardized computer readability tests to estimate the reading levels of the English guidebook's content in terms of grade level in the US educational system.⁵² Because the tests' measurement techniques vary, as do their reliability at various grade levels, we used all of them to compare results. We used each test to assess 5 pages (every eighth page, beginning on page 4) of 2 prototypes and, as shown in Table 2, of the final guidebook. The tests were the following:

TABLE 2.

Readability Assessments of the What Are My Medi-Cal Choices? Guidebook Final Version: California, 2006

Guidebook Section Titles	SMOG Testa	Fry Readability Testa	Flesch Reading Ease Testb (Grade Level)	Flesch-Kincaid Readability Testa
You Can Choose the Kind of Medi-Cal You Want	8.5	6.0	81 (6th grade)	5.7
Emergencies and Urgent Care	8.7	7.0	72 (7th grade)	6.8
Home Health Care and Nursing Homes	8.5	6.0	81 (6th grade)	5.6
Language, Interpretation, and Sign Language Services	9.9	8.0	65 (8th–9th grade)	7.3
How to Enroll in, Change, or Leave a Medi-Cal Health Plan	7.8	5.0	86 (6th grade)	4.6
Average scores	8.6	6.4	77 (7th grade)	6.0

Note. SMOG = Simplified Measure of Gobbledygook.

^aScore indicates average grade reading level.

^bAverage raw score (with grade reading level in parentheses).

• SMOG (the Simplified Measure of Gobbledygook test) is considered a validated tool to assess reading grade level.^40,53

• The Fry Readability Test is widely accepted by reading professionals.³⁶

• The Flesch Reading Ease Test is used because of its simplicity, efficiency, strength of validity, and strong correlation with other readability formulas.⁵⁴

• The Flesch-Kincaid Readability Test is incorporated into Microsoft Word's (Microsoft, Redmond, WA) readability software and is easy accessible. However, the formula is truncated at a 12th-grade level in Microsoft Word and frequently presents falsely low evaluations.⁵⁵

Beginning with the first prototype, we used all the evidence-based design principles specified in the Suitability Assessment of Materials (SAM) validated tool^36,38,40 to develop the guidebooks. Table 3 shows the scores for the final guidebooks. Although there are no national data on US adult reading levels, and reading and literacy levels are not directly correlated, estimates of an average adult's reading ability have ranged from seventh to ninth grade.^56–58 For this guidebook, we aimed at a reading level of sixth to eighth grade (a SAM score of 1), considering the lower literacy of older adults and some people with disabilities. However, because readability testing assesses only some aspects of understandability, we put strong emphasis on adhering to all SAM design principles and on iteratively conducting usability tests. L. N. and B. R, who have expertise in health communication, conducted independent assessments, and we assessed interrater agreement.

TABLE 3.

Suitability Assessment of the What Are My Medi-Cal Choices? Guidebook Final Version: California, 2006

Factor to be Rated	SAM Scorea	Comments
Content
Purpose is evident	2	Guidebook purpose is explicitly stated on the cover and inside; each page explicitly states content purpose.
Content about behaviors	2	The guidebook gives concrete ways to accomplish desired behavior changes.
Scope is limited	2	Guidebook is limited to very specific information; each 2-page chapter treats 1 aspect of that focus.
Summary or review included	2	The main message is summarized on each page either in a vignette or in a question-and-answer format.
Literacy demand
Reading grade level	1	Readability assessed at 6th- to 8th-grade level.
Writing style, active voice	2	The guidebook is written in conversational style, active voice, and simple sentences.
Vocabulary uses common words	2	Common words are used and technical and jargon words are defined with simple explanations.
Context is given first	2	Each chapter starts with a 1–3 sentence introduction to provide the context of that chapter.
Learning aids via “road signs”	2	Each topic has its own header, which serves as a guide to what the following information will include.
Graphics
Cover graphic shows purpose	2	The cover has bright colors, large font, and photographs of adults representing target audiences.
Type of graphics	2	Photos are culturally relevant to the target populations.
Relevance of illustrations	2	Photos are crisp and free of distracters, and show people making health care decisions.
Explanations of lists, tables, etc.	2	All bulleted lists have a relevant title and explanation.
Captions for graphics	2	Each photograph has a relevant title and vignette.
Layout and typography
Layout factors	2	Layout is consistent among chapters, including colors and shading, plentiful white space (50% on many pages), line length of less than 30 characters and spaces, high contrast between type and paper, and interior pages are not glossy.
Typography	2	Text type is in uppercase and lowercase serif; type is minimum 15 point; bolding used for key points; “all caps” is not used.
Subheads (“chunking”) used	2	All lists have subheads, and most comprise no more than 5 items.
Learning stimulation, motivation
Interaction used	1	Question-and-answer format is used in each chapter.
Behaviors are modeled and specific	2	Photographs and accompanying vignettes model specific desired behaviors.
Motivation and self-efficacy	2	Complex topics are divided into 2-page chapters that include topical vignettes or photos and questions and answers to aid comprehension and self-efficacy.
Cultural appropriateness
Match in logic, language, experience	2	Examples and concepts are culturally similar to those of the target populations; guidebooks are in 3 languages.
Cultural images and examples	2	Photographs include positive views of racially/ethnically diverse groups and people with a variety of disabilities.

Note. SAM = Suitability Assessment of Materials. The total SAM score was 42 out of a possible 44 (95%).

^a2 = superior rating; 1 = adequate rating; 0 = not suitable rating.

Linguistic Adaptation

After we developed the first English guidebook prototype, we began to adapt it for Spanish and Chinese speakers. We first developed a dictionary of preferred terms relevant to Medicaid health care that beneficiaries would need to understand. When necessary, we developed new terms in other languages. To accommodate linguistic variations, we tested prototypes with both Mexican and non-Mexican Spanish speakers and with Mandarin and Cantonese speakers. After consumer testing, we back-translated the guidebooks into English to confirm their conceptual equivalence. We also reviewed the English version with users of American Sign Language.

Efficacy Study

We tested the initial quantitative outcomes of the user-designed resources. After the guidebooks were printed, 61 000 were mailed to target Medicaid beneficiaries in 3 pilot counties. We conducted a randomized controlled trial that included pre-post telephone interviews with 319 intervention and 373 control English-, Spanish- and Chinese-speaking participants in those counties to evaluate recipients' use of the guidebook, how satisfied they were with it, and its effects on their knowledge, confidence, and intended behaviors related to health care choices. The study variables were drawn from communication issues identified by consumers and professionals during the participatory design process. The study is reported elsewhere.¹⁰

RESULTS

In participatory design processes (as described in the Methods section), formative research methods, analyses, and findings are linked sequentially and are difficult to separate, as is done in traditional outcomes research. In this section, out of thousands of findings, we highlight some key content and format findings that helped to iteratively refine the formative research methods and guidebook development. Detailed findings about Medicaid issues are reported elsewhere.^5,10 Overall, we found that given the complexity of health care choices for these Medicaid beneficiaries, it was necessary to iteratively develop and test 4 versions of the guidebooks before producing the final 40-page resource, titled What Are My Medi-Cal Choices?, as shown in Figure 1.

Professional Feedback

We aggregated professional input from the Advisory Group interviews and meetings and key informant interviews into major themes and recommendations. The majority of professionals felt that (1) consumers lacked adequate information to make health care choices; (2) many consumers did not know in which type of health plan they were enrolled; (3) non–English-speaking Medicaid recipients faced significant barriers to health care; (4) consumers had misconceptions and negative opinions about Medicaid; (5) information about Medicaid was available, but was not necessarily easy to use; and (6) providers did not explain Medicaid choices well to these groups.

Professionals also recommended that consumers examine these issues, and that specific sections of the guidebook be developed to address them. Professionals also suggested that the guidebook layout include (1) question-and-answer sections, (2) stories that relate to consumers' experiences with Medicaid, (3) side-by-side comparisons of health plan choices, (4) resources to refer consumers to sources of additional information, and (5) step-by-step instructions that help consumers make health care decisions.

Consumer Feedback

We aggregated consumer input from interviews, focus groups, and usability testing into major themes and recommendations. Consumers cited the following problems they experienced with Medicaid health care and information about it:

• Lack of specific information about Medicaid health care choices in their county;

• Medicaid health care information that was not understandable;

• Difficulty accessing health care providers, particularly specialists;

• Lack of information about interpreter and transportation services;

• Confusion about paying for prescriptions and other services;

• Feelings of mistrust or stigma about using managed health care; and

• Difficulties for people with disabilities to navigate health care systems.

In reviewing prototypes in focus groups and in usability testing, consumers recommended that the guidebook include the following:

• Clear explanations of all content, including terms used;

• Clear comparisons of health plan choices in county-specific charts;

• Phone numbers and information about local hospitals;

• Information about where to go for more information;

• Culturally adapted information for deaf consumers, including American Sign Language symbols;

• Realistic explanations of health care situations to which seniors and people with disabilities can relate; and

• Photos and stories of culturally diverse seniors and people with disabilities in health care situations.

Linguistic Adaptation

During consumer testing, we found that non-English speakers with limited English proficiency did not understand important health care terms in English. Because Medicaid beneficiaries would need to use such terms (e.g., “expedited appeal” and “share-of-cost”) to communicate with providers, we provided terms in their language and in bolded English. We also found it difficult to develop one guidebook that could be understood by both Mandarin and Cantonese speakers. Chinese consumers advised that the guidebook be written with traditional Chinese characters, an approach that tested well with both groups. Deaf consumers, critiquing an initial photo of a deaf patient communicating with a physician, prompted us to take a new photo showing corrected positions of the deaf patient, interpreter, and physician.

Iterative Assessment and Revision of Prototypes

During 11 months of consumer testing and professional feedback, each successive prototype was rated higher than its predecessor. For example, consumers considered the initial standard definitions and explanations about Medicaid plans too confusing and helped design sections about what each term meant and how each plan worked. They also wanted realism and precision in the stories and vignettes. For example, they thought the following photo caption did not accurately reflect the experiences they had obtaining medications:

The pharmacist told John that his drug was not covered anymore. John asked his doctor to fill out an authorization form. After that, his prescription was covered again.

Beneficiaries recommended that we change the sentence of the caption to reflect the time required for authorization.

After his doctor sends the form to the Plan, authorization should take no more than 2 business days.

Consumers reported better understanding of previously confusing topics and demonstrated an ability to find answers to questions in the guidebook and in county-specific resource inserts. Similarly, professionals considered successive versions more accurate and relevant. We have provided several “before” and “after” samples of guidebook sections and county inserts, with brief explanations about revisions, in the online supplement to this article (available at http://www.ajph.org). Here, we provide 1 short example of text changes consumers helped make on the introductory page of the guidebook. The “before” text read as follows:

Medi-Cal is California's health care program for many people with low incomes. In other states this program is called Medicaid. Some seniors and most people with disabilities can choose what kind of Medi-Cal they want. Most counties in California have both Regular Medi-Cal and Medi-Cal Managed Care Plans.

• Regular Medi-Cal is also called “Straight Medi-Cal” or “Fee-for-Service Medi-Cal.”

• Medi-Cal Managed Care is also called “Medi-Cal HMO.” HMO means Health Maintenance Organization. Most counties have at least 2 Medi-Cal Managed Care Plans.

Consumers felt there were too many terms for an introduction and not enough emphasis on what a choice would mean for them. They preferred the following revision, which was incorporated into the “after” text:

Medi-Cal is a California health care program for many people with low incomes. Many counties in California have 2 kinds of Medi-Cal. The 2 kinds are

• Regular Medi-Cal.

• Medi-Cal Health Plans.

You can choose which kind of Medi-Cal you want. With both kinds, you will get the same basic benefits and care. But the way that you get care may be different.

Readability, Usability, Suitability, and Efficacy

The target guidebook reading level was between sixth and eighth grade. Table 2 shows scores of the 4 readability tests. On average, tests indicated that the readability level of the text was grades 6.0 to 8.6. For 3 tests, average scores were between sixth and seventh grade; the SMOG score was eighth to ninth grade. The 4 average scores were used to determine the overall SAM “reading grade level” rating (Table 3). A sixth- to eighth-grade reading level is scored as 1 on the SAM.

As shown in Table 3, the SAM test scores showed that 95% of the categories assessed met the recommended criteria for easy-to-use and culturally relevant print information. Interrater agreement was κ = 1.0 (Cohen's coefficient κ).

At baseline, before reading the guidebook, fewer than 25% of participants understood their Medicaid health care choices. At follow-up, the intervention participants reported high guidebook usage (77%) and satisfaction (98%). The intervention group significantly improved their knowledge, confidence, and intended behaviors related to health care choices compared with the control group. Detailed results are reported elsewhere.¹⁰

DISCUSSION

The development of effective mass communication materials for vulnerable audiences such as seniors and people with disabilities on Medicaid has proved a challenging goal. These groups not only have the poorest health of any major population in the United States but also face some of the greatest communication barriers related to literacy, language, culture, and disability.^2,4–6,8 Empirical evidence indicates that communication for these groups is poorly adapted to their personal and social contexts.^2,5 Increasingly, however, these groups are being asked to make complex health care choices that can have serious implications for their health outcomes.^3,5 It is not surprising, but is of great concern, that only small percentages of Medicaid beneficiaries who are seniors or people with disabilities appear to effectively use the information they receive to make these choices.^3–5 Unfortunately, these groups also face barriers to accessing communication modes that are theoretically and empirically most effective—such as tailored computer-mediated communication and in-person advice.^21,22,25

Evidence from the past 2 decades suggests, encouragingly, that materials-based communication can be developed successfully to meet the needs of even the most vulnerable populations.^16,21–26 Proven design principles such as readability and cultural appropriateness have contributed greatly to improving communication.^{15,17,27,36,38,40} However, no set of design criteria can adequately codify the multiplicity of factors that affect people's comprehension and motivation. For example, readability tests provide a rough estimate of only some aspects of comprehension.

Participatory design leverages important design principles,^28,36,40 but goes beyond them to engage users in more complex, realistic, and nuanced scenarios as cocreators of communication.^19,25,28,29 This strategy has drawn from the theories, methods, and evidence of multiple disciplines, including marketing, engineering, human factors, sociology, and health, over the past 40 years.⁴² Participatory design is choreographic: researchers, writers, translators, and artists share information with each other in a series of carefully orchestrated steps.

Participatory design is also difficult. There is no one “formula.” Each design must be customized to fit the project goals and resources. The 9 formative research methods we used in this study go far beyond the scope of most traditional communication efforts for the intended audiences, and they required 18 months to complete. Our research and communication teams found that the exchange of findings and revision of prototypes in a tight time frame was a difficult task. However, the stakes are high: effective communication with the target audiences. Medicaid beneficiaries who are seniors or people with disabilities not only have high health risks but also account for $75 billion of the total $319 billion Medicaid budget, and their health care costs are rising rapidly.¹ A concerted effort to use participatory design to benefit this population should be a major public health goal.

Building on the literature and our 20 years of applying participatory design to improving mass communication for vulnerable populations, we have learned a number of lessons that are echoed by others in this relatively emergent area.

First, all the major subgroups in the intended audiences should be identified. In this case, that included engaging varied groups of people with disabilities, including those with physical, chronic illness, cognitive, sensory, and mental health disabilities and those who use American Sign Language. Further, the view of “users” should be expanded to include such stakeholders as providers, advocates, and policymakers. This project benefited greatly from close collaboration with colleagues from the California DHCS, who ensured that the guidebooks would meet programmatic and legal requirements, as well as with statewide providers who recommended ways to integrate the guidebooks into their services. The public health ecological systems view, advanced strongly in recent years,^12,59 provides good guidance: an intervention will be successful only when it fits into multiple contexts.

Second, critiques from participants should be carefully listened to. When intended users of the guidebooks recommended having specific information about local health care resources, that feature became a priority and the requested information was customized into special inserts for each county.

Third, attention should be paid to the granularity of linguistic variations. Although literal translation has been the most common way to develop communication for non-English speakers, linguistic variations within language groups provoke much confusion unless resolved with people who represent those groups. In this study, Mandarin and Cantonese speakers provided critical impasse-breaking advice to create a single Chinese language resource.

Fourth, the iterative process should be honored to its natural end. Codesigning effective communication is an interactive process in which diverse people define finer and finer changes until an acceptable communal product is achieved. Even though our first prototype was carefully designed to meet comprehensive clear communication principles and greatly exceeded the quality of typical health information, we could not have anticipated the many valuable changes that users would ultimately contribute.

Finally, a long view should be taken regarding costs. Although participatory design is likely to be initially more expensive than traditional approaches, its proven effectiveness should yield larger returns on investment, especially for vulnerable groups.

This study has limitations. Participatory design methods are still being developed and tested. Those we used were primarily qualitative, and findings cannot be generalized to everyone in the target audiences. However, we involved hundreds of participants in an iterative sequence and used 9 complementary formative methods that showed similar findings, suggesting strong content and convergent validity. Further, the efficacy study¹⁰ showed not only that the vast majority of Medicaid beneficiaries used and appreciated this resource, but that it also sufficiently improved their knowledge, confidence, and intended behaviors related to health care choices.

In 2008, the guidebooks were awarded the Institute for Health Care Advancement's first-place national health literacy award for published materials. We believe that this study adds to the literature on the power of participatory design to improve health communication and to promote health for populations with the greatest needs.