A PICTURE’S MEANING: THE DESIGN AND EVALUATION OF PICTOGRAPHS ILLUSTRATING PATIENT DISCHARGE INSTRUCTIONS

INTRODUCTION

Several studies have reported that the use of illustrations may improve patient comprehension of health communications. ^1–6 Additionally, a recent study showed a significant participant preference for patient information containing pictographs due to perceived improvements in readability and clarity.⁷ Another study demonstrated that by incorporating patient learning preferences, such as adding pictographs, understanding of patient material was improved.⁸ Pictographs have been shown to specifically improve recall and problem solving in individuals with little expertise in the given topic.⁹ However, developing effective illustrations remains a challenge. Illustrations must be appropriate for the healthcare context and be accessible to a range of patient education levels and cultural/ethnic backgrounds. Further, pictures themselves are not universally comprehensible.¹⁰

To improve cardiovascular discharge instructions, we created a large and publically accessible library of medical pictograph library¹¹ through participatory design. These pictographs were intended to improve patient care, as well as serve as a pubic database of images that can be used in a variety of communication efforts. The purpose of this study was twofold: first, we conducted recognition testing to evaluate the aforementioned pictographs created for cardiovascular discharge instructions. Second, we examined what illustration approaches work for specific types of medical concepts and subpopulations. Specifically, we tested a large set of pictographs (n=488) on a diverse population of subjects (n=150) for overall recognition rates by demographic groups, representation strategies, and semantic types. Further, we analyzed the effectiveness of representation strategies in relationship to recognition.

BACKGROUND & SIGNIFICANCE

The Centers for Medicare & Medicaid Services and The Joint Commission agencies have both issued mandates to provide patients with discharge instructions during the inpatient hospital discharge process.¹² Typical components of discharge instructions include indications for activity, diet, medications, plans for provider follow-up, wound care if applicable, signs and symptoms to be aware of, and provider contact information to call should a problem or question arise following discharge.¹³ In 2012, the Agency for Healthcare Research & Quality (AHRQ) reported that 20% of patients experience adverse events within the three weeks following hospital discharge and 20% of Medicare patients are readmitted within 30 days of when they were discharged.¹⁴ Approximately 17% of these adverse events were deemed to be preventable or could have been ameliorated¹⁴.

While there are many aspects to post-discharge care and care coordination that must be addressed to prevent adverse events after discharge, the design of patient-centered discharge instructions may have a significant impact on post discharge care comprehension and compliance. Unfortunately, discharge instructions can be difficult for patients to understand. Lack of discharge instruction comprehension has been demonstrated to affect patient satisfaction, as well as compliance with treatment plans.^15–17 In a recent study, 80% of patients discharged from an emergency department showed deficits in understanding home care instructions and 79% of patients showed deficits in understanding return/follow-up instructions¹⁸. Several studies have demonstrated that patients frequently lack full comprehension and recall of discharge instructions.^19–21

Health literacy and readability are the two major factors in understanding discharge instructions that consequently impact adherence. ^22–25 There is often a mismatch between a patient’s health literacy level and the readability of health information, such as discharge instructions, that are provided to patients who are expected to comprehend and then adhere to the instructions provided to them.²³ An intervention that has been found to improve readability and comprehension of health information is to include illustrations and pictures. ^{22, 26–29}

The recovery period that follows an inpatient hospital discharge is risky as complications can lead to an early readmission to the hospital.²² Therefore, it is essential for patients to be able to read and follow their discharge instructions. Our research team elected to focus our efforts on cardiology as the domain of all hospitalized inpatients is very broad and it was necessary to narrow our efforts in order to tackle this complex problem. We also have clinical experts in cardiology on our team which helped guide our targeted intervention. Additionally, there is a lot of consistent information in discharge instructions that can be generalized to other hospitalized populations. Another reason we chose cardiology is that in 2012, the Centers for Medicare & Medicaid Services (CMS) began a readmission reduction program that focuses on a discharge diagnosis of acute myocardial infarction, heart failure, or pneumonia.³⁰ Given the focus of CMS efforts to address early readmissions, we decided to target this research intervention to evaluate the effectiveness of pictographs used to enhance discharge instructions with cardiology discharge instructions as an effort to address early readmissions due to preventable complications that can be addressed with discharge instructions.

To improve patient comprehension of discharge instructions, we explored the use of pictographs as a tool to enhance discharge instructions. One way to create patient-friendly pictures is using a participatory design system. ^31–32 This process may potentially yield illustrations that are comprehensible to a wide audience.³³ We experimented with our own participatory design process, during which we peer-reviewed a convenience sample (150–200) of the resulting pictographs. Because participatory design is a process in which stakeholders, in this case patients, are actively involved in a design process, we asked a group of patients to draw pictures for the concepts in the cardiovascular discharge instructions. The drawings were collected and the shared design elements were analyzed with the goal of synthesizing a higher quality picture. This process revealed several challenges in this system. For example, patient participants had difficulty drawing, there was no convergence of designs and interpretation, and it was too time consuming for our team to review the hundreds of pictures created with this process.

Given our experience with participatory design, we elected to use a graphic artist with experience in medical illustration to create pictographs for discharge instructions in the realm of cardiovascular disease, and test these pictographs with a large audience. Our team, consisting of clinicians and informaticians, provided initial feedback. After initial review and approval, the resulting pictographs underwent patient testing, with patient recognition and comprehension as ultimate goals.

Our pictograph development was guided by a graphic lexicon which was developed by analyzing a sample of more than eight hundred pictographs according to three axes:³⁴ lexical category (the component of speech), semantic category (based on the meaning of the word), and representation strategy (how the concept is represented in the pictograph). Three basic representation strategies were identified: visual similarity, semantic association, and arbitrary convention (Table 1). These representation strategies were in agreement with the strategies identified in previous taxonomies, with the addition of several subcategories.

Table 1.

Representation Strategies

Representation Strategy	Definition	Example
Direct Representation	Pictograph has visual similarity to its referent
Arbitrary Representation	Pictograph is a symbol of an object or concept established by convention; has no visual similarity to its referent.
Indirect Representation	Pictograph suggests concept related to referent rather than visually representing referent.

The pictographs created through the above process have been deposited into the Open Access Consumer Health Vocabulary Web site,¹¹ and are utilized by a computer application we developed (Glyph) to enhance discharge instructions. Glyph automatically illustrates text with analogous pictographs using natural language processing and computer graphics techniques.³⁵ This system consists of various modules, including a graphic lexicon, graphic syntax and a computer application, which follow a pipeline design to assemble specific concepts into composite pictograms.

MATERIALS AND METHODS

Over 2600 unique instructions were obtained from the University of Utah Cardiovascular Service and from the Internet, from which we extracted over 800 distinct instruction items that were deemed to be illustratable. Our medical graphic designer developed pictographs to accompany each of these instructions. Among those, approximately 150–200 underwent study team review, prior to testing a larger set of pictographs with a diverse group of study participants.

All pictographs were stored in the Glyph database. Each was assigned a semantic type based on the UMLS semantic group according to the concept each represented. Of the 800+ images developed, we randomly selected 500 pictographs that contained representations of clinical concepts commonly used in discharge instructions for cardiovascular patients. Twelve images contained duplicate meaning that we eliminated; therefore, we tested 488 final images in this study. We designed questionnaires for participants to complete using free-text fill-in-the-blanks, with each question containing one pictograph, one discharge instruction, and one blank space. All instructions were extracted from real-world discharge documents.

A total of 150 questionnaires were computer generated with each containing 50 questions, enabling each pictograph to be tested 15 times. Some discharge instructions had more than one illustratable portion by design, however we did not allow for repetition of any instructions within any questionnaire in order to avoid providing hints. From the group, twelve individual pictographs were represented in two different questions each, as they occurred in two different instruction contexts. In analysis, they were treated as different items.

The University of Utah Institutional Review Board (IRB) approved this study. Research assistants recruited 100 study participants from the cafeteria area of the University of Utah Hospital and 50 from the Environmental Services Department via convenience sampling. We used convenience sampling because the population we recruited is a relatively good representation of people utilizing the healthcare system. The pictograph development system was designed for a broad target population, from novice to experienced patients, which includes patients with cardiovascular disease current and future patients, and their family/caregivers. The limitation of our selection is a lack of representation of critically ill patients. Inclusion criteria for participants included individuals 21 years of age or older, and able to speak, read, and write in English. Exclusion criteria included anyone unable to read, having any visual, cognitive, language, or other impairments that would prevent full participation in the study, and anyone who currently or previously worked with discharge instructions in any capacity. Informed consent was obtained from each participant, followed by provision with a randomly generated questionnaire. Participants were given up to 20 minutes to read the questions and fill in the blanks based on the pictographs; however, the majority required approximately 10 minutes to complete the questionnaire. We additionally collected demographic information including age, gender, race, ethnicity, education level, and first language. Each participant received a $10 gift card after completing the questionnaire.

From the 150 questionnaires, we collected responses for a total of 7500 questions. The questionnaires with participant’s hand-written responses were scanned and answers were transcribed into a database. Demographic data were coded for statistical analysis. The questionnaire results were evaluated against the complete phrases extracted from the original discharge instructions. The following 4-point Likert scale was used to score responses: 1= incorrect, 2= mostly incorrect, 3= mostly correct, and 4= correct. Three investigators rated 10% of the answers and an inter-rater agreement was calculated. Disagreements in rating were resolved through consensus. Inter-rater agreement was calculated using paired Cohen weighted Kappa. The estimated weighted Kappa was 0.89, 0.88, and 0.87 respectively. All scores were higher than the conventionally acceptable Kappa value of 0.85; therefore individual reviewers independently rated remaining responses thereafter. Table 2 below displays sample questions with sums of score ratings. Participants used question marks to fill in the blank when answers were unknown.

Table 2:

Sample Questions with High and Low Scores

Image	Sample Question	Sum of Rating	Correct Answer	Answers
	_______________ will make you feel weak and dizzy.	25	Low blood sugar	donating blood lowblood pressure less than 60 blood test low BP lowblood pressure narcotics giving blood, blood transfusion low blood sugar ? driving low blood pelse(pulse) test blood blood sueco(sugar)
	Tell your doctor if you have _______________	37	Kidney disease	lung disease sad lungs kidney ploms renal failure lung problems kidney problems ? ? kidney problems kidney disease lung problem kidney problems kidney problem kidney pain
	Call your doctor immediately if you have an abnormally _______________	51	Fast Heartbeat	Afib rapid heart beat fast heart beat too fast heart beat (rate) fast heart rate in your heart fast heart rate fast heart rate fast heart rate fast pulse heart is bad bite fast hart(heart) bead(beat) fast heart beat fast heart beat faster
	Call your doctor right away if you have _______________	57	Blood in your urine	blood in urine blood in urine blood in your pee blood in urine blood in urine blood in your urine bloody urine blood in urine blood in urine bloody urine blood in urine blood in uren(urine) Orin(urine) bleeding pee red orine(urine)

Analyses

Quantitative analyses were first conducted to provide overall descriptive statistics and the recognition rates in different demographic groups. Average scores and score distributions were calculated based on categorized semantic groups. Each rating ranged from 1 to 4, therefore the total score of fifteen ratings for each question ranged from 15 to 60. Total scores below 40 (i.e. <3 on the 1 to 4 scale or below “mostly correct”) were designated as “low-scoring”, indicating low recognition accuracy. Scores equal to or above 40 (i.e. >=3 on the 1 to 4 scale, or “mostly correct” or more) were designated as “high-scoring,” indicating high recognition accuracy.

Since pictograph recognition is an interaction between the pictures and the human subjects, a multivariable linear regression model was fitted to the recognition rates to assess the impact of the characteristics of the human subjects. A backward variable selection was used, beginning with all variables in the model and removing them in the order of least significance (largest p value), ending with all remaining variables with p < 0.05.

Qualitative analyses were then conducted in order to examine what pictograph characteristics correlated with high or low scoring groups, and to analyze the effectiveness of different strategies within the context of the semantic category conveyed. Pictographs were examined to identify which representation strategies were effective or ineffective, and why. After scoring the recognition test for the pictographs, each was classified and analyzed by (1) representation strategy, (2) strategies and subpopulations, (3) semantic type, and 4) cross-strategy design issues. Corresponding with the quantitative analysis, we considered pictographs that scored above 40 points to be “effective,” and those that scored below 40 points to be “ineffective,” for the qualitative analysis.

Pictographs were classified according to the representation strategies outlined by Nakamura & Zeng-Treitler³⁴ including: direct, indirect, and arbitrary. Figure 1 below displays examples of representation strategies. Pictographs using the direct representation method (a) “have a significant similarity between a pictograph and its referent;” pictographs using the indirect method (b) “explore semantic relations between a pictograph and its referent,” and pictographs using the arbitrary method (c) “are established by social convention.” ³⁴

Figure 1.

Representation Strategy Examples

A fourth hybrid category was classified for pictographs that contained both indirect and arbitrary elements. Because indirect representation is a large category, we further analyzed this classification by semantic type.

QUANTITATIVE RESULTS

Demographics

Among the participants, 44.7% of participants were male and 55.3% were female. Participants under 50 years of age comprised 75.9% of the sample. Race was comprised of 61.4% white, 12.9% Asian, and the remaining were individuals of other races. Non-Hispanic participants were 76.9% of the sample, 23.1% were Hispanic. The education level of 78.7% of participants was greater than 12^th grade, and 69.1% spoke English as native language. See Table 3 for detailed demographic description.

Table 3.

Study Participant Demographics (n=150)

Category	n(%)
Gender, n(%)
Male	67 (44.7)
Female	83 (55.3)
Age, year, n(%) 149
21–29	46 (30.9)
30–39	36 (24.2)
40–49	31 (20.8)
50–59	22 (14.8)
60–69	13 (8.7)
70–79	1 (0.7)
Race, n(%) 140
White	86(61.4)
Asian	18(12.9)
Other	15(10.7)
Black or African American	14(10)
American Indian/Alaska Native	4(2.9)
Native Hawaiian or Other Pacific Islander	3(2.1)
Ethnicity, n(%) 130
Hispanic	30(23.1)
Non-Hispanic	100(76.9)
Education, n(%) 150
≤4th Grade	3(2)
5 to 8th Grade	4(2.7)
9 to 12th Grade	25(16.7)
>12th Grade	118(78.7)
First Language, n(%) 149
English	103(69.1)
Other	20(13.4)
Spanish	16(10.7)
Chinese	10(6.7)

Recognition Rates

Among the 488 pictographs, 314 (63%) scored high ratings as “effective” (40 or above), and 186 (37%) scored in the “ineffective” group (under 40), across all participants. In other words, the majority of pictographs were well-recognized. On the 4 point Likert scale (1-totally incorrect to 4-totally correct), the mean ratings of a pictograph by male and female participants were 3.05 and 3.02, respectively. Among age groups, 40–49 scored the lowest mean rating. The mean rating scores for Hispanic non-Hispanic ethnicities were 2.7 and 2.9, respectively. Regarding race, whites scored higher overall (3) than non-whites (2.6). College education was associated with higher mean scores (2.9) over no college education (2.6). The English-as-native-language group scored higher (3) than the non-English-as-native-language group (2.5). See Appendix A for a detailed table of results.

A multivariate model of pictograph ratings is shown in Table 4. The average mean rating of participants of white race, has attended a minimum of some college, and speaking English as a primary language was 3.13. Nonwhite race reduced the rating by 0.20, having no college education reduced the rating by 0.32, and primary language other than English reduced the rating by 0.45. Age, gender and Hispanic ethnicity were dropped from the model, as they did not provide an independent contribution for the rating after controlling for three variables shown in the final model. In other words, race, education and primary language significantly contributed to pictograph recognition, and the contribution of each factor to recognition was independent.

Table 4.

Multivariable Linear Regression Model of Pictograph Rating

Predictor	Adjust Mean difference (slope)	95 % CI	P value
Non-white race	−0.20	−0.37, −0.02	0.03
Non-college	−0.32	−0.49, −0.15	<0.001
Non-English first language	−0.45	−0.63, −0.26	<0.001
Intercept	3.13	3.04, 3.22	<0.001

QUALITATIVE RESULTS

For this component of the analysis, we examined results of “effective” versus “ineffective” pictographs as categorized by representation strategy, subpopulations, semantic category, and identified design issues. A majority of the pictographs (63%) were recognized across all subjects.

1. Analysis by Representation Strategy

Overall Scores

Direct representation strategy pictographs averaged the highest scores, followed by arbitrary, indirect, and indirect with arbitrary (see Table 5).

Table 5.

Representation Strategy Average Scores

Representation Strategy	N	Average Score
Direct	165	48
Indirect	161	41.7
Arbitrary	5	46.4
Indirect with Arbitrary	157	38.2

Direct Representation Strategy

The direct strategy pictographs (n=166) attained the highest averaging recognition rate overall (score = 47.9), with 81% determined to be “effective” (scoring over 40) (n=135). These pictographs were most frequently utilized to represent objects. An example of a high scoring direct representation strategy image is shown in Figure 2 below in the pictograph for “shower” (score=60).

Figure 2.

Direct Representation Pictograph “Shower”

Indirect Representation Strategy

The indirect representation strategy (n=160) was the third highest averaging strategy overall (score = 41.5), with 58% that were “effective” (n=93). An example of a high scoring image of an indirect representation image is shown below in Figure 3 in the pictograph for “nutritionist” (score = 54).

Figure 3.

Indirect Representation Pictograph “Nutritionist”

Ineffective indirect representation strategy pictographs tended to represent very specific concepts that a participant may require some familiarity in order to interpret the images. Examples follow in Figure 4 below in the pictographs for (A) “low blood sugar” (score=25), (B) “trouble swallowing” (score=22), and (C) “Holter monitor makes noise” (score=29).

Figure 4.

Ineffective Indirect Strategy Examples

Arbitrary

Although the arbitrary strategy was used in few pictographs in this study (n=5), it scored the second highest average recognition rate (score = 46.4). While these pictographs did provide valuable information about the usability of several arbitrary symbols, we looked further at the indirect with arbitrary representation strategy for a more in-depth analysis. See Figure 5 below for an example of an effective arbitrary pictograph. This pictograph illustrates “hospital” (score = 57).

Figure 5.

Arbitrary Representation Pictograph “Hospital”

Indirect with Arbitrary

The indirect plus arbitrary representation strategy pictographs (n=157) were the least effective of all categories (average score = 38.3) with only 48% of the pictographs effectively recognized. Still, there were 23 indirect with arbitrary elements that were highly effective across all participants. For example, an image with the instruction “sit down” attained a perfect score of 60 (A), as did a pictograph illustrating “headache,” (B) (Figure 6).

Figure 6.

Successful Indirect Strategy with Arbitrary Elements Examples

For this analysis, we also grouped pictographs into two categories: effective (scoring 40 or above), and ineffective (scoring below 40), and then extracted the common elements across each group to identify the successful and unsuccessful components. We found that the effectiveness of arbitrary symbols varied depending on the context or message. For example, an arrow was used in several pictographs to express a cause and effect relationship. It was used successfully as a way to express change and motion in many pictographs that utilized the indirect with arbitrary representation strategy. However, the arrow symbol was ineffective in other cases. Figure 7 below provides both examples, with an arrow indicating, “do not raise your arm above the shoulder” (A) that was ineffective (score=22), but was used effectively in the “ hands swelling” pictograph (B) (score=51). Another unreliable arbitrary symbol was the use of curved lines to represent several instances of “motion,” which was effective in a “shaking” pictograph (score=49), and ineffective in a “twitching muscles” pictograph (score=29) (see Figure 8).

Figure 7.

Arrow Symbol Effectiveness Examples

Figure 8.

Ineffective Image with Curved Lines Illustrating “Twitching Muscles.”

Similarly to the indirect representation pictographs, images that were ineffective in the indirect with arbitrary category tended to contain specific concepts, such as signs or symptoms that are not visible to the naked eye, for example: hoarseness (A), urinary tract infection (B), and healing breastbone (C) (see Figure 9).

Figure 9.

Unseen Concept Examples

Pictographs that attempted to represent speed or time were met with varying interpretations, likely due to cultural background and exposure to the concept required for successful interpretation of arbitrary representations. For example, some participants reversed the traditional understanding of the speed of two commonly represented animals, such as in image (A) with rabbit being slow, and (B) and (C) with turtle being fast; one interpreted the representation of the rabbit literally. Examples follow in Table 6.

Table 6.

“Speed” Concept Participant Responses

A. Breath In & Out Quickly	B. Slow Pulse	C. Slow Heart rate
Score: 29	Score: 43	Score: 48
Answers	Answers	Answers
breath	slow pulse	defib
breath in and out quickly	slow pulse	slow heart beat
look and breath before you jump	slow pulse	slow heart beat
breathe in and out,(watch for rabbits)	slow heart rate	too slow heart beat (rate)
?	low heart rate	slow heart rate
?	high pulse rate	heart ache
breath quickly	high pulse rate	slow heart rate
?	slow HR	slow heart rhythm
exhale quickly	rapid pulse	slow heart rate
looking, put something in the mouth?	slow blood pressure	slow pulse
cough	moving slow	heart case sick
look	slow heart rate	slow hart(heart) beat
breathalizer(Breathalyzer)	slow pulse rate	slow heart beat
breathe
running

2. Analysis by Semantic Type

The semantics concepts influenced both the effectiveness and cognition of the representation strategies. Table 7 and Figure 10 display the distribution of the average rating by semantic group. Out of 13 semantic types, the categories of Objects (n=124), Activities & Behaviors (n=99), and Disorders (n=161) were represented most frequently. Of these, the highest scoring semantic type was Objects (average score = 47.3). The Activities and Behaviors category, which averaged a 44.3 score, used direct representation strategy in only 24% of cases. The Disorders category scored an average of 38.8, and used direct representation strategy in only 15% of cases. Thus, a relationship appears to exist among the scores of these semantic types and the percentage of cases that utilized the direct representation strategy. When the direct representation strategy was used to represent a semantic type, the scores were frequently higher. However, there were some exceptions to this trend in cases of semantic types with small sample sizes, including the categories of Devices (n=14) and Concepts & Ideas (n=18). The Devices category used the direct representation strategy in 61% of cases, yet averaged a score of only 40.8.

Table 7.

Distribution of Average Rating by Semantic Group

Semantic_Types	Average of Sum_Rating	Count	Score<40	%	Score>=40	%
Activities & Behaviors	44.3	99	34	34.3%	65	65.7%
Anatomy	46.7	10	2	20.0%	8	80.0%
Chemicals & Drugs	45.4	5	2	40.0%	3	60.0%
Concepts & Ideas	48.1	18	4	22.2%	14	77.8%
Devices	40.8	14	7	50.0%	7	50.0%
Disorders	38.8	161	78	48.4%	83	51.6%
Living Beings	38.0	1	1	100.0%	0	0.0%
Objects	47.3	124	26	21.0%	98	79.0%
Occupations	49.5	6	0	0.0%	6	100.0%
Organizations	48.5	4	1	25.0%	3	75.0%
Phenomena	28.5	2	2	100.0%	0	0.0%
Physiology	39.1	14	6	42.9%	8	57.1%
Procedures	38.8	42	23	54.8%	19	45.2%

Figure 10.

Average Sum Rating by Semantic Type

3. Cross-Strategy Design Issues

We identified cross-strategy design factors that influenced recognition across all representation strategies. One such factor was the level of detail used to illustrate images. Specifically, too much or too little detail can adversely affect pictograph recognition rates. Examples of images with too much (A) and too little detail (B) are illustrated in Table 8 below. In image (A), the participant responses indicated that the chest was the focus of recognition while the incision, the intended focus of the image, was not noticed. In image (B), the inaccuracy and variety of responses suggest that there was insufficient detail to cue the participants what the object actually was.

Table 8.

Unsuitable Image Detail Examples

A) Incision is warm	B) Bamboo
Score 17	Score 39
Answers	Answers
chest and throat	Asparagus
?	Bamboo
?	?
chest	looks like seaweed! Is this supposed to be asparagus?
chest	Fiber
chest	Onions
chest is buring	Asparagus
chest	Asparagus
surgical site	Asparagus
chest	Bamboo
24 hours	(asparagus)
24 hour	Asparagus
chess(chest)	Asparagus
droctor(doctor)	aspirlgess(asparagus)
breast	Asparagus

Another factor that can influence recognition is the use of color. Color can enhance communication of certain concepts as well as provide contrast in a small space. The ability to illustrate many ideas, such as “redness”, “nausea”, and “bleeding,” would not be possible or not as successful without the use of color.

DISCUSSION

In this study, direct representation strategy pictographs averaged the highest scores, followed by arbitrary, indirect, and indirect with arbitrary (Table 5). In our analysis by representation strategy, of the three main strategies (direct, indirect, or arbitrary), the direct strategy method was by far the most successful and reliable across all populations. In addition to being generally recognizable, some pictographs using this method also scored higher among non-native English speakers than native English speakers, and higher among participants with no education beyond 12^th grade than those with college education. However, because only objects can be illustrated via the direct strategy, there is a limit to what information can be represented using this method. The other representation strategies did yield satisfactory success in many cases, however not as predictably.

Overall, pictographs that contained elements with arbitrary and indirect representation strategies combined scored lower than either purely indirect or arbitrary strategies. The successful pictographs that used the arbitrary and indirect with arbitrary strategy were typically representations of activities, disorders, and exterior body parts that are visible to the naked eye, which all tend to be generally familiar concepts. Both indirect strategy and indirect with arbitrary strategy pictographs that were ineffective tended to illustrate concepts that are unfamiliar or difficult for the layperson to visualize, such as internal bodily functions or organs, medical devices, and specific diseases. Also within the arbitrary representation category, some arbitrary elements within pictographs were highly effective, while some were less consistent in their success and were unreliable for general use. Identifying the most successful visual elements of the pictographs was important in order to improve future pictograph design and redesign.

In our analysis by semantic type, the semantic category that met the most success was the Objects category. Not surprisingly, most (95%) pictographs in this category used direct representation strategy (which, as described above, was the representation strategy that was successful in 82% of instances used). The Devices category required some degree of prior familiarity with the context that the items relate to in order to be recognized, despite the accuracy of its representation. Further, the Concepts & Ideas category did not use the direct representation strategy in any cases, yet still scored an average score of 48.1 We postulate that one reason for this may be that the ideas represented in this category are more common, thus are more easily recognized by the general population without prior knowledge in order to be identified.

Future Work

Future work will include redesign of various images using the results of this study. After testing was conducted, 84 “ineffective” pictographs were determined to be eligible candidates for redesign. We carefully examined responses given by participants to gain insight on how these pictographs were interpreted. In so doing, we determined if the concept of these pictographs was illustratable or representable, and identified trends that contributed to high and low scores Because we had access to different versions of concepts that were illustrated in different ways, we were able to isolate what was different between images, and decipher the successful elements of effective images. This data will drive the redesign of eligible images for future work. Testing of the pictographs in actual clinical situations (e.g. discharge instruction education) is currently underway to build upon the results of this study to further investigate if pictograph enhancement increases patient compliance, resulting in decreases in hospital re-admittance.

Limitations

Limitations of this study included that demographics of the convenience sample were relatively skewed, with the majority of participants being of white race and native English speakers. Testing with individuals who have been exposed to specific subject matter relating to the content of the discharge instructions may have made the results more applicable, such as patients who have received cardiology care and possess a degree of prior familiarity with the related context.

In addition, some of the patient instructions were poorly written, being either too brief or misleading, which may have compromised the scores of the associated images. An example of an instruction that may have been too brief to provide adequate context follows in Figure 11, for which the sentence should read, “don’t skip your medication.” The pictograph was intended to illustrate the part of the sentence “skip medications” (score = 21).

Figure 11.

Instruction with Inadequate Context

CONCLUSION

While post-discharge care and coordination involves many complexities, the design and use of patient-centered discharge instructions may substantially impact post discharge care understanding and compliance. In this study we evaluated the degree to which a series of 488 pictographs were recognizable in the context of clinical instructions. A majority (63%) was determined to be successful based on high recognition scores. Predictors of successful recognition included participant factors such as: white race, male gender, college education and native English language. Pictograph features that predicted success included: use of direct representation strategy, proper level of image detail, and use in familiar contexts. These results were confirmed through both the descriptive statistics and multivariable regression model analyses. As a result of testing and evaluation of successful concepts across all representation strategies, we may predict successes and identify what elements can be effectively represented. The results of these analyses will inform our lexicon development in Glyph, as well as serve as a public resource for future development of illustrated patient education materials.

Appendix A

Means of Participant Recognition Ratings

Category	N	Mean	95% CI	P value
Gender				.95
Male	67	2.9	2.7, 3.0
Female	83	2.8	2.7, 3.0
Age, year				0.21
21–29	46	2.9	2.7, 3.0
30–39	36	2.9	2.8, 3.1
40–49	31	2.7	2.5, 2.8
50–59	22	2.9	2.6, 3.1
60–69	13	2.9	2.8, 3.1
70–79	1	3.4	--
Age, year				0.97
Per one age decade increase	149	−.001	−0.06, 0.06
Race				<0.001
White	86	3.0	3.0, 3.1
Asian	18	2.5	2.3, 2.7
Other	15	2.6	2.4, 2.9
Black or African American	14	2.4	2.1, 2.8
American Indian/Alaska Native	4	2.9	2.4, 3.5
Native Hawaiian or Other Pacific Islander	3	2.6	1.3, 3.8
Race				<0.001
White	86	3.0	3.0, 3.1
Non-White	54	2.6	2.4, 2.7
Ethnicity				0.04
Hispanic	30	2.7	2.5, 2.9
Non-Hispanic	100	2.9	2.8, 3.0
Education				0.001 Linear increase p-trend < 0.001
≤4th Grade	3	3.4	2.3, 2.5
5 to 8th Grade	4	2.2	1.8, 2.6
9 to 12th Grade	25	2.7	2.5, 2.8
> 12th Grade	118	2.9	2.8, 3.0
Education				<0.001
Non-college	32	2.6	2.4, 2.7
College	118	2.9	2.8, 3.0
First Language				<0.001
English	103	3.0	2.9, 3.1
Non-English	46	2.5	2.3, 2.6